erlang_js - Awesome

erlang_js

If you haven't heard, the Basho team released erlang_js today. Its a linked in driver that provides a Spidermonkey JavaScript context to run JS code for Erlang. This is interesting to me because it avoids the stdio overhead incurred by the current Map/Reduce system that CouchDB uses. So I did what any bored hacker would do: threw erlang_js into the CouchDB build system and hacked the view generation code to use the in-VM contexts.

Numbers

These times are for the "mega view" reported in seconds from raindrop-perf.py found here.

Code

Look here.

Next Up

The communication between Erlang and JS is unnecessarily converting Erlang -> JSON -> Spidermonkey Objects. I've written the code to go from the external Erlang representation to Spidermonkey objects directly so I plan on integrating that in the next couple days to see how these numbers change.

Code Might be Nice

Just thought that maybe people would be interested in the code that's used to talk to erlang_js. Its pretty straight forward, though not very elegant on my side.

% From couch_query_servers.erl
start_doc_map(_Lang, Functions) ->
    {ok, Port} = js_driver:new(),
    ok = js_driver:define_js(
        Port, <<"map_support.js">>, map_support(), 5000
    ),
    lists:foreach(fun(FuncSource) ->
        Source = <<"map_funs.push(", FuncSource/binary, ");">>,
        ok = js_driver:define_js(Port, Source)
    end, Functions),
    {ok, Port}.

map_docs(Port, Docs) ->
    Results = lists:map(
        fun(Doc) ->
            Json = couch_doc:to_json_obj(Doc, []),
            {ok, Results} = js:call(Port, <<"map_doc">>, [Json]),
            lists:map(
                fun(FunRs) ->
                    [list_to_tuple(FunResult) || FunResult <- FunRs]
                end,
            Results)
        end,
        Docs),
    {ok, Results}.

stop_doc_map(nil) ->
    ok;
stop_doc_map(Port) ->
    js_driver:destroy(Port).
% EOF

And map_support.js I wrote to avoid having to think to hard on the Erlang side of things:

// src/couchdb/priv/map_support.js
var map_funs = [];
var results = [];

var emit = function(key, value) {
  results.push([key, value]);
};

var map_doc = function(doc) {
  var ret = [];
  map_funs.forEach(function(func) {
    results = [];
    func(doc);
    ret.push(results);
  });
  return ret;
};
// EOF

Shoutout

Go, Basho, Go!