Emulating the Node.js module system in Ruby

One of my favourite features of Node.js is the Module system.

var arDrone = require('ar-drone');
var client  = arDrone.createClient();

client.takeoff();

There are a few differences in how require works in node compared to ruby's require that make it quite nice to work with.

The main one being that require returns an object/function/value which you then use in your code. In Ruby, when you call require, all the code from the required file gets loaded into the global name space.

Node's require:

var Lanyrd = require('Lanyrd')=

Ruby's require:

require('lanyrd')

Lanyrd # module available everywhere

There are pros and cons to each approach, but one big drawback to ruby's approach is that you can't load two different modules called Lanyrd at the same time as the second will clobber the first, overwriting any shared method names.

This only becomes a problem when you have dependencies of dependencies that require two different versions of a rubygem.

When you require a gem, that can then go off and require more gems that it depends on, but each one is required into the global namespace of the whole program.

To get around this issue we have bundler, which checks the whole dependency graph of your application for clashes and stops you ever starting the program if there are two dependencies that require different versions of the same library.

With Node when you require a module, it returns an object or function to work with which you assign to a variable, this object is declared within the module as exports.

In pi.js:

var PI = Math.PI;

exports.area = function (r) {
  return PI * r * r;
};

exports.circumference = function (r) {
  return 2 * PI * r;
};

And then loading it:

var Pi = require('./pi.js')

Pi.area(2)

What this means is requiring modules never pollute the global scope, and when modules require other modules, the code is completely contained and cannot clash with other modules requiring different versions of the same dependencies.

This goes a long way to avoiding potential dependency hell, where you can't use certain libraries with other particular libraries because they have clashing dependencies.

In node they never see each other where as in ruby land, all dependencies end up in the global namespace.

Could this work in ruby?

So I started investigating ways that this pattern could be replicated in ruby, the result is module_import.

require in ruby essentially does a couple of things, it searches the load path of the application and gem paths of the system to find a named library or path to a ruby file.

Then it evals that code within the global namespace (but actually much nicer than I've described it).

A module_import module would look like this:

module Pi
  def self.area(radius)
    Math::PI*radius*radius
  end
end

EXPORTS = Pi

The only difference is the EXPORTS constant at the bottom which is set to the object or module to be returned.

So only one thing can be returned, but you can always nest more modules inside the top level namespace.

You would then use this module by calling import rather than require:

require('module_import')
MyPi = ModuleImport.import('./pi.rb')
MyPi.area(2)

This means you can require a different version of the Pi module as a different name and they won't clash.

Where is really shines is with nested dependencies, the only objects that end up in the global namespace of your application are the ones you explicitly require, rather than everything down the dependency chain.

Drawbacks and tradeoffs

As with most things in life, this approach comes with some drawbacks.

For one thing, the current implementation is quite naive:

module ModuleImport
  def self.import(path)
    self.send(:remove_const, "Import") if self.const_defined?('Import')
    imported_module = self.const_set "Import", Module.new

    code = File.read(path)
    imported_module.module_eval(code)

    unless imported_module.const_defined?('EXPORTS')
      raise "File at #{path} doesn't export anything, use EXPORTS"
    end

    return imported_module::EXPORTS
  end
end

We're basically using module_eval inside of an empty module, which is a lot slower than require, it also won't work with C extensions.

It is backwards compatible with existing rubygems but if any imported library calls require that still ends up in the global namespace.

It also doesn't stop you from breaking out of that module and putting things the global name space, you can always get back to the global namespace by using :: in front of a class or module name.

It also makes monkey patching and duck punching more difficult to do as you don't know where to find the module you wish to change, so it pushes you towards doing more dependency injection, which may or may not enjoy.

Conclusion

As a fun experiment module_import works quite well, giving you similar functionality to the node module system, potentially saving you from dependency hell.

But it's nowhere near production ready, its completely untested and likely very slow for lots of imports.

I really wanted to start a conversation around how we handle dependencies and if there could be a better way.

There have been a lot of ideas from the ruby world taken over into node, and there is no reason why we can't take some hints from some of their best bits too.

Update

Unsurprisingly, I'm not the first person to have this idea, Michel Martens had a similar idea a few years ago: Cargo

The implementation is a bit cleaner, it uses Kernel.load which loads the file completely separately and returns only the exported module.

Discussion on Hacker News