Opal under a microscope: Object creation in Ruby and JavaScript

In this post we’ll learn how Ruby objects are mapped in JavaScript-land by the Opal compiler, how to call methods on them and how object instantiation works for both Ruby and JavaScript.

The basics: constants, methods and instance variables

The rule by which Ruby objects are compiled into JavaScript by Opal is quite simple. Constants are registered with their regular name under the window.Opal (i.e. the Opal property on the JavaScript window object). Methods are mapped to properties prefixed by a $ (dollar sign). Instance variables are just regular properties.

Example:

The following Ruby code:

class Foo
  def initialize
    @bar = 'BAR!'
  end

  def bar
    @bar
  end

  def baz
    'BAZ!'
  end
end

can be used from JavaScript in this way:

var Foo = Opal.Foo; // Constant lookup
obj = Foo.$new();   // calling method on class Foo

obj.bar             // accessing @bar      => 'BAR!
obj.$bar();         // another method call => 'BAR!'

obj.baz;            // a js property => undefined
obj.$baz();         // another method call => 'BAZ!'

As a Ruby developer you may be surprised that obj.baz returns undefined and not nil (actually window.Opal.nil) as that’s the value that you would expect to find while reading an instance variable for the first time.

What happens is that the Opal compiler will do just that, as long as you access those instance variables from Ruby code. In fact it statically analyzes the class’ code and pre-initializes to nil any instance variable for which it can find a reference.

Now that we have the basics let’s go further and put Foo.new under the microscope for both Opal and CRuby.

The birth of an object in 3 steps

By looking in detail how object instantiation is done in JavaScript by the Opal compiled code we’ll have a chance to learn how it actually works in CRuby.

The implementation of .new

At the cost of oversimplifying here’s the rough implementation of the #new method available to all classes in Ruby:

class Class
  def new(*args, &block)
    obj = allocate # allocates the memory in CRuby
    obj.send(:initialize, *args, &block) # forwards all arguments to #initialize
    return obj
  end
end

Breaking it down we see that:

  1. it calls #allocate in order to setup a raw object (in C it will also allocate the necessary memory)
  2. it forwards all arguments and block to #initialize
  3. it returns the object

Here’s how it looks in Opal (the code inside %x{} is JavaScript):

class Class
  def new(*args, &block)
    %x{
      var obj = self.$allocate();
      obj.$initialize.$$p = block;
      obj.$initialize.apply(obj, args);
      return obj;
    }
  end
end

(Class#new source)

As you can see the the only notable difference is in how the block is passed. Opal will store any block call on the method itself under the $$p property. This way blocks can’t be confused with regular arguments. Apart from that it’s clear that it’s fundamentally the same code.

Uncovering Class#allocate

The curious reader at this point is wondering what the allocate method does in JavaScript because surely it can’t manage memory. Let’s see the implementation:

class Class
  def allocate
    %x{
      var obj = new self.$$alloc;
      obj.$$id = Opal.uid();
      return obj;
    }
  end
end

(Class#allocate source)

Let’s break it down by line, but this time we’ll go in reverse order:

  1. The last line is the easiest one in which the code just returns the object: return obj.
  2. The middle one is still quite clear and seems to just assign a unique identifier to the object. That value will be the one returned by #object_id.
  3. The first and most important one is where stuff actually happens. The JavaScript new keyword is used to create an object whose constructor seem to be stored in $$alloc.

For those not very familiar with JavaScript I’ll show how objects are usually created:

// this function acts as the MyClass constructor
function MyClass() { this.foo = 'bar'; }

var obj = new MyClass;
obj.foo         // => 'bar'
obj.constructor // => MyClass

The awesome thing to me is that Opal manages to have an implementation that is really idiomatic in both Ruby and JavaScript.

3 Tricks to debug Opal code from your browser

1. Use pp

pp stands for Pretty Print and is part of the CRuby standard library. Usually what it does is just reformatting your output to make it readable.

require 'pp'
pp 10.times.map { 'hello' }

In CRuby would output:

["hello",
 "hello",
 "hello",
 "hello",
 "hello",
 "hello",
 "hello",
 "hello",
 "hello",
 "hello"]

Opal instead will just pass the object to console.log():

console.log output in Safari's console

2. Call Ruby methods from the browser console

Another good thing to keep in mind is how methods are mapped into JavaScript, the rule is really simple: $ is prefixed to the method name.

> Opal.top.$puts('hello world');
=> "hello world"

> Opal.Object.$new().$object_id();
=> 123

Once you know that, you can also learn how to call methods with and without blocks:

> Opal.send(Opal.top, 'puts', 'hello world');
=> "hello world"

> var object = Opal.send(Object, 'new');
> Opal.send(object, 'object_id');
=> 123

> Opal.block_send([1,2,3], 'map', function(n) { return n*2 });
=> [2,4,6]

> Opal.block_send([1,2,3], 'map', function(n) { return Opal.send(n, '*', 2); });
=> [2,4,6]

3. Inspecting instance variables

Last but not least remember that instance variables are mapped to simple unprefixed properties of the JavaScript object.

Opal code:

class Person
  def initialize(name)
    @name = name
  end
end

In the console (with JavaScript):

> var person = Opal.Person.$new('Pippo');
=> #<Person:123 @name="Pippo">

> person.name
=> "Pippo"

Conclusion

That’s all for now, happy hacking!

Dead simple view layer with Opal and jQuery

Dead simple view layer with Opal and jQuery

Lately I noticed that I started to port from project to project a simple class that implements a basic view layer with Opal and jQuery.

In this post we will rebuild it from scratch, you can consider it as an introduction for both Opal and Ruby, and maybe also a bit of OOP

The View class

If you still generate your HTML on the server (making happy search engines) and progressively add functionality via CSS and JavaScript then this class is probably a good fit.

Let’s start by defining our API. We want an object that:

  • takes charge of a piece of HTML
  • can setup listeners and handlers for events on that HTML node
  • can be easily composed with other objects
  • exposes behavior, hiding implementation

Step 1: An object, representing a piece of HTML

Say we have this HTML, representing a search bar:

<section class="search">
  <form>
    <input type="search" placeholder="Type here"></input>
    <input type="submit" value="Search">
  </form>
</section>

This is how we want to instantiate our view:

Document.ready? do
  element = Element.find('section.search')
  search_bar = SearchBar.new(element)
  search_bar.setup
end

The SearchBar class can look like this:

class SearchBar
  def initialize(element)
    @element = element
  end

  attr_reader :element

  def setup
    # do your thing here…
  end
end

Step 2: Adding behavior

Now that we have a place let’s add some behavior. For example we want to clear the field if the ESC key is pressed and to block the submission if the field is empty. We need to concentrate on the #setup method.

feature 1: “clear the field on ESC”

<pre><code class="ruby">class SearchBar

# …

ESC_KEY = 27

def setup
element.on :keypress do |event|
clear if event.key_code == ESC_KEY
end
end

private

def clear
input.value = ”
end

def input
@input ||= element.find(‘input’)
end
end

As you may have noted we’re memoizing #input and we’re searching the element inside our current HTML subtree. The latter is quite important, especially if you’re coming from jQuery and used to $-search everything every time. Sticking to this convention will avoid down the road those nasty bugs caused by selector ambiguities.

feature 2: “prevent submit if the field’s empty”

<pre><code class="ruby">class SearchBar

# …

def setup
element.on :keypress do |event|
clear if event.key_code == ESC_KEY
end

form.on :submit do |event|
  event.prevent_default if input.value.empty?
end

end

private

def form
@form ||= element.find(‘form’)
end
end

YAY! Sounds like we’re almost done!

Step 3: Extracting the View class

Now seems a good time to extract our view layer, thus leaving the SearchBar class with just the business logic:

class View
  def initialize(element)
    @element = element
  end

  attr_reader :element

  def setup
    # noop, implementation in subclasses
  end
end


class SearchBar < View
  def setup
    # …
  end

  private

  # …
end

Step 4: Topping with some “Usability”

Now that the View class has come to life we can add some sugar to make out lives easier.

Default Selector

We already know that the class will always stick to some specific HTML and its selector, it’s a good thing then to have some sensible defaults while still allowing to customize. We’ll define a default selector at the class definition, this way:

class SearchBar < View
  self.selector = 'section.search'
end

Document.ready? { SearchBar.new.setup }

The implementation looks like this:

class View
  class << self
    attr_accessor :selector
  end

  def initialize(options)
    parent   = options[:parent] || Element
    @element = options[:element] || parent.find(self.class.selector)
  end
end

ActiveRecord style creation

I’d also like to get rid of that Document.ready? that pops up every time. As always let’s define the API first:

class SearchBar < View
  self.selector = 'section.search'
end

SearchBar.create

And then the implementation

class View
  # …

  def self.create(*args)
    Document.ready? { create!(*args) }
    nil
  end

  def self.create!(*args)
    instance = new(*args)
    if instance.exist?
      instances << instance
      instance.setup
    end
    instance
  end

  def exist?
    element.any?
  end

  def self.instances
    @instances ||= []
  end
end

While there we also added a check on element existence and a list of active instances so that we can play nice with JS garbage collection and instantiate the class even if the HTML it needs is missing (e.g. we’re on a different page).

Conclusion

Hope you enjoyed and maybe learned a couple of things about Opal and opal-jquery, below are some links for you:

The full implementation is available in this gist: https://gist.github.com/elia/50a723e6133a645b4858

Opal’s website: http://opalrb.org
Opal API documentation: http://opalrb.org/docs/api
Opal-jQuery API documentation: http://opal.github.io/opal-jquery/doc/master/api
jQuery documentation: http://jqapi.com (unofficial but handy)
Vienna, a complete MVC framework: https://github.com/opal/vienna (unofficial but handy)

Enter ju-jist, the gist runner

Neo: Ju jitsu? I’m gonna learn Ju jitsu?

Forewords

In the last article we explored the internals of D3.js, the data visualization library.
In doing that we ended up with a number of files in a gist.

This morning I thought it’d be nice to put up the thing into one of those JavaScript fiddle sites, I looked up a bunch of them: CodePen, JSFiddle, JS Bin but none of them allowed for arbitrary extensions or loading a code from a Gist1.

I had to build my own.

The Plan

  1. load and run gists by visiting URLs in this form: http://ju-jist.herokuapp.com/USER/GIST_ID/FILE_NAME
  2. eventually add a home page that will build the right url from the GIST_ID defaulting to index.html as file name

Section 1: Rack Proxy

The first thing I did is to preparing a simple proxy rack application that would extract the gist id and
file name from the URL:

parts = %r{^/(?<user>[^/]+)/?(?<gist_id>w+)/(?<file>.*)(?:$|?(?<query>.*$))}.match(env['PATH_INFO'])
gist_id = parts[:gist_id]
file = parts[:file]
user = parts[:user]

Note here how handy are actually named Regexp groups (introduced in Ruby 1.9).

Then let’s be ol’ school and use open-uri to fetch urls:

contents = open("https://gist.githubusercontent.com/#{user}/#{gist_id}/raw/#{file}")

Pass it over to Rack:

[200, {}, [contents.read]]

And wrap everything in a rack app:

# config.ru
def call(env)
  parts = %r{^/(?<user>[^/]+)/?(?<gist_id>w+)/(?<file>.*)(?:$|?(?<query>.*$))}.match(env['PATH_INFO'])
  gist_id = parts[:gist_id]
  file = parts[:file]
  user = parts[:user]
  contents = open("https://gist.githubusercontent.com/#{user}/#{gist_id}/raw/#{file}")
  [200, {}, [contents.read]]
end

run self

Section 2: The URL builder

Next I prepared a simple form:

<!-- inside index.html -->
<form>
  <input type="text" id="user_and_gist_id">
  <input type="submit" value="open">
</form>

And then I used Opal and Native with some vanilla DOM to build the URL and
redirect the user.

# gist-runner.rb
$doc  = $$[:document]
input = $doc.querySelector('input#user_and_gist_id')
form  = $doc.querySelector('form')

form[:onsubmit] = -> submit {
  Native(submit).preventDefault
  user_and_gist_id = input[:value]
  $$[:location][:href] = "/#{user_and_gist_id}/index.html"
}

And let Rack serve static files:

use Rack::Static, urls: %w[/gist-runner.rb /favicon.ico], index: 'index.html'
run self

Conclusion

Ju-Jist is up and running, you can see the code from the last article gist live on ju-jist.

The code is available on GitHub.


  1. Actually JSFiddle has some docs for loading Gists, but I wasn’t able to make it work. CodePen and others allow for external resources, but GitHub blocks the sourcing of raw contents from Gists

Learning D3.js basics with Ruby (and Opal)

I always wanted to learn D3.js, problem is JavaScript is too awesome and that kept turning me off… till now!

Let’s take a random tutorial that we will attempt to translate into Ruby, for example this: http://bl.ocks.org/mbostock/3883245.

A couple of thing we need to bear in mind are these: D3 is not object-oriented and it’s a pretty complex library. We’re expecting some problems.

To have something to compare it to we may note, for example, that JQuery is basically a class ($) that exposes methods that returns other instances of the same class. That makes things easy enough when we try to use it from an OO language like Ruby.

Part I — The Setup

Following the tutorial, let’s setup the HTML page:

<!DOCTYPE html>
<html>
<meta charset="utf-8">
<style>
/* …uninteresting CSS from the tutorial here… */
</style>
<body>
<script src="http://d3js.org/d3.v3.min.js" charset="utf-8"></script>

<!-- OUR ADDITIONS HERE -->
<script src="http://cdn.opalrb.org/opal/current/opal.js"></script>
<script src="http://cdn.opalrb.org/opal/current/native.js"></script>
<script src="http://cdn.opalrb.org/opal/current/opal-parser.js"></script>

<script type="text/ruby" src="./app.rb"></script>

</body>
</html>

As you probably have noted we added a couple of libraries from the Opal CDN.

First we added opal.js, that is the runtime and core library that are necessary to run code compiled with Opal.

Then there’s native.js that we’ll use to interact with native objects (more details in this other post).

And last we have opal-parser.js and app.rb (that is declared as type="text/ruby"). The parser will look for all script tags marked as text/ruby and will load, parse and run them.

In additon we’re also providing a data.tsv file as described in the tutorial.

Part II — Code Ungarbling

About the process of translating

The approach I used to translate this code is quite simple, I copy/pasted all the JavaScript from the tutorial into app.rb and commented it out. Then I translated one line (or sentence) at a time tackling errors as they came.

Using Native

The first thing we need to do is to wrap the d3 global object with Native, so that is ready for Ruby consumption.

d3 = Native(`window.d3`)

Now let’s look at the first chunk of code

original code:

var parseDate = d3.time.format("%d-%b-%y").parse;

var x = d3.time.scale()
    .range([0, width]);

var y = d3.scale.linear()
    .range([height, 0]);

var xAxis = d3.svg.axis()
    .scale(x)
    .orient("bottom");

var yAxis = d3.svg.axis()
    .scale(y)
    .orient("left");

converted code

d3 = Native(window.d3)
time  = d3[:time]
scale = d3[:scale]
svg   = d3[:svg]

date_parser = time.format("%d-%b-%y")

x = time.scale.range([0, width])
y = scale.linear.range([height, 0])

x_axis = svg.axis.scale(x).orient(:bottom)
y_axis = svg.axis.scale(y).orient(:left)

By trying to run this code we discover early on that somehow Native is failing to deliver calls via method missing.

The reason is that bridged classes are a kind of their own and turns out that D3 tends to return augmented anonymous functions and arrays. Now both Array and Proc are bridged classes. That means that they are the same as their JS counterparts: Array and Function.

Native will have no effect on bridged classes (no wrapping) and therefore all calls to properties added by D3 will end in calls on undefined.

To solve this we’ll manually expose the methods on those classes, the final code looks like this:

module Native::Exposer
  def expose(*methods)
    methods.each do |name|
      define_method name do |*args,&block|
        args << block if block_given?
        `return #{self}[#{name}].apply(#{self},#{args.to_n})`
      end
    end
  end
end

Proc.extend Native::Exposer
Array.extend Native::Exposer

Proc.expose :range, :axis, :scale, :orient, :line, :x, :y, :parse, :domain
Array.expose :append, :attr, :call, :style, :text, :datum

The complete code can be found in this gist.

UPDATE: see the code in action on Ju-Jist, and read about building ju-jist in this article

A better solution

Let’s hope that in the future the Opal team (me included) will add method missing stubs1 to all bridged classes (instead of just adding them BasicObject).

Conclusion

We probably didn’t learn very much about how to use D3.js, but we discovered a bit of its internals, exposing an interesting style of JavaScript.


  1. (which is the underlying mechanism that powers method_missing support in Opal) 

Using native JavaScript objects from Opal

Question: can I call JS functions from Opal?
Answer: totally!

Opal standard lib (stdlib) includes a native module, let’s see how it works and wrap window:

require 'native'

window = Native(`window`) # equivalent to Native::Object.new(`window`)

Now what if we want to access one of its properties?

window[:location][:href]                         # => "http://dev.mikamai.com/"
window[:location][:href] = "http://mikamai.com/" # will bring you to mikamai.com

And what about methods?

window.alert('hey there!')

So let’s do something more interesting:

class << window
  # A cross-browser window close method (works in IE!)
  def close!
    %x{
      return (#@native.open('', '_self', '') && #@native.close()) ||
             (#@native.opener = null && #@native.close()) ||
             (#@native.opener = '' && #@native.close());
    }
  end

  # let's assign href directly
  def href= url
    self[:location][:href] = url
  end
end

That’s all for now, bye!

window.close!

“Use Node.js FOR SPEED” — @RoflscaleTips

I obeyed, therefore I wrote an opal NPM package.

Now I can trash Rails and opal-rails and start working on Node only!

server = Server.new 3000
server.start do
  [200, {'Content-Type' => 'text/plain'}, ["Hello World!n"]]
end

For it I had to write the Server class:

class Server
  def initialize port
    @http = `require('http')`
    @port = port
  end

  def start &block
    %x{
      this.http.createServer(function(req, res) {
        var rackResponse = (block.call(block._s, req, res));
        res.end(rackResponse[2].join(' '));
      }).listen(this.port);
    }
  end
end

Put them in app.opal and then start the server:

$ npm install -g opal      # <<< install the NPM package first!
$ opal-node app

The running app

YAY! now I can roflSCALE all of my RAils apps!!1

Cross posted from Eliæ

Stop writing JavaScript and get back to Ruby!

Cross posted from: Eliæ

Remeber that feeling that you had after having tried CoffeeScript getting back to a project with plain JavaScript and felt so constricted?

Well, after re-writing a couple of coffee classes with OpalRuby I felt exactly that way, and with 14.9KB of footprint (min+gz, jQuery is 32KB) is a complete no brainer.

So let’s gobble up our first chunk of code, that will salute from the browser console:

puts 'hi buddies!'

You wonder how that gets translated?

Fear that it will look like this?

'egin',cb='bootstrap',u='clear.cache.gif',z='content',bc='end',lb='gecko',mb='g
cko1_8',yb='gwt.hybrid',xb='gwt/standard/standard.css',E='gwt:onLoadErrorFn',B=
'gwt:onPropertyErrorFn',y='gwt:property',Db='head',qb='hosted.html?hello',Cb='h
ref',kb='ie6',ab='iframe',t='img',bb="javascript:''",zb='link',pb='loadExternal
Refs',v='meta',eb='moduleRequested',ac='moduleStartup',jb='msie',w='name',gb='o
pera',db='position:absolute;width:0;height:0;border:none',Ab='rel',ib='safari',
rb='selectingPermutation',x='startup',m='hello',Bb='stylesheet',ob='unknown',fb
='user.agent',hb='webkit';var fc=window,k=document,ec=fc.__gwtStatsEvent?functi
on(a){return fc.__gwtStatsEvent(a)}:null,zc,pc,kc,jc=l,sc={},Cc=[],yc=[],ic=[],
vc,xc;ec&&ec({moduleName:m,subSystem:x,evtGroup:cb,millis:(new Date()).getTime(
),type:nb});if(!fc.__gwt_stylesLoaded){fc.__gwt_stylesLoaded={}}if(!fc.__gwt_sc
riptsLoaded){fc.__gwt_scriptsLoaded={}}function oc(){var b=false;try{b=fc.exter

Nope! it’s Chuck Testa

(function() {
  return this.$puts("hi buddies!")
}).call(Opal.top);

But it will be obviously a mess to integrate it with existing javascript!!1

Let’s how to add #next and #prev to the Element class:

class Element
  def next(selector = '')
    element = nil
    `element = jQuery(this.el).next(selector)[0] || nil;`
    Element.new element if element
  end
end

But I’m on Rails!

Here you served:

gem 'opal-rails'

Which il provide you a requirable libraries for application.js

//= require opal
//= require rquery

and the .opal extension for your files

# app/assets/javascripts/hi-world.js.opal

puts "G'day world!"

A template handler:

# app/views/posts/show.js.opal.erb

Element.id('<%= dom_id @post %>').show

and a Haml filter!

-# app/views/posts/show.html.haml

%article= post.body

%a#show-comments Display Comments!

.comments(style="display:none;")
  - post.comments.each do |comment|
    .comment= comment.body

:opal
  Document.ready? do
    Element.id('show-comments').on :click do
      Element.find('.comments').first.show
      false # aka preventDefault
    end
  end