Why JSON sucks for user configuration

In the current era of JavaScript JSON has become super common. In this article I want to make a point about it not being suitable for user configuration.

The good

Let’s start with the good parts. It’s not XML. Being born as a data interchange format from the mind of Douglas Crockford it shines for its simplicity in both the definition and easiness in parsing. That’s pretty much it. As a configuration tool the only good thing I can say is that many people are accustomed to it, and usually already know its rules.

The bad

As popular as it is the JSON format has started to be overused, especially for letting users to configure stuff (that’s the point of the article). The first thing to notice here is that JSON is not intended to be used that way, instead we can quote from the official page:

JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate.

source: http://json.org

What I’d like to highlight is that JSON is a compromise between write/readability for both humans and machines. A data interchange format can easily be almost unreadable to humans, that happens all the time with binary formats but a configuration format should instead be biased towards humans.

The ugly

But what’s wrong with JSON?

I see three things that make JSON fail as a configuration format:

Strictness

The format is really demanding. Contrary to the ordinary JavaScript literal object format (which is where it comes from) JSON mandates double quotes around keys and strict absence of commas before closed parentheses.

Lack of comments

The absence of comments is perfectly fine in a data interchange formats where the payload will probably travel on wires and needs to be as tiny as possible. In a configuration format conversely it’s plain non-sense as they can be used to help the user explaining each configuration option or for the user to explain why he chose a particular option.

Readability

While JSON is quite readable when properly formatted, this good practice is not enforced by the format itself, so it’s up to coders good will to add returns, tabs and spaces in the right places. Lucky enough they usually do.

Conclusions

JSON has become very popular thanks to JavaScript to the point that is also a super common format for user configuration. It wasn’t designed to do so and in fact it does an awful job.

Almost anything is a good alternative, from INI to CSON. The one I like the most tho is YAML (which in turn has been erroneously used as a data interchange format). Here is the YAML tagline:

YAML is a human friendly data serialization standard for all programming languages.

source: http://yaml.org

Enter ju-jist, the gist runner

Neo: Ju jitsu? I’m gonna learn Ju jitsu?

Forewords

In the last article we explored the internals of D3.js, the data visualization library.
In doing that we ended up with a number of files in a gist.

This morning I thought it’d be nice to put up the thing into one of those JavaScript fiddle sites, I looked up a bunch of them: CodePen, JSFiddle, JS Bin but none of them allowed for arbitrary extensions or loading a code from a Gist1.

I had to build my own.

The Plan

  1. load and run gists by visiting URLs in this form: http://ju-jist.herokuapp.com/USER/GIST_ID/FILE_NAME
  2. eventually add a home page that will build the right url from the GIST_ID defaulting to index.html as file name

Section 1: Rack Proxy

The first thing I did is to preparing a simple proxy rack application that would extract the gist id and
file name from the URL:

parts = %r{^/(?<user>[^/]+)/?(?<gist_id>w+)/(?<file>.*)(?:$|?(?<query>.*$))}.match(env['PATH_INFO'])
gist_id = parts[:gist_id]
file = parts[:file]
user = parts[:user]

Note here how handy are actually named Regexp groups (introduced in Ruby 1.9).

Then let’s be ol’ school and use open-uri to fetch urls:

contents = open("https://gist.githubusercontent.com/#{user}/#{gist_id}/raw/#{file}")

Pass it over to Rack:

[200, {}, [contents.read]]

And wrap everything in a rack app:

# config.ru
def call(env)
  parts = %r{^/(?<user>[^/]+)/?(?<gist_id>w+)/(?<file>.*)(?:$|?(?<query>.*$))}.match(env['PATH_INFO'])
  gist_id = parts[:gist_id]
  file = parts[:file]
  user = parts[:user]
  contents = open("https://gist.githubusercontent.com/#{user}/#{gist_id}/raw/#{file}")
  [200, {}, [contents.read]]
end

run self

Section 2: The URL builder

Next I prepared a simple form:

<!-- inside index.html -->
<form>
  <input type="text" id="user_and_gist_id">
  <input type="submit" value="open">
</form>

And then I used Opal and Native with some vanilla DOM to build the URL and
redirect the user.

# gist-runner.rb
$doc  = $$[:document]
input = $doc.querySelector('input#user_and_gist_id')
form  = $doc.querySelector('form')

form[:onsubmit] = -> submit {
  Native(submit).preventDefault
  user_and_gist_id = input[:value]
  $$[:location][:href] = "/#{user_and_gist_id}/index.html"
}

And let Rack serve static files:

use Rack::Static, urls: %w[/gist-runner.rb /favicon.ico], index: 'index.html'
run self

Conclusion

Ju-Jist is up and running, you can see the code from the last article gist live on ju-jist.

The code is available on GitHub.


  1. Actually JSFiddle has some docs for loading Gists, but I wasn’t able to make it work. CodePen and others allow for external resources, but GitHub blocks the sourcing of raw contents from Gists

Heroku Websockets

One month ago Heroku [announced websocket support](Websockets in public beta on Heroku).

If you need a pub/sub solution, Pusher remains an unbeatable alternative: its free plan is good for small apps, and you can setup it in seconds.

But if you need a classic client-server solution (i.e. to talk to each client separately) there were no chances to develop it on Heroku.

Now it should be easy enough to achieve this. You can find the official guide for ruby here.

And if you are wondering about performances, read this.