Redis API caching: A simple use case

Last saturday MIKAMAI hosted an interesting meetup, the Open Source Saturday Milano.

The format for the OSS is simple, people form groups and work on contributing to existing open source projects, or start new ones.

I couldn’t resist, and pitched my idea of adding an optional layer cache to ruby-lol, an open source wrapper to the Riot Games API that I contributed writing.

I got just one person interested in the project, but we were ready to go, and nothing would have stopped us, ruby-lol needed some cache, and boy, we wanted to cache that calls!

A layer cache for an API wrapper is usually a good idea, and something most people using said wrapper have to implement anyway. Being in a love relationship with Redis, we decided to go that way.

Now, everyone does caching in Redis, but everyone does it their own way, so feel free to complain about the way we did it, and feel even freer to contribute better solutions to our problem!

Our starting point was this:

client = Lol::Client.new "my_api_key"

We wanted to change it like this:

client = Lol::Client.new "my_api_key", :redis => "redis://whatever.local:6379", :ttl => 900

Our initialization already supported an option hash, so we just had to delegate redis initialization to a new method:

    def initialize api_key, options = {}
      @api_key = api_key
      @region = options.delete(:region) || "euw"
      set_up_cache(options.delete(:redis), options.delete(:ttl))
    end

    def set_up_cache(redis_url, ttl)
      return @cached = false unless redis_url

      @ttl = ttl || 900
      @cached = true
      @redis = Redis.new :url => redis_url
    end

Don’t be scared by all those instance variables, they are all supported by proper written accessor methods!

After having changed the initialization we hit our first problem. You call the API methods like this:

summoner = client.summoner.by_name "intinig"
team = client.team.get summoner_id

client.summoner and client.team are respectively instances of SummonerRequest and TeamRequest (both subclasses of the more general Request) and they have no access to client. So we had to add the capability, when instantiating said classes, to pass them caching data.

To do that we did this:

    # Returns an options hash with cache keys
    # @return [Hash]
    def cache_store
      {
        redis:  @redis,
        ttl:    @ttl,
        cached: @cached,
      }
    end
    
    # Now we can get a new Request like this
    SummonerRequest.new(api_key, region, cache_store)

With all the supporting work done we could move into caching itself. All FooRequest classes do the dirty work with Request#perform_request, the method that gets the real data from the API with the help of httparty.

This is the pre-cache version of Request#perform_request:

    # Calls the API via HTTParty and handles errors
    # @param url [String] the url to call
    # @return [String] raw response of the call
    def perform_request url
      response = self.class.get(url)
      raise NotFound.new("404 Not Found") if response.respond_to?(:code) && response.not_found?
      raise InvalidAPIResponse.new(response["status"]["message"]) if response.is_a?(Hash) && response["status"]

      response
    end

The cached version just adds two if blocks that handle the caching logic. Please don’t bash my usage of if here 🙂

    # Calls the API via HTTParty and handles errors
    # @param url [String] the url to call
    # @return [String] raw response of the call
    def perform_request url
      if cached? && result = store.get(clean_url(url))
        return JSON.parse(result)
      end

      response = self.class.get(url)
      raise NotFound.new("404 Not Found") if response.respond_to?(:code) && response.not_found?
      raise InvalidAPIResponse.new(response["status"]["message"]) if response.is_a?(Hash) && response["status"]

      if cached?
        store.set clean_url(url), response.to_json
        store.expire url, ttl
      end

      response
    end

If you’re smart, and I am pretty sure you are, you will have some questions: what is store? Why are you passing through JSON?

Store is just a method that returns the Redis instance we passed it during initialization. It’s called store because in a future we might add support for more cache stores 🙂

JSON came in handy when we hit our second problem: httparty returns a hash, and we didn’t want to destructure it into several Redis keys. The easiest way to proceed was serializing it using JSON.

WOAH! All this wall of text for less than ten new lines of code? You bet! The point of the article was that implementing a Redis cache on top of existing code is really easy 🙂

Last, but not least: TL;DR: Implementing a Redis cache is easy, do it, do it now!

PS. Check the whole (awesomely specced) code at https://github.com/mikamai/ruby-lol

jQuery injection for profit and profit, cause fun is overrated

Working on a project for an important client (more on that in the next months), we came up with the need to inject jQuery on a page, making it available to your functions.

In addition to that we wanted to be sure it didn’t conflict with other jQueries the page was already using, and, if a new version of jQuery was already available, use that instead of injecting a new one.

This whole injection thing is not very complicated per se. You just have to be sure you’re not messing up with what’s already in the page.

Let’s walk through our solution to see how we implemented it. All the code you see here will be CoffeeScript, the javascript translation is really straightforward and won’t be provided 🙂

We start with an empty html document, something like this:

<!DOCTYPE html>
<head>
  <meta charset="utf-8" />
  <script src="jquery-injection.js"></script>
</head>
<body>
</body>

jQuery-injection.js is the compiled version of our coffeescript source.

The simplest way to inject jQuery would be something like this:

@Injector = 
    injectjQuery: ->
        script = document.createElement('script')
        script.src = "//ajax.googleapis.com/ajax/libs/jquery/2.0.3/jquery.min.js"
        head = document.getElementsByTagName('head')[0]
        
        head.appendChild(script)

We’re creating a globally available Injector object with an injectjQuery function that creates a script element, sets its src attribute and adds it to head.

This is all fine and dandy, but what if we already have jQuery? This would load jQuery twice and there would be namespace conflicts. What we want to do is check for an existing jQuery and use jQuery.noconflict() to make sure they don’t clash.

checkAndInjectjQuery: ->
    if jQuery?
      if @versionCompare(jQuery.fn.jquery, "2.0.3") < 0
        @injectjQuery(true)
      else
        @jq = window.jQuery
    else
      @injectjQuery(false)   

What’s going on in here? We check if jQuery exists, and that its version is at least 2.0.3. The version check is done via versionCompare, a coffeescript simplified port of this code I found on StackOverflow.

If jQuery exists and is sufficiently new, we just map this.jq to window.jQuery. Our local jq variable is what we’re gonna use inside our functions instead of using $.

If you paid enough attention, you will have noticed we’re calling injectjQuery with a parameter.

  injectjQuery: (saveOriginal) ->
    script = document.createElement('script')
    script.src = "http://ajax.googleapis.com/ajax/libs/jquery/2.0.3/jquery.min.js"
    head = document.getElementsByTagName('head')[0]

    if saveOriginal
      @jq = $.noConflict(true)

    head.appendChild(script)

injectjQuery(true) will assume that there’s another jQuery in the page, so it saves it (temporarily) in this.jq, and then cleans after it, making sure it won’t be there when the new jQuery loads.

Wait, did I say “when the new jquery loads”? You can bet I did! Everything we’re doing is asyncronous, so we can’t be sure that the new jQuery is already available.

Avoiding potential issues with that requires some callback magicks. First we edit injectjQuery, adding, immediately after the last line, the following code

@checkLoaded(saveOriginal)

checkLoaded is a function that keeps calling itself until jQuery is effectively loaded.

 checkLoaded: (saveOriginal) ->
    window.setTimeout( =>
      if !jQuery?
        @checkLoaded(saveOriginal)
      else
        if saveOriginal
          temp = @jq
          @jq = $.noConflict(true)
          window.jQuery = temp
          window.$ = jQuery
        else
          @jq = jQuery

        @continueInitialization()
    , 500)

It recognizes the same saveOriginal parameter we used before, restoring the original jQuery where it belongs, and leaving us with our new jQuery, available through this.jq.

continueInitializazion is the function that should handle all of your post-jquery-loaded needs.

In this gist you’ll find the complete Injector class, with some bonus console.log calls to better understand what’s going on.

Thank you for reading up to this last paragraph, and have fun injecting jQuery where you please.

6 Tips for Full Stack Open Source RubyGems Development

Call me old fashioned but up to really recently I used to write my rubygems the old way: manually. 

You know, the .gemspec, the various directories, everything.

It was kinda boring, and that’s why my contributions to the gems ecosystem became scarcer than ever.

Around last December though I wanted to start working on a new project, ruby-lol (more on it in a new article), and I wanted to have everything people expect on a modern gem:

  • continous integration badge
  • coverage badge
  • gem version badge
  • code quality badge
  • gem dependency badge

The most important thing though was minimizing the effort I had to put in to obtain all of that.

So, without further ado, here’s a list of tips to streamline your gems development:

1. bundle gem your-gem-name

This won’t be big news for anyone, but the best way to start developing a gem is calling

$ bundle gem your-gem-name

from the terminal. It will create a directory with everything you need to start coding in there. Edit the license, the readme, and few fields on the gemspec and you can start happily tapping on your keyboard.

2. use a gem version badge

Once you release your gem (hint: gem build your-gem-name.gemspec && gem push your-gem-name-version.gem) you can add a gem version badge to your readme. It’s pretty simple, open the gem page on rubygems.org (i.e. ruby-lol) and click on the badge link. 

You’ll be redirected to badge.fury.io, where you can pick your favorite choice of markup and paste the badge in your readme file. Commit, push and you’re done 🙂

3. continous integration with travis-ci.org

I am officially in love with Travis. No one can not be in love with a working and easy to use hosted continous integration service that is FREE for open source development. 

So how do we set it up? Easy. Sign in travis-ci.org with your github account and give it access to your public projects. Follow the three simple steps and you’re done.

After you have your build up and running remember to copy and add the badge markup to your readme.

4. easy code metrics with code climate

Code quality is something that usually makes people argue. I know I do a lot. Code quality metrics are even worse. I know people that swear by them, and people that think they’re worthless.

I don’t know where you are with this, but I like showing off the quality of my code, and easily finding weak spots I overlooked.

Code climate is a (pricey) service that is very good at analyzing your code and allowing you to navigate around its issues. I am a paid subscriber, but like many companies they also have a free way to check the code quality of gems.

Their add github repository page is what you’re looking for. 

As for the other services remember, when you’re done, to add the badge to your readme.

5. test coverage, some love it, some hate it

As with code metrics, test coverage is something that at least makes people talk.

I like keeping track of the coverage of my tests/specs, because knowing that everything I have written is covered by tests makes me work with a more relaxed state of mind.

Coveralls.io is another great service, free code coverage for opensource projects, easy to setup, it integrates perfectly with Travis.

Once again, setting it up is easy. Signup with github, give access to public projects, get the badge and put it in your readme.

6. keep up with your dependencies

Last but not least, Gemnasium is a service that allows you to keep track of your gem dependencies and notifies you when you become outdated.

Free for open source projects, you might be familiar with the set up process. Sign in with github, copy the badge when you’re done 🙂

Gemnasium ends the list of our list of tips. If you want, let me know what you think of it and whether there are more easy-to-set-up and useful services. Have fun!

Emacs My Way

Everyone knows Emacs is a way of life.

You start learning it the first day and you never stop learning. If you think you’ve learned it all then you’re doing it wrong.

I started reading about Emacs when a funny joke about it was “EMACS: Eight Megabytes and Constantly Swapping”, but I never thought I would eventually get into it.

About twenty years have passed before I decided to give it a try, and since then I had to spend around two full years memorizing key chords. I am a happy user now, and here’s some tips to shorten your learning path 🙂

image

I will assume you have your standard Emacs installation working. I use homebrew on OSX to install it with:

    brew install emacs --HEAD --use-git-head --cocoa --srgb

Once you do that make sure it runs, and then clone my emacs.d repo. Save it as ~/.emacs.d, launch emacs and you’re almost done 🙂

It will take a while installing several libraries and it might require you a couple of Emacs reboots to finish installing everything.

Since you’re here reading and waiting that Emacs is done doing its magic stuff let’s discuss how my setup works.

Everything is based on el-get. El-get is a kind of meta-package-manager for Emacs.

It allows you to install and manage packages from multiple sources. Package.el, Emacswiki, github, you name them!

El-get also allows you to quickly whip up your recipes for forked or newer version of packages, very useful (check ~/.emacs.d/el-get-user-recipes/autotest.rcp to check one).

El-get also has a way to neatly package custom init files for the packages you install with it (check ~/.emacs.d/el-get-user/init). 

I use el-get to install all the different packages I use in my daily routine. As of today, but it varies wildly with time, I use:

  • ag, emacs frontend to the silver_searcher, blazing fast search in your projects. AMAZING!)
  • autotest, to continously run your test suite in a buffer
  • bundler, to run bundler commands from emacs, kinda quirky for now
  • coffee-mode, for your Coffeescript needs
  • gist, very useful when there’s a snippet you want to share with the outside world right now
  • go-mode, because the cool guys program in Go now, don’t they?
  • ido-ubiquitous, extends the essential ido-mode to every type of minibuffer query
  • magit, because once you learn to use git from emacs you won’t ever go back to the terminal anymore
  • markdown-mode, for your README.md 🙂
  • paredit, when you want to hack lisp dialects you also want your parentheses not to get in your way
  • php-mode, no comment here
  • powerline, to improve the look and feel of your status line
  • rspec-mode, run specs from emacs, get back results that are hotlinked to the sources. Very useful.
  • ruby-compilation, run scripts and rake and other Ruby stuff from Emacs
  • rvm, allows you to switch ruby versions and gemsets from the editor
  • smex, similar to ido-mode, for M-x commands
  • yaml-mode, sooner or later you’ll need to edit yaml files too 🙂
  • zenburn-theme, for a color theme that improves your health

As you can see from my packages, I am mostly a Ruby developer, but you can easily switch, remove and add to the packages you use by using M-x el-get-install, or edit ~/.emacs.d/init.el.

In addition to init.el and the el-get-user folder, there are four more files in my installation:

  • functions.el, here you can find some functions I defined for general usage. I am moving most of the things you can find here in el-get-user/init
  • defaults.el sets up most of your environment. You know, defaults 🙂
  • languages.el holds configuration for programming languages major modes. I am also moving code away from here and into their el-get-user/init files
  • system.el contains information about your system. You should for sure edit the following line to match your system, or lots of things won’t work
    (if (not (getenv "TERM_PROGRAM"))
    (setenv "PATH"
            (shell-command-to-string "source $HOME/.bash_profile && printf $PATH")))

In addition to using lots of packages to help your daily endeavor, there are some defaults I put it to ease the transition from other less powerful  younger editors (I used to use Textmate).

Remember, you will get lost in Emacs at first, and you will panic. Don’t! Just remember that C-x C-c allows you to quit and try again 🙂

Once your muscle memory wraps around the key chords you need for your programming needs, you’ll be a better person, and a happier programmer.

And you’ll also love Emacs, like I do 😉

Kim Jong Un approves

Note from the editorial staff: This is the last post on dev.mikamai.com until next year! Thanks everyone for following us, on January we’ll be back with more Emacs Lisp, Ruby, Arduino, Raspberry PI, Python, Go, PHP, Prestashop, …

Instarecipe!

We’re happy, if not proud, of introducing our entry in the 2012 Rails Rumble competition: Instarecipe.

Instarecipe is a simple application that allows you to search for recipes and tries to give you results as fast as possible.

It also integrates some external social media, allowing you to post on facebook and pinterest, and watch videos, taken from youtube, on how to make your favorite recipes.

We’d love to teach you how to use it, but it’s pretty simple. Run to instarecipe.com and start searching!

The application has been coded in around 30 hours during 48rails, an Italian competition parallel to Rails Rumble.

We used Ruby, a lot of Javascript, and several graphic assets found around the Internet.

We had lots of fun building it, and hope you’ll have fun playing with it 🙂