Geolocation, can we really trust it?

Recently our team has been working on a web app that uses geolocation. We use geolocation to identify real-world geographic location of an user and allow them to check into an event.
A fundamental requirement is the accuracy of the user location. Luckily, HTML5 geolocation API is supported from most of mobile browsers, provides an interface to query device’s location and its accuracy is quite high.

The app rewards the user for participating to located events, so we need to be sure that user geolocation is not forged.
The major problem is the high number of possibilities for forgeing geolocation, for example:

We tried to solve the problem by matching the results of HTML5 geolocation with several geolocation services such as Maxmind GeoIP and Akamai Edgescape, without success. Using Maxmind we’ve encountered problems with location accuracy. Maxmind offers an IP location service; when you query their database with IP an API returns a possible geolocation. We have found that some mobile carriers do not change the device IP address for several days, thus Maxmind database answers with the same result even if you keep traveling.

Akamai Edgescape, instead gives you the location of the pop that served your request. The limit of this method is that the pop can be near the user location or not.

In the end we can say that requiring user’s location and integrating geolocation service is quite simple; on the other hand making sure that the location is not forged is pretty hard.

Cucumber and Browser Geolocation

Several days ago I’ve started writing an application with browser geolocation support. The main problem encountered has been that of testing user interaction with the geolocation service. It is not possible to interact with browser geolocation so you need a way to mock the geolocation api in test environment. I’ve found, this old but still interesting, post wich resolve my problem.

http://lucaguidi.com/2011/02/13/html5-geolocation-testing-with-cucumber.html

Docker on your Mac using Vagrant

The first time I tried docker I used boot2docker without complete knowledge of it. The idea behind boot2docker is really quite clever: since we can’t run docker natively on Mac, we install a bare-bones Linux VM that can run docker, and then we communicate with it using a docker client running on our host.

After some initial tests I started looking for a different solution considering that I already installed Vagrant on my machine.

This guide explain how to use docker on mac using Vagrant with VirtualBox or Parallels as provider without installing boot2docker.

Prerequisites:

Vagrant installation

$ brew cask install vagrant

Provider installation

Vagrant has the ability to manage different provider such as VirtualBox, Parallels and VMware. By default it uses VirtualBox which is free, other providers need license to run.

If you want to use default provider just install VirtualBox:

$ brew cask install virtualbox

Otherwise if you have already installed Parallels on your machine:

$ vagrant plugin install vagrant-parallels

The Vagrant plugin installer will automatically download and install vagrant-parallels plugin.

Docker installation

$ brew install docker

Build and run your docker VM

Create a new directory, name it docker and create Vagrantfile:

$ mkdir docker
$ cd docker
$ touch Vagrantfile

Now edit your Vagrantfile and add:

Vagrant.require_version ">= 1.6.5"

VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = "yungsang/boot2docker"
  config.vm.network "private_network", ip: ENV['BOOT2DOCKER_IP'] || "10.211.55.5"

  config.vm.provider "parallels" do |v, override|
    override.vm.box = "parallels/boot2docker"
    override.vm.network "private_network", type: "dhcp"
  end

  # Fix busybox/udhcpc issue
  config.vm.provision :shell do |s|
    s.inline = <<-EOT
      if ! grep -qs ^nameserver /etc/resolv.conf; then
        sudo /sbin/udhcpc
      fi
      cat /etc/resolv.conf
    EOT
  end

  # Adjust datetime after suspend and resume
  config.vm.provision :shell do |s|
    s.inline = <<-EOT
      sudo /usr/local/bin/ntpclient -s -h pool.ntp.org
      date
    EOT
  end
end

Now you’re able to run your VM:

### VirtualBox
$ vagrant up

### Parallels
$ vagrant up --provider parallels

Finally, you’ll need to set an environment variable called DOCKER_HOST that will tell your Docker client on your host machine the URI for the Docker daemon running on the VM.

$ export DOCKER_HOST="tcp://`vagrant ssh-config | sed -n "s/[ ]*HostName[ ]*//gp"`:2375"

Once you’ve completed all these steps, you should have the Docker client installed and a VM capable of starting Docker containers. To test it out, try the command docker run hello-world from your host:

$ docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from hello-world
a8219747be10: Pull complete
91c95931e552: Already exists
hello-world:latest: The image you are pulling has been verified. Important: image verification is a tech preview feature and should not be relied on to provide security.
Digest: sha256:aa03e5d0d5553b4c3473e89c8619cf79df368babd18681cf5daeb82aab55838d
Status: Downloaded newer image for hello-world:latest
Hello from Docker.
...

Good Dockering! šŸ˜‰

Tip

In some case you can encounter this error “Error response from daemon: client and server don’t have same version”. To solve this problem install older or newer Docker version on your machine.

$ brew tap homebrew/versions
$ brew install homebrew/versions/docker
$ brew search docker
docker
homebrew/versions/docker133
homebrew/versions/docker141
homebrew/versions/docker150
$ brew install homebrew/versions/docker150

SparkĀ Core tips and tricks

Spark Core’s team has built an incredible WEB IDE (Spark Build) to allow you to develop your application directly from your browser. When your Spark Core is connected to the internet you can deploy your code directly from Spark Build.

Spark Build includes the Spark Apps section, which displays the name of the current app in your editor, as well as a list of your other applications and community-supported example apps.

Spark Build is a powerful tool and it’s perferct for beginners; but youā€™d surely prefer to use your beloved text editor, thus being a lot faster.

Here some tips to improve your experience:

Use Spark CLI

With Spark CLI you can interacts with your cores directly from your terminal. First make sure you have node.js installed. Then open your command prompt or terminal and enter:

npm install -g spark-cli

Organize your code

Build a basic structure for your projects. Your first applications will be a large copy/paste of code found on internet, but with the right organization you can reuse library and easily find the code you have written. For example:

In this example I’ve divided Spark Core dev folder in 3 subfolders apps, firmware, libraries.

Compile and Flash from command line

Spark CLI gives you the ability to compile your project with all the dependent library files. For example if you want to compile thermostat_relay.ino from the above example you can run:

spark compile thermostat_relay.ino ../libraries/Adafruit_DHT.cpp ../libraries/Adafruit_DHT.h 
../libraries/HttpClient.cpp ../libraries/HttpClient.h 
--saveTo ../firmware/firmware_thermostat_relay.bin 

This command takes your library files as arguments and --saveTo defines where the generated firmware will be saved.

If build succeded you can run command to flash firmware on Spark Core:

spark flash XXXXXXXXXXXX ../firmware/firmware_thermostat_relay.bin

Replace XXXXXXXXXXXX with your core id.

Serial outputs

When your Spark Core is connected to your computer through usb you can debug it using the serial port. In my experience this is a good way to start and understand what is going on under the hood on your core.

spark serial monitor /dev/cu.usbmodem1421

Keep in mind that the serial port path can change. To discover your current connected Spark Core just run:

spark serial list

Skylight, just give a try

Three months ago I’ve started using Skylight as app monitor and profiler, and I’m very happy with it. Skylight relies on running an agent that collects performance information from Rails app. Skylight has a minimal and simple interface. In my experience having a large and overwhelming amount of information and metrics can be disorienting. Tools like New Relic come with a lot of features and graphs, but someimes you can’t find the right information in the right place.

I definitely suggest trying Skylight out to profile your application.

You can try Skylight for free on Heroku. The installation is super simple and the installation screen provides real-time feedback during gem installation and configuration.

Fast and fancy spec with spring + spectator

Several months ago I’ve started using the spectator gem. Iā€™m using the Rspec testing framework with the spectator event watcher as the test runner. Spring allows us to watch the test run almost immediately after hitting save of a file in the editor.

In my experience spectator is easy to to use, you don’t need to configure anything, it just works out of the box. Tools like guard on the other hand have long list of options to run with.

Using spectator is pretty simple, just install the spectator gem gem install spectator and then run spectator in your root project. Now every time you modify a file in your project, spectator searches and launches the associated spec.

Our goal is to integrate spring in test environment in order to improve startup performance and thus speed up the entire process.

My current gemfile setup icludes:

# Gemfile

group :development, :test do
  gem 'rspec-rails', '~> 3.0'
  gem 'spring'
end

group :development
    gem 'spring-commands-rspec'
    gem 'spectator'
end

spring-commands-rspec implements the rspec command for Spring. As you can see in spring-commands-rspec readme, this gem generates a bin/rspec file like this:

# bin/rspec

#!/usr/bin/env ruby
begin
  load File.expand_path("../spring", __FILE__)
rescue LoadError
end
require 'bundler/setup'
load Gem.bin_path('rspec-core', 'rspec')

To verify that the installation of spring-commands-rspec is successful you can run ./bin/rspec and then verify that spring is running with spring status.
The first startup will take a moment. Now that spring has started the app server for you, subsequent runs will use a forked process from this server instead and return faster.

To integrate faster ./bin/rspec command in spectator just create a .spectator.rb file in your root project as above:

# .spectator.rb

ENV['RSPEC_COMMAND'] = './bin/rspec'

Have a nice and fast TDD!

API, better with spec

Rails is a perfect tool to create robust web APIs that can serve different client applications at the same time. Even if building a REST api is pretty simple you need to integrate a complete tests suite to be prepared to modify and maintain your code for future features.

In a previous post we show how to build a small api able to retrieve and store temperatures coming from an Arduino board. In the example below I want to illustrate some tecniche to spec your api controller in order to test expected behaviour.

This code explains how to test a controller shown here.

require 'rails_helper'

RSpec.describe Api::TemperaturesController, :type => :controller do
  describe 'GET #index' do
    context 'without data' do
      it 'returns no temperatures' do
        get :index
        expect(response.status).to eq 404
        expect(response.body).to eq('[]')
      end
    end

    context 'with some data' do
      before { create(:temperature) }
      it 'returns all temperatures' do
        get :index
        expect(response.status).to eq 200
        body = JSON.parse(response.body)
        expect(body).to be_kind_of(Array)

        temperature_value = body.first['value'].to_f
        expect(temperature_value).to eq 20.0
      end
    end
  end

  describe 'POST #create' do
    let(:valid_token) { ActionController::HttpAuthentication::Token.encode_credentials('valid_token') }
    let(:invalid_token) { ActionController::HttpAuthentication::Token.encode_credentials('invalid_token') }

    context 'without token' do
      it 'respond 401 unauthorized message' do
        post :create, { 'temperature': { 'value': 20 } }
        expect(response.status).to eq 401
      end
    end

    context 'with invalid token' do
      it 'respond 401 unauthorized message' do
        request.env['HTTP_AUTHORIZATION'] = invalid_token
        post :create, { 'temperature': { 'value': 20 } }
        expect(response.status).to eq 401
      end
    end

    context 'with valid token' do
      it 'respond 201 message' do
        temperature_value = '20.32'
        request.env['HTTP_AUTHORIZATION'] = valid_token
        post :create, { 'temperature': { 'value': temperature_value } }
        expect(response.status).to eq 201

        body = JSON.parse(response.body)
        expect(temperature_value).to eq temperature_value
      end
    end
  end
end

In this example we also test token based authentication. Rails offers the authenticate_with_http_token method, which automatically checks the Authorization request header for a token and passes it as an argument to the given block:

authenticate_with_http_token do |token, options|
  ...
end

In our example Api::TemperaturesController inherith from BaseController which includes authentication process.

class Api::TemperaturesController < BaseController
  before_action :authenticate, only: :create

  ...

end
class BaseController < ActionController::Base

  def authenticate
    authenticate_token || render_unauthorized
  end

  def authenticate_token
    authenticate_with_http_token do |token, options|
      token == 'valid_token'
    end
  end

  def render_unauthorized
    self.headers['WWW-Authenticate'] = 'Token realm="Application"'
    render json: 'invalid token', status: 401
  end
end

From arduino to rails through api

Some years ago I used pachube.com / cosm.com to save a stream of data generated from a sensor attached to an Arduino board. Unfortunately Ā this service is no more available, but you can build your own data stream repository with rails and heroku.

In this example we read temperatures from an Arduino sensor and then we save data via API using a rails app.

There are several guides to build REST APIs using rails. This tutorial integrates some best practices but you need to focus your work on a smaller subset, depending on your need. Rails and JSON give you the perfect combination for a fast and comprehensive developement.

Rails api controllers

# /app/api/temperatures_controller.rb

module Api
  class TemperaturesController < ActionController::Base
    protect_from_forgery
    skip_before_action :verify_authenticity_token, if: :json_request?

    def index
      temperatures = Temperature.all
      if temperatures.any?
        render json: temperatures
      else
        render json: {}, status: :not_found
      end
    end

    def show
      temperature = Temperature.find(params[:id]) rescue nil
      if temperature
        render json: temperature
      else
        render json: {}, status: :not_found
      end
    end

    def create
      temperature = Temperature.new(temperature_params)
      if temperature.save
        render json: temperature, location: api_temperature_path(temperature.id), status: :created
      else
        render json: { errors: temperature.errors }, status: :unprocessable_entity
      end
    end

    ...
    ...

    protected

    def json_request?
      request.format.json?
    end


    private

    def temperature_params
      params.require(:temperature).permit(:value)
    end
  end
end

Test locally

During your development you can test your API with curl, here are some examples for saving and retrieving JSON data feeds:

# create
curl -v loalhost:3000/api/temperatures -X POST 
     -H "Accept: application/json" 
     -H "Content-Type: application/json" 
     -d '{"temperature": {"value": 20}}'

# index
curl -v loalhost:3000/api/temperatures

# show
curl -v loalhost:3000/api/temperatures/1

Your goal now is to create a POST request from Arduino just like the one generated with curl.

Arduino post request

To send data to a rails app you need an Anduino Ethernet Shield + Arduino or an Arduino Yun. In this example we are using an Anduino Ethernet Shield and we run on it this code:

#include <SPI.h>
#include <Ethernet.h>

// assign a MAC address for the ethernet controller.
// Newer Ethernet shields have a MAC address printed on a sticker on the shield
// fill in your address here:
byte mac[] = {0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED};

// fill in an available IP address on your network here,
// for manual configuration:
IPAddress ip(192,168,1,10);
// initialize the library instance:
EthernetClient client;

char server[] = "tantiauguriatutti.herokuapp.com";   

unsigned long lastConnectionTime = 0;       
boolean lastConnected = false;              
const unsigned long postingInterval = 60000;

float temperature;  
int reading;  
int lm35Pin = 5;
float referenceVoltage;

void setup() {
  // Open serial communications and wait for port to open:
  Serial.begin(9600);

  analogReference(INTERNAL);
  referenceVoltage = 1.1;

  // start the Ethernet connection:
  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    // DHCP failed, so use a fixed IP address:
    Ethernet.begin(mac, ip);
  }
}

void loop() {
  // if there's incoming data from the net connection.
  // send it out the serial port.  This is for debugging
  // purposes only:
  if (client.available()) {
    char c = client.read();
    Serial.print(c);
  }

  // if there's no net connection, but there was one last time
  // through the loop, then stop the client:
  if (!client.connected() && lastConnected) {
    Serial.println();
    Serial.println("disconnecting.");
    client.stop();
  }

  // if you're not connected, and ten seconds have passed since
  // your last connection, then connect again and send data:
  if(!client.connected() && (millis() - lastConnectionTime > postingInterval)) {
    reading = analogRead(lm35Pin);
    temperature = (referenceVoltage * reading) / 1023;
    sendData(temperature);
  }
  // store the state of the connection for next time through
  // the loop:
  lastConnected = client.connected();
}

// this method makes a HTTP connection to the server:
void sendData(int thisData) {
  // if there's a successful connection:
  String JsonData = "{"temperature": {"value": ";
  JsonData = JsonData + thisData;
  JsonData = JsonData + "}}";
  if (client.connect(server, 80)) {
    Serial.println("connecting...");
    // send the HTTP PUT request:
    client.println("POST /api/temperatures HTTP/1.1");
    client.println("Host: tantiauguriatutti.herokuapp.com");
    client.println("User-Agent: Arduino/1.0");
    client.println("Accept: application/json");
    client.print("Content-Length: ");
    client.println(JsonData.length());

    // last pieces of the HTTP PUT request:
    client.println("Content-Type: application/json");
    client.println("Connection: close");
    client.println();
    client.println(JsonData);
  } 
  else {
    // if you couldn't make a connection:
    Serial.println("connection failed");
    Serial.println();
    Serial.println("disconnecting.");
    client.stop();
  }
   // note the time that the connection was made or attempted:
  lastConnectionTime = millis();
}

The example above shows how to manage ethernet configuration, transform reading from analog input sensor and how to send JSON data to our Rails app.

Keep in mind it is possible to use another board to collect and send data, for example Spark Core and Electric Imp are perfect beacause already integrated with WiFi module.