Jenkins and Docker on OpsWorks

Hi guys! Today I want to talk about a WIP task. So keep that in mind as you read this article, thanks 😀

I have to set up a system with Jenkins that runs its builds inside separate Docker containers.

First of all, I have to set up this environment on OpsWorks, so to keep it simple, for now, I do everything on a single instance that takes care of all the tasks.

Gathering some recipes

To archive this objective I need a couple of recipes from different cookbooks, in detail:

  • Docker Cookbook from this I use docker::install obviously to install docker, and docker::registries to login into a remote docker register such as Quay.io.
  • Jenkins Cookbook from this I use only jenkins::master to have a working jenkins installation for a master node.

In order to use these recipes I have created a custom cookbook that you can find here: My Cookbook

Inside the repo you can find also an override of a configuration template file. I need this ovveride only for defining the Jenkins user from the stack settings.

Setup

First of all I’ve created a Stack on OpsWorks and added a custom layer.
In the “Custom Chef Recipes” section I have used My Cookbook as repository URL an then in the setup section I’ve added:

docker::install docker::registries jenkins::master

Before adding an instance to this layer I have created an EBS volume mounted on /vol/jenkins in this way I can keep my Jenkins data across machines creation and destruction cycle.

In order to do this I have to tell Jenkins where to save its data and I do this through Stack setting “Custom JSON”:

{
  “jenkins”: {
    “master”: {
      “home”: “/vol/jenkins”,
      “user”: “root”,
      “port”: “80”
    }
  }
}

I have also defined which user runs Jenkins and on what port. For now, to keep it simple, the user is root and the port is 80. I know, I should have created a jenkins user and set the right permissions. I could also run Jenkins on a different port and expose it through NGINX and proxy; but for now this works.

Now I add an instance to the layer and wait for the setup.

Config Jenkins

Once the instance setup is finished I have a machine with Docker and Jenkins up and running. Now it’s the time to configure Jenkins and allow it to use Docker.

To do this I just follow the “Jenkins – Docker Plugin” documentation, with a little customization for the Docker slave starting from evarga/jenkins-slave. I allowed the root user permission to log in via ssh.

However at the time of this article the plugin documentation misses a little step which, unfortunately is essential for my configuration. Let me show you with an example.

I create a basic Jenkins job that runs a simple uname -a. After the setup of the Docker plugin I check the fields to run this job inside a Docker container:

image

But building this job, I receive:

Linux nemesis . . . GNU/Linux

Where nemesis is the name of my instance on OpsWorks, so the command is executed on the master node and not in a Docker slave container.

I have found out I can solve this problem by defining a “Label Expression” to restrict where my job can run. Now if I insert the label of my docker container docker-slave-test.

image

Building it again, now I receive:

Linux 55310b7d848c . . . GNU/Linux

Where 55310b7d848c is the ID of the Docker container that Jenkins has started and uses as slave.

So at last I have a machine with Jenkins that can run its builds on an on-the-fly created slave inside a Docker container.

The are some further steps to do before it can be considered complete, but for now it works and helps us with our CI process!

Bye and see you soon!

Leave a Reply

wpDiscuz