How to Expedite Continuous Testing

How to Expedite Continuous Testing

Using a combination of Jenkins and Circle CI, plus some Ruby and GitHub know-how, will make for a smooth process.

The prescription for continuous testing is deceptively simple: Automated unit tests, version control and a continuous integration server. We wanted to first get past the buzzwords and talk about how to actually do it, by getting started with Ruby and GitHub.

Today, we'll install Jenkins and CircleCI, hook them to our GitHub repository and get continuous automated build/inspect going, step by step. Along the way we'll talk about some common setup problems; you may want to read the whole article first, then try to follow the step-by-step instructions.

[ Commentary: Continuous Delivery to Speed Software Delivery, Reduce Stress ]

Once installed, the Jenkins service will be administered through a Web browser, where we'll point to a version control system (it supports many) and a build script. The instructions below are for Linux and Mac; for PC installation instructions, follow the directions in the final section of this article.

Installing Jenkins for Linux and Mac

Start by downloading and install Jenkins on OSX or Linux. After running the install, you'll find Jenkins configurations to start and stop it as a service.

On Linux and Mac, run these commands to start and stop Jenkins, respectively:

sudo launchctl load /Library/LaunchDaemons/org.jenkins-ci.plist

sudo launchctl unload /Library/LaunchDaemons/org.jenkins-ci.plist

Windows users will start and stop Jenkins using services in the control panel; it may be under "System Tools."

Once Jenkins is running, you can get to it by typing http://localhost:8080 into a browser; enter your IP address if you know it. If something else is running on port 8080, edit the file org.jenkins-ci.plist and restart Jenkins to change the port. Most companies put their CI server on a "named"subdomain server and run over port 80, making the URL something like

Git and GitHub Security Setup

Secure Shell (SSH) keys provide authentication. You'll use them to register your account with GitHub and, later, to prove who you are when adding and committing. If you haven't already, generate SSH keys for your operating system. Once you've generated SSH keys and registered them with GitHub, grab an OAuth Token per GitHub's help system. Save this key string in a safe place; you'll need it in a second.

Right now, Jenkins is an open server; anyone with the URL can do anything with it. Before configuring Jenkins, set up security. Otherwise anyone on the Internet can kick off builds and do unsavory things with your code.

Go to Manage Jenkins -> Configure Global Security and click "Enable Security." This will enable Jenkins to create its own user database, allowing users to sign up. Logged-in users can do anything. Make sure you allow user registration so you can register yourself, and feel free to disable the registration after you have created your user.

[ Feature: Programmers Pick 7 Great GitHub Integrations ]

Install Jenkins Plugins

Next, it's time to install some plugins. You'll need GitHub to get our code, the Ruby Version Manager (RVM) and Green Balls, which changes the color of a good build to green.

Visit the Manage Jenkins -> Manage Plugins then click the "Available"tab. Search for "GitHub Plugin" and check the box. Do the same for RVM and Green Balls.

Now you'll need to hook Jenkins to GitHub. Time to use that OAuth token. Go to Manage Jenkins -> Configure System. Scroll down until you see the Git Plugin, and set it to have your email and full name (which you put the key into at the beginning of this article).

Keep scrolling until you find the GitHub Plugin. Set it to "Let Jenkins auto-manage hook URLs," along with your GitHub username and the OAuth Token you have saved.

Create a Project in Jenkins

Go to the Jenkins home screen and click New Project. Name the project something appropriate and enter the GitHub URL.

Now, connect to the actual Git repository. Scroll to Source Code Management, select Git and enter the repository URL. Since you've set up the Jenkins user to have access via an SSH key, I recommend adding the Jenkins user credentials so you can use it for both public and private repositories. Finally, click Add under the credentials box.

Select the Jenkins user credentials in the credential dropdown. It should look like this:

You also need to decide what will trigger a new build -- and, on Linux or Mac, set everything to run in an RVM managed environment and to use Ruby 2.1.1. In the example below, the programmer setup a special collection of libraries, called a gemset, to use ruby 2.1.1. To do this from the command line, use the following code:

'rvm use 2.1.1@factory --create'

On Windows, don't use RVM. Instead, poll SCM, with a schedule of "* * * * *". (Don't enter the quotes; this will set up Windows to check version control every minute.) To slow down polling, research the parameters for a cron tab.

You're almost there. You just need to add your build step. Add Build Step and use the Execute Shell option. The script you need to enter is below. (On Windows, you can skip the first line.

#!/bin/bash -x

ruby -c factory.rb

ruby -c factory_multi3.rb

cd test

ruby run_all.rb

If everything works as expected, you should see a lovely Green Ball of success. This, of course, assumes all test cases passed. Otherwise, Red Balls of shame and failure will be prominently displayed. The Console Output will show your errors, which can be very helpful in diagnosing the issue.

Overcoming Common Jenkins Setup Problems

If you encounter an error on build, don't lose heart. Different versions of operating systems, Ruby, Git and libraries make this rather common. Instead, click on the build details, then Console Output, and try to work the problem backwards. Here are a few common ones.

The 2.1.1@factory doesn't exist.You may need to create it from the command line with a command such as this:

rvm use 2.1.1@factory --create

RVM fails to create the correct environment.Jenkins runs as its own user and may need permissions when invoking RVM. On Linux, add permissions for the Jenkins user. On Mac, the command line is this:

sudo dseditgroup -o edit -a jenkins -t user admin.

If that doesn't work, you can turn off the RVM plugin; the application will use whatever version of Ruby is installed first in your path.

The version of Ruby isn't compatible with minitest.If you can't get Ruby 2.1.1 to run, you may get an error about an uninitialized constant in run_all.rb that traces to '<top (required)>' in factory_test.rb. Fix that by changing 'class TestFactory < MiniTest::Test' to 'class TestFactory < MiniTest::Unit::TestCase.'

There are plenty of other problems to work through. If errors on the Console Output suggest that look one system isn't building properly, try cutting and pasting the error message, as you can often find answers on If you get really stuck, reach out to a Ruby expert; he or she is likely to notice a simple missing checkbox. You'll get there.

Putting Continuous Integration in the Cloud

By now, you probably agree that keeping track of versions, permissions and users turns out to cause a great deal of pain. Wouldn't it be nice if someone else hosted the continuous integration? If you could just point it to a GitHub account and a build script, and let someone else could do the building?

[ Case Study: Continuous Deployment Done In Unique Fashion at ]

As one last project, we'll use Circle CI that does just that. Go to the site, register with your GitHub account and follow the steps. Add your GitHub repository. When the initial build kicks off, CircleCI attempts to automatically figure out what framework and tests should be run. You will get an error for our example factory app ...

This is a simple enough fix. Add a new file to the root of the project called circle.yml. The contents should be as follows:



- ruby -c factory.rb

- ruby -c factory_multi3.rb

- ruby test/run_all.rb

Note: This can also be done by going to the Project Settings -> Test Commands and entering the three commands into the Test Commands text box.

After committing this to the repository and pushing it to GitHub, I didn't get a build; it took five whole minutes to realize that I'd mistyped "circle.yml" as "circle.yaml" in Git to rename and resend it.

Total time to set up Jenkins: More than four hours.

Total time to set up and run Circle CI: Less than 20 minutes.

There are plenty of reasons to stick with a local server: The data, the builds and everything else remains internal, under complete control of the organization, from the version of the Ruby interpreter to the exact gems that are installed.

But 20 minutes compared to four hours?That's something to think about.

Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.

Join the newsletter!


Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

Tags softwareapplication developmentGitHubDevelopment toolsCircle

More about LinuxRVMSpeedSSH

Show Comments