Integrating Jenkins with Github, Node.js and Slack

Having an automated environment to run builds every time a code push happens on the source code repository and broadcasting the status of the build to the team members will ensure code base sanity and help the development team avoid having cat fights over whose commit caused the nightly build to fail.

This blog will help you setup a continuous integration environment using Jenkins for a Node.js project hosted in a private Github repository with integrated Slack notifications.

Assumptions

• You have basic understanding of how git and Github works.
• Atleast rudimentary Node.js skills. If not the code is shared here and the steps to integrate it with the environment is provided below.
• Basic Ubuntu knowledge since I am installing Jenkins in an AWS ubuntu instance.

We have set the agenda, lets dive in

Installing Jenkins

In order to install and setup a Jenkins server integrated with Github, we need a linux instance(Sorry! I am not a window’s fanboy) with a public IP. So I am setting up an AWS instance running Ubuntu 14.04 LTS operating system. You can also use any VPS providers or in-house servers but just make sure you have an public ip

Note: The setup can also be done locally without a public ip but in that case instead of Github intimating Jenkins on new code pushes, Jenkins must poll Github for changes which is not efficient and has API limitations.

To begin with, ssh into the server and install the basic essential libraries and Java jdk and jre.

sudo apt-get update
sudo apt-get install build-essential
sudo apt-get install python-software-properties 
sudo apt-get install default-jre
sudo apt-get install default-jdk

After installing the basic libraries, lets install Jenkins by executing the following commands

wget -q -O — https://pkg.jenkins.io/debian/jenkins-ci.org.key | sudo apt-key add -
sudo sh -c ‘echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list’
sudo apt-get update
sudo apt-get install jenkins

The above commands, along with installing Jenkins, also creates a user with the name jenkins which executes the automated services, launches Jenkins service as a daemon up on start.

Now Jenkins server is listening on port 8080and it can accessed through <server IP>:8080. If you want to change the port number to which Jenkins is listening, edit the /etc/default/jenkins to replace the line HTTP_PORT=8080 to HTTP_PORT=<preferred port number>

Setting up Jenkins

There are a few steps to follow for setting up Jenkins for the first time. Access the Jenkins server url from your favourite browser(read as chrome) and you will be asked to provide an admin password to unlock Jenkins and this password can be found in the following location

/var/lib/jenkins/secrets/initialAdminPassword

Once you provide the password, you will asked to create the first admin user

After the admin creation, Jenkins will prompt you to install the plugins. You can either opt to install suggested plugins or select the list of plugins you need to be installed

Once the plugins are installed, we are all set to create a new Job in Jenkins.

Tada!! Jenkins is ready

Setting up Node.js codebase

The Node.js code consists of a simple express.js application listening to a port 3000 and supporting tests to validate the same. The code is shown below and can be cloned from this repo.

Commit and push the code to Github. Now we can install Node.js in the same server that is running Jenkins using these commands

curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -
sudo apt-get install -y nodejs

Alternatively, you can also setup url to the version you require. Since we will running mocha to execute the test suites, install mocha globally

sudo apt-get install -g mocha

Integrating Jenkins with Github

Our Jenkins server has been sitting idle for a while now. Prior to setting up a new job in Jenkins, let us install Github plugin for Jenkins. From the home page, click on Manage Jenkins on the left side menu. Then click on Manage Plugins

On the available tab, search for Github and install the plugin. Once the plugin installation is complete, lets create a new job

Click on the create new jobs link and on the subsequent screen select Freestyle project and enter a desired project name and click Ok.

On the next screen we must configure the project details. As this is a Github project, mark the check box GitHub project beneath the project name and provide the project url. In the Source Code Management section provide the git url . In case of private git repositories, it is necessary to provide permissions to Jenkins to clone your private repo. You can provide username and password or preferably create Personal Access Token in Github and provide the details here.

Then check the Build when a change is pushed to Github checkbox in the Build Triggers section

In order to configure the build functionalities, select the Add build step dropbox in the Build section of the configuration page. Select Execute shell and in the command text area below add the linenpm install . Lets first test the integration before running the actual test suite.

Click save and we are good to go. Now lets test if the handshake between our Jenkins server and Github is successful. On the left menu for the project, select the Build Now option and Jenkins schedules a build to run immediately.

Now that we have successfully communication with Github, lets run the test cases(remember we only added npm install in our execute script earlier). Just add mocha to the Execute Shell command box(You can do that from the configure link on the left menu of the project) and run a build. If it fails, check the Jenkins logs. To find the logs, click on Manage Jenkins from the left menu in the home page and select System log url

If the issue is that Jenkins could not find mocha, change the mocha command in the Execute Shell command box to the mocha installed in the local build folder (i.e.)

./node_modules/mocha/bin/mocha

As suggested by Simon RENOULT, it is better to run the binaries using npm rather than executing using the relative path of the executable. So you can add npm test to the build step rather than the mocha path. Also add the following in the package.json file

"scripts": {
 "test": "mocha"
 }

Run the build and once the build is successful, you can also check the logs for the confirmation.

Automating builds on new git pushes

Now we have successfully integrated Github with Jenkins and have run our test cases. But our goal is to run automated builds every time a push is made and not to run the builds manually. This also can be done by 2 ways

• Creating a web hook in Github and github pings the Jenkins server when a code push is done
• Configure Jenkins to constantly poll from Github and schedule a build when it detects a push

Annnnnnd you have guessed it right, it is better and optimal to go with the web hook option. Lets see how that can be done.

Click on the settings link on the Github repository and click on the Webhooks link from the side menu. In the webhooks page, add the Payload URL which by default is

http://<jenkins ip>:<jenkins port>/github-webhooks/
Don't forget to add the trailing slash

Select Just the push events radio button and add the webhook.

Immediately once the webhook is created, Github pings the appropriate server to check for the connectivity

If the ping is successful, we are ready to rock. Now to test this, lets make a change to the code and push the same. Once pushed, you can immediately see one more request in the webhooks page and you can notice a build being scheduled.

We have successfully scheduled a build when a push is made to repository. Our next stop is receiving slack notification with the build status.

Integrating Jenkins with Slack

For Jenkins to notify slack, we need to install Slack Notifications plugin in Jenkins. By now, you must know how to do this, so go ahead and install the plugin

On the Slack end, install the Jenkins CI app in your Slack account

Once Jenkins CI app is installed, select the channel in which you wanted the notifications to be posted and then add Jenkins integration which is facilitated by the token generated by Slack.

Once you have completed the set up at the Slack end, you need to configure Jenkins. Goto Manage Jenkins in the left menu and click on Configure System. In that page look for Global Slack Notifier Settings and add the team domain and the integration token generated by Slack in the earlier step.

Once the handshake is successful between slack and Jenkins, you can see a notification in the slack channel

The last piece of work to fit everything together is configuring our job in Jenkins for Post Build slack notification. Go to configure section the left menu within the Job page. In the configuration screen, select the Add post build action drop down in the Build section. You can now see a Slack Notifications option.

Select all the build notification options that you think will be useful for you and in the advanced options select send repeated failure notifications, if you notification for consecutive build failures.

Finally all the gazillion set up steps are done. Its time to test our continuous integration environment. Make a change to the code and push it to Github and you must see a build being initiated successfully in Jenkins and once the build is complete, you must get a notification in your Slack channel

Optionally you can modify the message that is seen in the notification. You can also use various environment variables provided by Jenkins and its various plugins to make the notifications more eventful .

If you have a code review process using Github’s pull request mechanism, it will be more resourceful to run the build for every submitted pull requests along with the commits reducing the effort of the reviewers. Jenkins provides another plugins, which makes the above to be configured conveniently. I will write a segment on this in my subsequent blogs.