Personal Programming Notes

To err is human; to debug, divine.

Jenkins World 2017

Some interesting talks in Jenkins World 2017 are summarized in this blog post.

Jenkins Pipeline on your Local Box to Reduce Cycle Time

Interesting proposal (with demo, a big plus) to resolve the following problems (added my own experience):

  • Non repeatble builds due to allocated physical slaves with reused workspaces.
    • The idea is equivalent to use Dockerized Jenkins for local job runs before running on central Jenkins.
    • My current alternative is to use containerized Jenkins slaves using Kubernetes plugin.
  • Merging of two companies (YNAP + GerritForge)
    • Not applicable. However, for large corporates, two product teams can feel like two companies.
  • Need for fast feedback
    • Same
  • One master cannot suit all: central CI not flexible to local team needs.
    • Maybe applicable for team CI and company-wide CI.
  • Scalability: central CI only handle master job runs instead of multiple feature job runs.

Reference:

default Jenkinsfile
1
2
3
4
5
6
7
8
9
10
11
12
13
14
#!/bin/env groovy

node {
    checkout scm

    echo env.JENKINS_URL
    if(env.JENKINS_URL ==~ /.*central-pipeline.*/) {
        echo "loading central"
        load "central/Jenkinsfile"
    } else {
        echo "loading local"
        load "local/Jenkinsfile"
    }
}()
local Jenkinsfile
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
{ -> docker.image('slave-sbt').inside
    {
        stage('Build') {
            sh sbt('compile')
        }
        stage('Test') {
            sh sbt('test')
        }
        stage('Deploy Locally') {
            sh sbt('docker:publishLocal')

            def dockerTag = (sh(script: "git describe --tags HEAD", returnStdout: true)).trim()
            cleanUpOldContainers(dockerTag)
            runContainer(dockerTag)

            def containerUrl = getRunningContainerURL(dockerTag)

            echo "Running on $containerUrl"
            currentBuild.setDescription("$containerUrl")


        }

        stage('Push to GitHub') {
            withCredentials([[$class: 'UsernamePasswordMultiBinding',
                              credentialsId: 'github',
                              usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD']]) {
                sh "git branch --list ${env.BRANCH_NAME} || git checkout -b ${env.BRANCH_NAME}"
                sh "git push -f https://$USERNAME:$PASSWORD@github.com/NET-A-PORTER/jenkinsworld-2017-sample-app HEAD:refs/heads/${env.BRANCH_NAME}"
            }
        }
    }
}

...
central Jenkinsfile
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
{ -> docker.image('slave-sbt').inside
    {
        stage('Build') {
                sh sbt('compile')
        }
        stage('Creating Docker container') {
                sh sbt('docker:stage')
        }
        if (env.BRANCH_NAME == "master") {
            stage('Deploy to Heroku') {
                withCredentials([[$class: 'UsernamePasswordMultiBinding',
                                  credentialsId: 'heroku',
                                  usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD']]) {
                    sh "echo 'machine git.heroku.com login $USERNAME password $PASSWORD' > ~/.netrc"
                    sh "chmod 600 ~/.netrc"
                    sh "git push -f https://git.heroku.com/jenkinsworld-2017-sample-app.git HEAD:master"
                }
            }
        }
    }
}

def sbt(target) {
    return "sbt -no-colors $target"
}

Managing Jenkins with CloudBees Jenkins Team

CloudBees Jenkins is the Enterprise version of Jenkins, as opposed to Open-Source Jenkins. This enterprise version offers the following features:

  • Beekeeper Upgrade Assistant: Plugin upgrade compatibility, Compliance report.
  • Upgrade simulations to make sure the upgrade is compatible with current jobs/plugins.

Like many other tools presented in the conference, you can do many “awesome” things IF you pay for it. The only problem is if you have the budget to pay for it.

Let’s build a Jenkins Pipeline

Basic tutorial on Jenkinsfile with Declarative syntax. Not that interesting if you are already familiar with Jenkinsfile.

Developing Pipeline Libraries Locally

If you have ever tried developing Pipeline Libraries, you may have noticed how long it takes to deploy a new version to server to discover just another syntax error. Oleg will show how to edit and test Pipeline libraries locally before committing to the repository (with Configuration-as-Code and Docker).

Securing a Jenkins Instance

See this post.

Scaling Jenkins with Kubernetes

We are already using Kubernetes plugin in production. The talk mostly recaps how to use the plugin.

Declarative Pipelines in Jenkins

New features and improvements in Declarative Jenkinsfile are announced. Those include:

  • changelog: examines the commit log messages with a regular expression.
  • changeset: the affected file paths in the changelog.

Codifying the Build and Release Process with a Jenkins Pipeline Shared Library

Almost all of the Jenkins workflows practiced at FireEye are already implemented in our Jenkins. It’s assuring to know that we are using the best practices and not way behind the pack. However, I got a glimpse of potential struggles (mostly from inertia) migrating from a legacy Jenkins system (1.x) to Jenkinsfile-based system (2.x).

Mozilla’s Declarative + Shared Libraries Setup

How Mozilla is using Declarative Pipelines and shared libraries together.

Pipelines At Scale: How Big, How Fast, How Many?

Interesting point: Each pipeline step is similar to a database transaction: each step requires an atomic record in case of Jenkins restart. Therefore, it is better to have a large enough transaction than multiple small transactions (e.g., sh step).

For comprehensive study of this topic (Jenkins performance), there are many articles/blog posts written by same author (Sam Van Oort).

JenkinsPipelineUnit: Test your Continuous Delivery Pipeline

JenkinsPipelineUnit allows unit testing the Jenkinsfile locally by mocking the Pipeline steps. In my opinion, it seems only useful for reducing the syntax mistakes when developing Jenkinsfile OR global shared library. You still have to verify functionality in a real Jenkins system. Its compatibility with Declarative syntax is also questionable, admitted by the speakers/authors. In addition, it is NOT for developling a Jenkins plugin. In that case, use JenkinsRule.

Securing a Jenkins Instance

Demo “Securing a Jenkins Instance” at Jenkins Booth in Jenkins World 2017 by Claudiu Guiman.

A set of minimum steps every Jenkins Admin should follow so his public-facing Jenkins instance doesn’t turn into a Bitcoin mine.

nginx configuration
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
server {
    listen 80;
    server_name demo-001.eastus.cloudapp.azure.com;
    location / {
        proxy_set_header        Host $host:$server_port;
        proxy_set_header        X-Real-IP $remote_addr;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header        X-Forwarded-Proto $scheme;
        proxy_pass              http://localhost:8080;
        proxy_redirect          http://localhost:8080 http://demo-001.eastus.cloudapp.azure.com;
        proxy_read_timeout      90;
    }

    # block requests to /cli
    location /cli {
        deny all;
    }

    # block requests to /login
    location ~ /login* {
        deny all;
    }
}
Running nginx
1
2
3
$ cp ~/demo/default /etc/nginx/sites-enabled/default
$ sudo service nginx restart
$ sudo ufw deny 8080

If your Jenkins is hosted in AWS/Google Cloud, you should consider setting this up.