How this site is built with Jekyll and Jenkins

Introduction

Since a friend asked me about how I’m doing this - this short blogpost will describe how one can deploy a simple webpage (or a simple blog) like this webpage using the Jekyll static site generator as well as the Jenkins continuous integration tool (i.e. how it’s done for this page). Note that this is really one of the fastest and easiest hacks using Jenkins and not a sophisticated build pipeline.

What is Jekyll anyway? Jekyll is a really simple and blog aware static site generator for simple personal blogs, project or organizational sites that has been written in Ruby and is highly modifyable. In it’s most basic version it’s an engine that can parse markup languages like markdown and apply HTML templates, generate indices and handle data files. Jekyll is also the engine used by GitHub pages. Only some of the most basic features will be used during this blog post.

And Jenkins? Jenkins is one of the most popular continuous integration systems. Continuous integraion solutions are normally triggered by an external event (like a commit into a source repository, time or some external build trigger) and execute a whole build pipeline. This pipeline contains instruction on how to build software, how to run various tests and how to run deployments. Continuous integration also allows to keep track about the build process of an application and have some fixed and less error prone testing process than manually executing such stuff. Jenkins is also highly customizable and only rudimentary features of Jenkins will be used in this blog post.

I’m assuming that Jekyll and Jenkins are already up and running.

Automate Jekyll build process

First one should automate the Jekyll build process. I’m currently using a standard GNU Makefile to do the main build process. This is done mainly due to historic reasons of this page - I’ve not used Jenkins from the start on.

My makefile:

• Scans all blogposts for tags and creates tag indices as Markdown pages inside the tags folder. This is done like described earlier.
• Building the site using Jekyll
• Deploy the page using rsync

all: rebuildtags buildsite deploy

build: rebuildtags buildsite

rebuildtags:

    ./mktags.sh

buildsite:

    jekyll build
    -rm _site/Makefile
    -rm _site/*.sh

deploy:

    -rsync -av --checksum ./_site/ user@example.com:/usr/www/

.PHONY: buildsite deploy rebuildtags build

Note that I’m using only .PHONY targets here since I didn’t want to list any files required and there wasn’t that much gain in reducing the build time by preserving unchanged content. Rsync only uploads changed content anyways.

Using Jenkins to execute the build

As soon as the Makefile really works one can run the build job on a Jenkins machine. I’ve configured an own Jekyll build node but one can of course also run on the Jenkins master itself. I’ve assumed that the SSH key used during upload is present on this machine too.

The Jenkins job I’m using for this page:

• Prevents concurrent builds
• Is hosted at a private GitHub repository
• Has set a GitHub build trigger that’s automatically set by Jenkins
• Has an SSH agent configured in the Jenkins job description that allows ssh access to the target host during the gmake deploy task
• Polls the repository once a day to check for modification in case a GitHub callback has been missed.

I’m currently building using a simple Jenkins Pipeline script:

pipeline {
    agent none
    stages {
        stage('Build Jekyll') {
            agent {
                label 'freebsd && amd64 && jenkins'
            }
            environment {
                OS = 'FREEBSD'
            }
            stages {
                stage('Checkout/Download') {
                    steps {
						sh 'git reset --hard'
                        sh 'git clean -fx'
                        checkout changelog: true, poll: true, scm: [$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'CloneOption', timeout: 120]], submoduleCfg: [], userRemoteConfigs: [[url: 'git@github.com:EXAMPLE/EXAMPLE.git', credentialsId : 'JENKINS_GITHUB_SSHCREDENTIAL_ID']]]
                    }
                }
                stage('Verify signatures') {
                    steps {
                        sh 'git verify-commit HEAD'
                    }
                }
                stage('Tag indexing') {
                    steps {
                        sh 'gmake rebuildtags'
                    }
                }
                stage('Jekyll') {
                    steps {
                        sh 'gmake buildsite'
                    }
                }
                stage('Tests') {
                    parallel {
                        stage('Validation') {
                            steps {
                                sh 'gmake validate'
                            }
                        }
                        stage('Linktest')
							steps {
                            	sh 'gmake linkcheck'
							}
                        }
                    }
                }
                stage('Deploy') {
                    steps {
                        sh 'gmake deploy'
                    }
                }
            }
        }
   }
}

As one can see I’m using:

• Git Checkout:
  • Poll set to true to allow Jenkins to keep a polling and hook baseline to detect modifications.
  • Keep a changelog to see which commits influenced which builds.
  • Using SSH keys for GitHub that are stored inside the Jenkins credential store
• Before every build the signature of the HEAD commit is verified using the git verify-commit HEAD function. Note that only the head is checked using verify-commit in this case. This requires that the OpenPGP keys that are allowed to push into the repository are contained inside the keystore on the machine that’s running the build.
• After that I’m calling the different stages of the Makefile using a simple Shell call to GNU make. This is done that way out of historical reasons (one could also perform all options directly using Jenkins - using a Makefile also allows to run operations manually without any CI influence)
• The checks I’m running (validating generated output, validating links, etc.) is not shown in this blog post.

One might also decide to include the build pipeline into a Jenkinsfile included directly inside the source repository. This is a really good idea so one also adds versioning to the build pipeline - and is capable of bootstrapping the whole build and deployment process with a single action from the source repository.

This article is tagged: Programming, Web, Shellscript, Jenkins, Jekyll