Automate build, test and deploy of a static Jekyll site
Over the last months my interest in code integration and deployment increased a lot. As a consequence I tried to automate everything around this site.
In this post I want to provide a complete tutorial on how to automatically build, test, integrate and deploy a Jekyll site to a FTP server. I describe how to use Bundle and Rake to create a virtual Ruby environment and how to use GitLab CI to execute the automation tasks.
In the past I developed the blog on my local machine, using jekyll serve to preview changes I made. If i had a state where I was confident that it is good enough to be published at ayeks.de, i pushed the files manually in Filezilla. Needless to say that this procedure takes some time. That was one of the reasons why I published so few posts in the past - the effort to publish changes was just to high.
Enough motivation to integrate everything in a pipeline. What is the current setup?
- local GIT repository for development
- Gitlab GIT repository as central repository
- Gitlab CI Pipelines for build, test and deploy
- ayeks.de Web and FTP server
Using a Rakefile for a Jekyll project
Just like the gitlab-ci file does task automation in GitLab, a Rakefile automates Ruby tasks. Jekyll is written in Ruby, so why don’t use it to automate it.
We will use the Rakefile as middleman between the GitLab CI script and the Jekyll codebase. We will later call Rake tasks from our command line and the gitlab-ci.yml file.
Rake and Bundle
A rakefile is code written in Ruby. We define tasks which can be called. To make everything as reproduce able as possible we will use bundle as our virtual environment for Ruby. In order to use the bundle you have to install it (eg. apt-get install bundler).
Install Rubygems with the Gemfile and Bundle
Then create a Gemfile where you list all your dependencies for the project. My Gemfile looks like that:
In the first two lines I define UTF8 encoding because of some strange ASCII errors which I get otherwise.. The third line defines the source for the Ruby gems. The rest of the gems are used by Jekyll and the tests. Also include rake to be able to execute the Rakefile tasks.
Install the Ruby gems with bundle install
Create the Rakefile for Jekyll
Now we are able to execute our virtual environment with bundle. Create a file calls Rakefile. I will go through the file step by step, have a look here our scroll down for the complete file.
First, we want to test if we can serve the Jekyll site with Rake. Insert the following into the Rakefile:
We import the Jekyll gem and create the task serve. In the task we will call the Jekyll serve command. For more information on the Rakefile have a look at the documentation. To call this task type into the command line: bundle exec rake serve
Sidenote: On my machine the live reload through rake wont work, so I call bundle exec jekyll s for the live reload. The benefit of using bundle for everything Ruby related is, that you don’t have to install the Gems with apt-get. Just use bundle for that.
The clean task removes generated files. The build task depends on the clean task, which will be executed before running the build script. It generates the static files and stores them to _site.
The test_html task uses HTMLProofer which checks the HTML syntax, validates the links and images.
The task test_structure tries to access a generated post. If the file is not accessible to directory structure of the posts has changed. That is bad because all external links would be broken. I don’t want that happen - therefore this check should always be true.
Thats the complete Rakefile of my project. For local testing I simply call bundle exec rake ci which builds and tests the Jekyll project:
So now we are able to reproduce the environment for all the Ruby stuff with bundle and rake. Now we can call these tasks in GitLab CI.
GitLab CI Security preparations
For the deploy-to-FTP-part we will use GitLabs secret variables. In its documentation they state:
CAUTION: Important: Be aware that secret variables are not masked, and their values can be shown in the job logs if explicitly asked to do so. If your project is public or internal, you can set the pipelines private from your project’s Pipelines settings. Follow the discussion in issue #13784 for masking the secret variables.
We do not want to our secret variables to become public. Therefore, before anything else, disable public pipelines:
If you save credentials in your GitLab repository, everyone that has access to your GitLab account can read those credentials. Therefore I strongly recommend to activate GitLabs 2-Factor Authentication in your profile settings:
Using GitLabs secret variables
To be able to deploy to FTP we will set our credentials as secret variables. Go to your projects settings - CI / CD - Secret variables. Add three individual variables for your FTP username, password and the host.
I checked the box for protected because I want that the variables can only be used when building my protected master branch. If you want be able to use the credentials in every branch, leave the box unchecked. Read more about protected branches here.
In the end you should have created three variables:
I will describe the gitlab-ci file in detail. Scroll down or go here for the complete file. The GitLab runners will start when a .gitlab-ci.yml file can be found in the root directory of the repository. In the end, we will have a pipeline that looks like that:
gitlab-ci.yml Global settings
First of all, we define the docker image for all runners:
Then we define the different stages. We use the default stages:
We want to build the Jekyll site into the folder _site. Therefore we need to cache this folder between the different stages and runners. To do that we use the GitLab cache. We also want to store the vendor folder where all the Ruby stuff will be saved.
Before every job we want to install all the stuff required to run Jekyll and our tests. To do so, we use before_script. Bundle installs everything that is defined in the Gemfile.
gitlab-ci.yml Build job
The build job generates all the static files with Jekyll through the Rakefile. build is the name and the stage of the job. In the script block you defined the commands you want to execute in the docker container, just like you execute these scripts on the local machine. The GitLab artifacts are used to store the _site folder, accessible from the outside. If anything goes wrong after the build, you can download the folder and have a look at the files. The artifacts will be deleted after 1 week to save disk space.
gitlab-ci.yml Test jobs
I run two tests in my repository. Both checks will be executed in parallel in stage test. The first test, named test_html performs HTML syntax checks for the Jekyll sites.
The other test checks if the structure of the posts is still the same.
If all checks were successfull and the runner is executed in the master branch the deploy job will be executed. It first installs lftp using apt-get. Then it uses our secret variables to story the static HTML site at the defined FTP server.
How the lftp command works in detail:
Now we put everything together:
Have a look at the complete pipeline
Go to CI/CD overview of your project. When you commit something you will an overview similar to mine:
Both times the pipeline finished successfill. The branch cicd only has 2 stages because the deploy stage will only be executed in the master branch. The deploy stage was executed when merging the branch into master.
Thanks a lot for following through the tutorial. If you have any more questions on automating Jekyll sites with GitLab CI and Rake feel free to open an issue.