pyratelog

Hugo and gitlab ci cd



It is about time I get this weblog started again, it has been too long…

As I had decided to migrate my code repositories off Github it felt right I should move my weblog from being hosted on Github Pages. At first I was going to move to Gitlab Pages and keep everything the same, but where is the fun in that!

Instead of Jekyll, the static site generator Hugo was recommended to me. I also wanted to play around with Gitlab’s CI/CD pipeline tool, so thought the weblog migration would be a great project. I have migrated all my old posts to the new platform as well, so you won’t miss out on anything!

new projects

Generate a new Hugo site, for this example I will be calling mine ‘pyratelog’

hugo new site pyratelog

Navigate into the new directory and initialise it as a git repository

cd pyratelog
git init

Create a new project in Gitlab

new_gitlab_project

Add your new Gitlab project as a remote repo to your Hugo site and make an initial commit if you want

git remote add origin git@gitlab.com:pyratebeard/pyratelog.git
git add .
git commit -m "initial commit"
git push -u origin master

Your Gitlab project should now be populated with a config.toml file and the ‘archetypes’ directory.

initial_commit

I won’t keep mentioning when to commit changes to git as we all work differently. We will come to it a bit later when we configure our CI/CD pipeline.

configure hugo

Let us add a theme to our Hugo project, in this case I will use my own ‘futuremyth’ theme

git submodule add https://gitlab.com/pyratebeard/hugo-futuremyth.git themes/futuremyth
echo 'theme = "futuremyth"' >> config.toml

I have added in the ‘paginate’ variable to change the default of 10 items to 5, and also set a static directory for use with images in my log entries

cat >> config.toml << EOF
paginate = "5"
staticDir = ["static"]
EOF

I found it is a good idea to change some of the cache directories. There was an issue I had in my Gitlab CI/CD pipeline with root permissions being set on a directory, causing the pipeline to fail

cat >> config.toml << EOF
[caches.images]
dir = ":cacheDir/_gen"
[caches.assets]
dir = ":cacheDir/_gen"
EOF

You should also edit the ‘baseURL’ and ‘title’ variables in your config.toml.

You can start Hugo on your local machine in development mode using

hugo server -D

If you navigate to http://localhost:1313 you should see a fairly empty page. To add new content you run

hugo new posts/hello_world.md

You change the path to whatever you want, and it will be created under the ‘content’ directory.

If you left your deployment server running you should see that in your browser the site should automatically updates. You first entry should show the title of your post and the date. You can open the markdown file in your favourite editor and start writing below the second set of hyphens (---). Everything between the hyphens is metadata for the page. You can add more if you like, I add a ‘summary’, ‘categories’, and ‘tags’ in the following way

summary: How I set up a Hugo website and deployed with Gitlab's CI/CD pipeline
categories: [tech]
tags: [website, hugo, devops, gitlab, automation]

We can now build our site by running

hugo

This won’t include our first post because we have left the draft variable as true. When you are ready to publish change it to false and build the site again. You can build with drafts included by running

hugo -D

autodevops

There a many ways you can host a website, and many ways you can use Gitlab’s CI/CD pipeline to automate the process. The method I have opted for is to run my Hugo site in a docker container on a DigitalOcean droplet. I have chosen not to use docker-compose to include the Nginx reverse proxy as I host other things behind Nginx and don’t want it to be restarted each time I post a log entry.

On a server with Docker already installed you can set up your Nginx reverse proxy with a Let’s Encrypt companion to deal with SSL.

First, we need to create an new network

docker network create nginx-proxy

Then we can start the Nginx container

docker run -d --name nginx-proxy \
	-p 80:80 -p 443:443 \
	--net nginx-proxy \
	-v /etc/nginx/certs \
	-v /etc/nginx/vhost.d \
	-v /usr/share/nginx/html \
	-v /var/run/docker.sock:/tmp/docker.sock:ro \
	jwilder/nginx-proxy

Confirm the container is running by running docker ps, the output should look like this but with a different container id

CONTAINER ID        IMAGE                 COMMAND                  CREATED             STATUS              PORTS                                      NAMES
ab7626dd1bec        jwilder/nginx-proxy   "/app/docker-entrypo…"   2 seconds ago       Up 1 second         0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp   nginx-proxy

Next we can start the Let’s Encrypt companion container (change the email address)

docker run -d --name nginx-proxy-letsencrypt \
	--volumes-from nginx-proxy \
	-v /var/run/docker.sock:/var/run/docker.sock:ro \
	-e "DEFAULT_EMAIL=youremail@yourdomain.tld" \
	jrcs/letsencrypt-nginx-proxy-companion

In the Hugo repository we need to create a .gitlab-ci.yml file so that we can harness the power of the CI/CD pipeline.

Enter the following in to the file

build:
  stage: build
  image: mapitman/docker-hugo:latest
  services:
      - docker:dind
  before_script:
      - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
      - git submodule update --init --recursive
      - docker build --pull -t $CI_REGISTRY_IMAGE:latest .
      - docker push $CI_REGISTRY_IMAGE:latest

deploy:
  stage: deploy
  image: docker:latest
  services:
      - docker:dind
  tags:
      - deploy
  before_script:
      - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
      - docker pull $CI_REGISTRY_IMAGE
      - docker run -d --name "$CONTAINER_NAME" --expose 1313 --net nginx-proxy -e VIRTUAL_HOST=log.pyratebeard.net -e LETSENCRYPT_HOST=log.pyratebeard.net -v $(pwd):/src $CI_REGISTRY_IMAGE

You will notice some variables in the file beginning with $CI_REGISTRY_* and one called $CONTAINER_NAME. These are variables we declare in Gitlab.

If you use Multi-factor Authentication (MFA) on your Gitlab account you will need to generate a Personal Access Token to use in place of your password. To do this navigate to your account settings and under ‘Access Tokens’ fill in the Name and tick the ‘api’ scope. If you don’t enter an expiry date the token will not expire.

personal access token

Make sure you copy the access token, we will need it for the next step.

Navigate to your repository in Gitlab then to ‘Settings’, ‘CI/CD’, and expand the ‘Variables’ section. Enter the following Key/Value pairs

Key Value
CI_REGISTRY registry.gitlab.com
CI_REGISTRY_IMAGE registry.gitlab.com/username/project_name
CI_REGISTRY_USER username
CI_REGISTRY_PASSWORD personal_access_token
CONTAINER_NAME anything

Mark the CI_REGISTRY_PASSWORD variable as ‘Protected’ and make sure you click ‘Save variables’.

Before we push our new Hugo project to Gitlab we need to configure a Runner. Gitlab Runners are used to execute the jobs in our pipeline.

At first I was trying to use a docker runner to build and deploy my project. Building a new docker image was easy, using kaniko, but I struggled to get the deploy section working. In the end I brought it right back to the Keep It Simple Stupid (KISS) philosophy.

On the server install a Runner following the instructions here.

gitlab runners

Use the token that is shown in your repo CI/CD settings under ‘Runners’, add the tag ‘deploy’, and select the ‘shell’ executor.

Make sure you add the gitlab-runner user to the docker group

sudo usermod -aG docker gitlab-runner

Right, we are almost ready to go! The final file we need is a Dockerfile. This tells docker what we want our image to look like. Enter the following into your Dockerfile, changing the base URL as required

FROM jojomi/hugo

COPY . /src
WORKDIR /src

ENV HUGO_WATCH=true
ENV HUGO_THEME=futuremyth
ENV HUGO_BASEURL=https://log.pyratebeard.net

RUN hugo

Now publish your Hugo site by just running hugo again. Make sure all your changes are committed and push

git push

If you navigate to the CI/CD Pipelines page in your Gitlab project your should see the jobs being run.

first pipeline

Both jobs in the pipeline should complete successfully. Here is a breakdown of what the runner is doing:

  • build phase
    • building a new docker image containing our hugo project
    • pushing the new image to our gitlab project’s container registry
  • deploy phase
    • pulling our new docker image from the registry onto our server
    • starting a new container using the image

There is one final thing we have to add to our .gitlab-ci.yml file to ensure the next time you push nothing breaks. In the deploy script, between the docker pull and docker run commands enter the following

docker stop $CONTAINER_NAME
docker rm $CONTAINER_NAME
docker rmi -f $(docker images --filter "dangling=true" -q)

These lines make sure to stop and remove the container with the name you have specified, docker doesn’t like duplicates. The third line removes and old images to keep things tidy.

I hope this, fairly long, post helped you in someway. If you want to get any further information you can get in touch on mastodon, or any other way mentioned on my home page.


Category: workshop

Tags: website hugo devops gitlab automation