Continuous Deployment using GitLab CI/CD
Introduction
In software development task automation is very essential as it reduces the chance of introducing errors and makes the process faster while developing applications.
A typical workflow of a software deployment process without automation will be to either use FTP(file transfer protocol) client to transfer the files to a server or manually pulling updates from the project git repository to the server for every change made.
Consider the scenario below:
- Developer A commits an update to a project repository
- The updates get reviewed by developer B who then gets back to Developer A if there are any changes to correct.
- When code review is done, someone needs to login into the server to pull the updates.
The above scenario repeats itself every time a change is made to the project. Now, imagine there are about five developers working on that project which could even be more.
By repeating these steps errors are bound to happen. The code might not be properly reviewed, proper testing is not carried out before updating the server’s code-base and the server might not be updated on time which makes the development process slow and painful. Here is where continuous deployment and integration comes in. GitLab CI/CD is a tool built into GitLab that allows you to apply all the continuous methods (continuous integration, delivery, and deployment) to your software with no third-party application or integration needed.
Goal
In this tutorial, we’ll be creating a build process that automates a case scenario similar to the manual software deployment process we discussed earlier.
Using GitLab CI/CD, we’ll automate a build(Install app dependencies), test(run unit test) and deploy to our server.
After our implementation this is what our deployment process will look like:
- When Developer A commits to a project repository.
- GitLab runs our defined pipeline, we’ll talk more on this in coming sections.
- Which installs our application dependencies.
- Run unit set we have defined.
- then login into our server using SSH (this will be explored in depth in the coming section)
- and update the application on the server with changes made.
Sounds fun? excited? Let’s get it done. 💪
Prerequisites
Before we proceed let’s take a look some things we need to put in place
- You must have a GitLab account
- A GitLab hosted repository.
- A remote server that has Git, Node, NPM (Node Package Manager), and pm2(Process Manager for Node) installed. You can get one on Digital ocean
- Basic understanding of using the command line.
For the sake of this tutorial, I created a simple express web app which can be found here https://gitlab.com/03balogun/express.git we’ll be deploying this app.
Let’s get our app up and running
- Login into your remote server.
- Change directory to where you will like to clone the project.
- Install application dependencies.
- And start the application node process using PM2.
// cd into your project directorygit clone https://gitlab.com/03balogun/express.gitcd expressnpm installpm2 start bin/www -n express
This is a onetime project set up the subsequent update will be automated. 🤗
To access the running project visit http://your-host-address:3000 from your browser, you should see something similar.
Configuring CI/CD Pipeline
In this section, we’ll be configuring our GitLab Pipeline to run our jobs.
Pipelines are the top-level component of continuous integration, delivery, and deployment.
Pipelines comprise of the following:
- Jobs that define what to run. For example, code compilation or test runs.
- Stages that define when and how to run. For example, tests run only after code compilation.
GitLab CI/CD pipelines are configured using a YAML file called.gitlab-ci.yml within each project. In this file, we will define the scripts to run for each job.
When we define the .gitlab-ci.yml file in our projects GitLab automatically detects and runs the defined jobs with a tool called GitLab Runner.
Create .gitlab-ci.yml in your project root directory and add the snippet below:
Let’s break down what the above configuration does :
Theimage parameter is used to specify the Docker image to run our defined jobs. Which in our case is Node v10.11.0
we used the cahe parameter to specify the directory to cached across jobs, which in this case is the node_modules.
The stages parameter is used to define stages that can be used by jobs, the arrangement of the stages determines the order which our job runs.
- First, all jobs of the
buildare executed in parallel - If all jobs of
buildsucceed, thetestjobs are executed in parallel. - If all jobs of
testsucceed, thedeployjobs are executed in parallel. - If all jobs of
deploysucceed, the commit is marked as passed. - If any of the previous jobs fails, the commit is marked as failed and no jobs of the further stage are executed.
We defined our first job install dependencies, which has the stage and script parameters.
The stage keyword is used to specify the stage which this job should run, which we set as build
script is a required keyword which is used to specify the shell script to be executed by GitLab runner.
We declared the second job test application which runs the unit test for our application.
If we commit and push the current state of our .gitlab-ci.yml file, GitLab will create a pipeline for our branch and run the jobs which we have specified inside this file.
Below is a screenshot of the jobs we have specified, it executed successfully 😄
There’s one last job we are yet to define, the job that deploys our code to the server. We’ll get this done right after we finish generating ssh keys. Let’s go 💃
SSH key configuration
SSH means Secure Shell, its a secure protocol for securely logging in to and running programs on remote machines across a network, with encryption to protect the transferred information and authentication to ensure that the remote machine is the one desired.
SSH keys are a matching set of cryptographic keys which can be used for authentication. Each set contains a public and a private key.
Generate an RSA key pair on your local computer :
Generate an RSA key pair on your local computer :
- Open your terminal then run
cd ~/.ssh/this is the default directory where your generated public and private keys save. run ssh-keygenyou get the below prompt asking you to set the file path to save to generated key.
$ Enter file in which to save the key (~/.ssh/id_rsa):- Leaving this empty saves the keys in the default directory and allows SSH client to find the keys automatically.
- Next, you’ll be prompt to enter a passphrase as an additional security measure for the generated keys, entering a passphrase during SSH key generation will require re-entering it anytime the private key will be used. On GitLab runner we won’t be able to type passwords, so feel free to leave this empty by pressing the enter key.
- Your RSA SSH key will be generated and saved in
~/.ssh/id_rsaa hidden folder within your systems home directory. You’ll get a screen similar to the below after your key has been successfully generated.
Your identification has been saved in id_rsa.Your public key has been saved in id_rsa.pub.The key fingerprint is:SHA256:QDWO6ch1dzLdbWAqHLeCO2jzysMC3LC+uZw1jjVE0a8 username@hostThe key's randomart image is:+---[RSA 2048]----+| ....o . . o || o.+ + + = o || . =.+ B = . o|| .o + +.o * . || . ++ =.S || +...Eo . || . .=. . || ..B.+o. || Bo..o. |+----[SHA256]-----+
Copy SSH public key to your remote server :
There are several ways to do this, in this tutorial we’ll be using the ssh-copy-id. The ssh-copy-id tool is included in many Linux distributions’ OpenSSH packages.
Open your terminal and type:
ssh-copy-id username@your_remote_ipThis will prompt you to enter your user password for the remote server.
After typing in the password, the contents of your ~/.ssh/id_rsa.pub key will be appended to the end of the user account’s ~/.ssh/authorizes_keys file.
Copying your public key to server lets us login into the server without a password prompt.
You can now log into the server with that account without a password:
ssh username@your_remote_ipAdd your private key to GitLab
- Login into GitLab
- Goto your project settings
- Click on CI/CD from the settings drop down and select it
- Expand the environment variables section and enter SSH_PRIVATE_KEY as variable key and your private ssh key as variable value.
Defining deployment Job
Add the snippet below to your .gitlab-ci.yml file
Let’s break down what the above does :
We defined a job named deploy which runs during the deploy stage.
Within the deploy job, the script keyword holds a couple of commands to be executed in our build environment.
- First, we checked if the executable file for
ssh-agentexists, if not we then install it - run the
ssh-agentcommand which is used to hold private keys for public key authentications. sss-add<(echo"$SSH_PRIVATE_KEY")command adds our ssh private key from our defined environment variable.- we then establish a connection to our remote server and also run a command that changes directory to our project directory, pulls the updates from our repo, install dependencies and restart our application. Our application will then be up and running on our server.
Now let’s commit and push our updates for .gitlab-ci.yml to our project repository.
Our pipeline should run three jobs, similar to the below screenshot.
Clicking on the pipeline number(#54431527) shows more details about the jobs.
Clicking on the deploy job shows more details. Where we can see that our application was successfully pulled from the repository and restarted successfully.
Conclusion
We have successfully automated our application deployment process by using GitLab CI/CD to create a build, test and deployment process. Now we can make our application deployment process fast and painless. More can still be achieved using GitLab CI. Explore the infinity stones at your fingertips, use the resources below as a guide to help you unlock zen mode CI/CD techniques with GitLab CI/CD. Feel free to shoot me an email at 03balogun@gmail.com or drop a comment here If you get stuck and need help.
Don’t forget to clap as hard as you can if this article was helpful 😃
