A deep Dive into an End-to-End Jenkins Project: Unveiling Seamless CI/CD

Today's live session was a captivating journey of an end-to-end Jenkins project by Shubham Londhe, offering a comprehensive glimpse into the world of Continuous Integration and Continuous Deployment (CI/CD). With Jenkins, the industry-standard automation tool, at the helm, participants were treated to a firsthand exploration of how to orchestrate a seamless pipeline that spans from code integration to deployment.

Setup 2 EC2 instances (Master and Agent)

Launch EC2 Instances:

Log in to your AWS Management Console, navigate to EC2, and launch two instances. Make sure they are in the same region and have the necessary security groups and key pairs configured.

Connect both EC2 instances using SSH keys

Generate SSH Key Pair:

Open a terminal on your local machine, access the MASTER instance and run the following command to generate an SSH key pair.

ubuntu@ip-172-31-47-48:~$ cd .ssh/
ubuntu@ip-172-31-47-48:~/.ssh$ ls
authorized_keys
ubuntu@ip-172-31-47-48:~/.ssh$ ssh-keygen
ubuntu@ip-172-31-47-48:~/.ssh$ cat id_rsa.pub 
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8hwOR4LLcBIQyydToUA0C3rU1j4wpWPhn6vbJyDdeGuR8tuNWgIj8Lox7vd8lvBgWThn5qVDMwP1LU9hEFhfjCAhJCOksNrK74akCnArw3dVGkFpH5RQk4aJ4nxep6c+GY+3jQwt6/sZ3eb4TN4c5TKL1HLVCcmLaKPhJ+d9XEDEiTnIDWyM6W0K34ekAYwMn52UCYaidxa5yNGmpPMGpxknDSyl1MUm8wmFEMmTGj5jB6/wk86PFMLAPWrwcQMVHi/cY6pUkVqNgca5osAMcejAcRcpSGC4fsUTPfTH6VgPk8uztAc/Y0dkGZJ3MwJcFfhZueeznhjRh/zgbc18rPB+waiiZdk0xVNPMWub6evYRAiOw80UgtAdEbqtF/VapqP0oY4lnOuQdjS7ZtvlQ4tTtPepdMELwSUDHxRPd1qhE8YP0OTYVEOymAAtGfRhQ7xJ5AggbxJEEoD8IkDy+hoXDP8iqiZgOOE3IoZhYp+xTisjfeYyDzrwJU3ef8Lc= ubuntu@ip-172-31-47-48
ubuntu@ip-172-31-47-48:~/.ssh$

Replicate the Key in Agent instance:

Copy the public key from MASTER, open another terminal and access the AGENT instance and paste the public key in the file-"authorized_keys"

Connect MASTER to AGENT:

Open the MASTER terminal and use the below command to link MASTER and AGENT. Whereas "3.92.31.167" is the Public IPv4 address of the AGENT instance and this can be found on AWS page EC2 dashboard.

ubuntu@ip-172-31-47-48:~$ ssh ubuntu@3.92.31.167

Install Java & Jenkins

Installing Jenkins:

In order to connect Jenkins from Master to Agent node, we need to use the below commands to install Java and Jenkins. After installation, the system might require a reboot.

Java installation command:

sudo apt update
sudo apt install openjdk-17-jre
java -version
openjdk version "17.0.7" 2023-04-18
OpenJDK Runtime Environment (build 17.0.7+7-Debian-1deb11u1)
OpenJDK 64-Bit Server VM (build 17.0.7+7-Debian-1deb11u1, mixed mode, sharing)

Jenkins installation command:

curl -fsSL https://pkg.jenkins.io/debian/jenkins.io-2023.key | sudo tee \
  /usr/share/keyrings/jenkins-keyring.asc > /dev/null
echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
  https://pkg.jenkins.io/debian binary/ | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update
sudo apt-get install jenkins
systemctl status jenkins

Setting up inbound rules:

In order to access Jenkins via web-browser we need to setup inbound rules for MASTER instance. Once the rule has been setup we can then access Jenkins using the public IPv4 address: custom port. In this case, we used port 8080

Unlock Jenkins:

The Jenkins start page will show the file path where the admin key is stored.

Locate the Key:

Access the key using the below command, copy it and paste it on the Jenkins page.

ubuntu@ip-172-31-47-48:~$ sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Install Plugins:

Jenkins plugins enhance the capabilities of your Jenkins server, enabling integration with different tools, services, and technologies. Simply click on Install Recommended Plugins and the process will start

Complete steps:

Fill in all the user details and complete the steps. Your Jenkins is Ready to go !!!

Connect Jenkins (Master) to a new node (Agent)

Setup a node/ agent:

Up until now, we have learned about the pipeline being created, build and deploy on the local machine. Now the local machine (i.e. Master) will allocate the work to be done on a Node/ Agent. This can be done by selecting Distributed Build-> Set up an agent.

Filling out the form:

To create a new node (build agent) in Jenkins, you need to fill out a form that provides the necessary configuration details. Here's a basic outline of the form to build a new node in Jenkins:

Node Name: dev-agent

Descriptions: This agent will handle dev builds

#of Executors: 1

Remote root directory: /home/ubuntu

Labels: dev

Usage: use this node as much as possible

Launch method: Launch agents via SSH

Host: 3.92.31.167. This is the IPv4 address of the Agent instance and this can be found on AWS EC2 instance.

Credentials: Add credentials-> Jenkins.

• Kind: SSH Username with private key

• ID: dev-ssh-key

• Description: this is dev key credentials

• Username: ubuntu

• Private Key: Select Enter directly. Key- Paste the ssh private key from the Master instance and Click-> Add

Credentials: Select the above created credentials (i.e. ubuntu)

Host Key Verification Strategy: Non Verifying Verification Strategy. This strategy implies that the SSH connection being configured in this context will not verify the host's key at all. It will establish the connection without performing the customary key matching process. And Click-> Save

Connection Established

You have now successfully Connected Jenkins (Master) to a new node (dev-agent)

Creating a new pipeline

Creating a new pipeline in Jenkins involves defining a series of stages and steps to automate the build, test, and deployment processes. The task will not be deployed on the Master but on the Agent.

Create New Item:

• Click on "Create a job" in Jenkins.

• Enter a name for your pipeline (e.g., "node-todo-cicd").

• Select "Pipeline" and click "OK."

GitHub Project:

• This option will get the code from the GitHub

• Enter the URL of the repository i.e.- https://github.com/AminChivilkar/node-todo-cicd

  

Build Triggers:

• select -> GitHub hook trigger for GITScm polling

• any changes done on GitHub will reflect on the job

Adding Webhooks to Github

• Go to the Master instance on AWS console

• Select security-> security groups-> and make sure port 8080 is open

• Now copy the public IPv4 address of Master instance:8080

• Open GitHub-> Go to intended repository-> settings-> add webhooks-> paste in Payload URL-> public IPv4 address of Master instance:8080/github-webhook/

  Example-> http://18.215.254.55:8080/github-webhook/

  

Writing Groovy Scripts:

The script shows that the pipeline will run on the node - dev-agent with three stages- Code, Build/ test & Deploy. Once the script is written, we can save it.

pipeline {
    agent { label 'dev-agent' }

    stages {
        stage('Code') {
            steps {
                git url 'https://github.com/AminChivilkar/node-todo-cicd.git', branch: 'master'
            }
        }

        stage('Build & Test') {
            steps {
                sh 'docker build . -t node-todo-app'
            }
        }

        stage('Deploy') {
            steps {
                sh 'docker-compose down && docker-compose up -d'
            }
        }
    }
}

Install Docker and Docker Compose (Agent):

As we can see that the groovy script contains docker and compose commands, it is necessary to install docker & compose on the Agent instance.

sudo apt-get update && sudo apt install -y docker.io && 
sudo systemctl start docker && sudo systemctl enable docker && 
sudo apt install -y docker-compose

Adding a user to the Docker group:

We need to add the current user to the docker group on a Linux system.

sudo usermod -aG docker $USER

Building pipeline:

We are now one click away and once executed the pipeline will start to build.

Stage View

Now we have successfully Deployed node JS app on the Agent instance.

Final Output

After changing the inbound rules for the agent instance on AWS console, we can now finally access the application using the public IPv4 address: 8000.

The pipeline was built and deployed on the Agent instance and everything was exercised on the Master instance.

Conclusion:

In this brief documentation, we've explored how we can deploy a pipeline on nodes/ agents without the need to deploy the pipeline in the local machine or master instance.

Hope you like my post. Don't forget to like, comment, and share.