Building Scalable CI/CD Pipelines using Jenkins Agents
Jenkins CI/CD (Continuous Integration/Continuous Deployment) is an integral part of the software development life cycle (SDLC) that focuses on automating the build, test, and deployment processes. It involves using Jenkins, an automation server, to continuously integrate code changes, build software, run tests, and deploy applications.
In large organizations with multiple development teams working on different projects, a single Jenkins server may not be sufficient to handle the workload. That's why Jenkins agent comes in here.
Jenkins agents allow you to distribute the build and test jobs across multiple machines or virtual environments, providing scalability and ensuring that each team has dedicated resources to execute their tasks.
Jenkins Master (Server)
Jenkins Master is the Jenkins service itself and where Jenkins is installed. It is also a web server for deciding how, when, and where to run tasks.
Jenkins’s master node holds all key configurations. Jenkins master server is like a control server that orchestrates all the workflow defined in the pipelines. For example, scheduling a job, monitoring the jobs, etc.
Jenkins Agent
Jenkins agents, also known as slaves, are separate nodes that connect to the Jenkins master and handle the execution of jobs. They can be physical machines, virtual machines, or containers. Each agent is assigned a unique label that identifies it.
When you create a Jenkins job, you can specify the agent on which the job should run.
When you trigger the job, the Jenkins master schedules the job to run on the specified agent. The agent then runs the job and sends the results back to the Jenkins master.
In addition to security, it's also important to monitor and manage your Jenkins agents. This includes regularly checking the status of agents and ensuring that they are running the latest software updates and patches.
Overall, Jenkins agents are a valuable tool for managing and scaling your Jenkins environment. By distributing workloads across multiple machines, you can increase efficiency, reduce build times, and improve overall productivity. If you're using Jenkins, it's worth exploring the use of agents to see how they can benefit your team and your workflow.
Let's Create an agent by setting up a node on Jenkins
Step -1:
Create 2 EC2 instances in AWS master and agent.
Step -2:
Install Java and Jenkins on Jenkins-master and configure port 8080 as we have done so many times in our previous blogs
Step -3:
Generate SSH key on the jenkins-master using ssh-keygen command in the .ssh/ folder.
cat id_rsa.pub
Step -4:
Then on the Jenkins-agent server, paste the public key from the Jenkins-master in the authorized_keys file.
vim authorized_keys
Save and exit the file.
Step -5:
Make sure to install Java on the agent server.
For future requirements, we can also installed docker and docker-compose commands
Bellow command list:
sudo apt-get update
sudo apt-get install openjdk-17-jre -y
sudo apt-get install docker.io -y
sudo apt-get install docker-compose -y
Step -6:
Now from the Jenkins master, we can connect the Jenkins agent server
Go to master terminal →
cd .ssh
ssh ubuntu@demo agent ip address
Then press Enter
ssh ubuntu@54.89.0.6
We have successfully connected Master with an agent
Step-7:
Now go to the Jenkins Dashboard and click on " setup an agent"
Enter the node name and select type and click on create.
Add details of the agent node.
now add credentials Jenkins and paste the private key from master id_rsa into those credentials
Click add button
Now select the credentials option, then select “
ubuntu( this is dev key credentials”
Finally, click to save button.
Now click "OK" and Node will be connected and online.
Check the agent's log from the log section whether it is connected or not.
The agent is connected now (make sure Java is installed in your slave node or agent ).
Now create a new job or item with the pipeline
Create a job pipeline for our node-todo app
Enter the item name and select the pipeline option then click the OK button.
Give a general description.
Before writing the pipeline script we need to set up the GitHub webhook
So, go to your GitHub project then add the master terminal/Jenkins server ip address in the webhook section.
Now we have to pass the pipeline script in the script.
Now go to your project and change the code and back to the Jenkins dashboard and see
Go to demo agent security group and add nodejs project port number in enbound rules:
Now open the browser and copy the demo agent IP address and add nodejs port number then hit enter
Example→
54.84.135.252:8000
The application is running !!!!!!
Now configure the pipeline for pushing the image into the docker hub.
Now we have to pass the pipeline script in the pipeline to test the script.
Now go to Manage Jenkins > Credentials > System > Global Credentials > globals > + Add Credentials
Add details and click on Create.
Now configure the pipeline for pushing the image into the docker hub.
Now, build your project.
Now Open the 8000 port on Jenkins-agent, now the app can be accessed from the browser.
Our application image is also pushed to DockerHub.
Are you looking to streamline your Jenkins pipeline configuration without making any modifications to your Jenkins server? Look no further! Introducing the revolutionary process that empowers developers to write Jenkins pipeline code directly in Jenkinsfiles, all within the confines of your code repository.
Step-1
We can build the application using the Pipeline script from SCM by passing the GitHub URL where "Jenkinsfile" is present in the repository.
Now in the pipeline script, use "Pipeline script from SCM"
Step-2
add image name in the docker-compose yml file
Step-3
SCM sections select Git →Add repository url link → Branch select Master/ Main
Then save, change the code, and automatically build it.
In Jenkinsfile write script code
Now the application is up & running and there is a new step added by Jenkins, Declarative Checkout SCM.
By leveraging Jenkins pipelines with Jenkinsfiles, you unlock a world of possibilities. Developers gain full control over their pipeline's destiny, allowing them to version, test, and iterate on their pipeline configurations effortlessly. With this newfound autonomy, teams can seamlessly manage and scale their pipelines across different Jenkins instances, guaranteeing consistent and reliable builds.
That's the power of automation tools !!!!!!!
Thank you for reading this blog. If you found this blog helpful, please like, share, and follow me for more blog posts like this in the future.
— Happy Learning !!!