Devops Docs

Build Automation: CI/CD with Jenkins

Introduction to build automation

  • Need a dedicated server
    • Test environment is prepared
    • Docker credentials configured
    • All necessary tools installed
    • Trigger build automatically
  • In a build automation process
    • Code is automatically checked out
    • Code is tested
    • Application is built
    • Pushed to repository
    • Deploy to server
  • Tools
    • Jenkins
      • Software that you install on server
      • Has UI for configuration
      • Install all the tools you need (Docker, Gradle, Maven, etc)
      • Configure the tasks (run tests, build app, deployment, etc)
      • Configure the automatic trigger of the workflow
  • What you can do with Jenkins
    • Run tests
    • Build artifacts
    • Publish artifacts
    • Deploy artifacts
    • Send notifications
  • Needs to integrate with other tools
    • Docker
    • Build tools
    • Repos
    • Deployment servers
  • Has plugins
  • What do you need to configure?
    • Run tests
      • Build tools need to be available
        • npm test, gradlew test, mvn test
      • Configure test environment (eg test database)
  • Build Application
    • Build tools or Docker
  • Publish Docker Image
    • Store Credentials in Jenkins
    • Jenkins user must have access to all these technologies and platforms

Install Jenkins

  • 2 ways
    • Install Jenkins directly on OS
      • download package and install on server
      • create separate Jenkins User on server
    • Run Jenkins as a Docker container
  • Must have at least a gigabyte of RAM
  • 8080 - Jenkins
  • Need to open port 50000 as well
  • This is where jenkins master and worker nodes communicate
  • We'll mount volumes too.

Install docker

  • Visit docs to install docker
  • NB: Installing with snap comes with some issues
apt update
docker run -p 8080:8080 -p 50000:50000 -d -v jenkins_home:/var/jenkins_home jenkins/jenkins:lts

Jenkins UI tour

  • 2 roles for Jenkins App
    • Jenkins Administrator
      • administers and manages Jenkins
      • sets up Jenkins cluster
      • installs plugins
      • backup Jenkins data
  • Developer or DevOps teams
    • creating the actual jobs to run workflows

Installing Build Tools

  • Create Job to automate your apps workflow
    • Depending on your App (Programming Language) you need to have different Tools installed and configured on Jenkins
  • 2 ways to install and configure those tools
    • Jenkins Plugins
      • just install plugin (via UI) for your tool
    • Install tools directly on server
      • enter in remote server and install
      • inside the Docker container, when Jenkins runs as a container

Configure Plugin for Maven

  • Global tool configuration on UI

Install npm and Node in Jenkins container

  • Enter container as a root user
docker exec -u 0 -it <id> bash

# Check distro
cat /etc/issue # Debian
  • So we need the node and npm installation guide for Debian
apt update
apt install curl
apt install nodejs npm -y

Jenkins Basics Demo

Create a Simple Freestyle Job & Plugin Configuration

  • We'll use the Freestyle Project
  • In prod, we don't use this. We use the Pipeline and multibranch pipeline
  • Build steps
    • Execute shell
    • Can run npm commands because we installed it in the container. Can't run maven commands
    • Can use Invoke top-level Maven targets since we have the Maven plugin installed
  • Even though installing a tool on the server requires a bit more effort, it is more flexible
  • The plugin gives you a UI that is limited to the provided input fields
  • For a tool to show up on the Global Tool Config page, it has to be installed as a plugin

Configure Git Repository

  • SSH

    • Enter container as Jenkins user
    • Create ssh key
    • Paste private key in UI
    • Paste public key in github
    • Run ssh -T git@github.com
    • Restart Jenkins

  • Jobs are saved in a directory

    • /var/jenkins_home/jobs

  • Git checkout data

    • /var/jenkins_home/workspace

  • Do something from Git repo in Jenkins Job

    • Add commands in execute shell
chmod +x freestyle-build.sh
./freestyle-build.sh

Docker In Jenkins

Make Docker Available In Jenkins

  • Attach a volume to Jenkins from the host machine
  • We have docker on our droplet
  • So we mount the docker runtime directory from our droplet into the container as a volume
  • This will make docker available inside the container
  • We'll restart our container with new volumes
docker run -p 8080:8080 -p 50000:50000 -d \
> -v jenkins_home:/var/jenkins_home \
> -v /var/run/docker.sock:/var/run/docker.sock \        # Docker volume
> -v $(which docker):/usr/bin/docker jenkins/jenkins:lts      # Docker runtime
  • Give permission to docker.sock file to Jenkins user
  • Log in to container as root
chmod 666 /var/run/docker.sock
  • If there is an issue with docker in the container
    • Create a new Jenkins container without the docker runtime volume
docker run -p 8080:8080 -p 50000:50000 -d -v jenkins_home:/var/jenkins_home -v /var/run/docker.sock:/var/run/docker.sock jenkins/jenkins:lts

Install docker in Jenkins container

curl https://get.docker.com > dockerinstall && chmod 777 dockerinstall && ./dockerinstall

Change permissions to allow Jenkins user to read and write to docker.sock

chmod 666 /var/run/docker.sock

Build Docker Image

  • In build step
docker build -t java-maven-app:1.0 .

Push image to DockerHub

  • Create credentials for DockerHub in Jenkins
    • Route: /manage/credentials/store/system/domain/_/
  • In build environment, use secret text to reference credentials
  • In shell script
docker build -t alfredasare/devops-demo-app:jma-1.0 .
docker login -u $USERNAME -p $PASSWORD <private repo domain>
docker push alfredasare/devops-demo-app:jma-1.0

To fix password insecure error

docker build -t alfredasare/devops-demo-app:jma-1.1 .
echo $PASSWORD | docker login -u $USERNAME --password-stdin
docker push alfredasare/devops-demo-app:jma-1.1

Push Docker Image to Nexus Repository

  • Need to allow insecure URLs on Jenkins like we did for Docker Desktop
    • Add to daemon.json
vim /etc/docker/daemon.json

daemon.js

{
  "insecure-registries": ["46.101.137.161:8083"]
}

Restart Docker

systemctl restart docker
  • After restarting docker, all containers are stopped
  • Start them back up
  • Change permissions to /var/run/docker.sock inside the container again
  • Enter container as root
chmod 666 /var/run/docker.sock
  • Create credentials for Nexus

Build script

  • Change credentials to Nexus credentials
docker build -t 46.101.137.161:8083/java-maven-app:1.1 .
echo $PASSWORD | docker login -u $USERNAME --password-stdin 46.101.137.161:8083
docker push 46.101.137.161:8083/java-maven-app:1.1

Freestyle to pipeline jobs

  • Freestyle jobs with multiple steps aren't recommended
  • Freestyle jobs should have 1 job/step
    • One job for integration tests
    • Another for building the app
    • Another for deploying the app
    • Not suitable for complex workflows
  • Pipeline
    • Suitable for CI/CD
    • Scripting - pipeline as code

Introduction to pipelines

  • Create pipeline
    • Configuration
      • Pipeline: Pipelines are scripted
      • Configuration is done in a script and not the UI
      • Script is written in Groovy (similar to Java)
  • Groovy Sandbox
    • You can execute a limited no. of Groovy functions, for that you don't need approval from a Jenkins admin
  • Best practice to IaC:
    • Pipeline Script should be in your Git Repo
  • We'll use Pipeline Script from SCM (Source Code Mgmt)
  • Specify Script Path as Jenkinsfile
    • Jenkins looks for a Jenkinsfile after checking out your app
  • Pipeline syntax
    • Can be written as a Scripted or Declarative Pipeline
    • Scripted
      • First syntax
      • Groovy engine
      • Advanced scripting capabilities, high flexibility
      • Difficult to start
    • Declarative
      • More recent addition
      • Easier to get started with but not that powerful
      • Pre-defined structure

Required fields of Jenkinsfile

  • Pipeline must be top-level
  • agent - where to execute
    • Could be a node or executor on that node
    • Relevant with a Jenkins cluster
    • pipeline + agent == node
  • stages: where the work happens
    • stages and steps

Jenkinsfile

pipeline {
    agent any

    stages {
        stage("build") {
            steps {
            }
        }

        stage("test") {
            steps {
            }
        }

        stage("deploy") {
            steps {
            }
        }
    }
}

node {
    // groovy script
}

Advantages of Pipeline Jobs over Freestyle jobs

  • Parallel jobs
  • User input
  • Conditional statements
  • Set variables
  • No reliance on plugins
  • No need to manage plugins
  • No need to manage different jobs and their plugins for your CI process

Jenkins Syntax

Post Attribute In JenkinsFile

  • Execute some logic AFTER all stages executed
  • Conditions
    • always
    • success
    • failure
post {
    always {

    }
}

Define conditions for each stage

stage("test") {
    when {
        expression {
            BRANCH_NAME == 'dev' || BRANCH_NAME == 'master'
        }
    }
    steps {
        echo 'testing the application...'
    }
}

Example 2

def gv
CODE_CHANGES = getGitChanges() // Groovy script that checks for code changes

pipeline {
    agent any
    stages {
        stage("init") {
            steps {
                script {
                    echo "initialize"
//                     gv = load "script.groovy"
                }
            }
        }
        stage("test") {
            when {
                expression {
                    BRANCH_NAME == 'dev'
                }
            }
            steps {
                echo 'testing the application...'
            }
        }
        stage("build jar") {
            when {
                expression {
                    BRANCH_NAME == 'dev' && CODE_CHANGES == true
                }
            }
            steps {
                script {
                    echo "building jar"
                    //gv.buildJar()
                }
            }
        }
        stage("build image") {
            steps {
                script {
                    echo "building image"
                    //gv.buildImage()
                }
            }
        }
        stage("deploy") {
            steps {
                script {
                    echo "deploying"
                    //gv.deployApp()
                }
            }
        }
    }
}

Environmental Variables

Using Credentials in Jenkinsfile

  • Define Credentials in Jenkins UI
  • credentials("credentialId") binds the credentials to your env variable
  • For that you need Credentials Binding Plugin
    • Credentials Plugin: allows you to store credentials in Jenkins
    • Credentials Binding Plugin: allows credentials to be bound to environment variables for use from misc build steps

Jenkinsfile

def gv

pipeline {
    agent any

    //////////////////////////////////
    environment {
        NEW_VERSION = '1.3.0'
        // SERVER_CREDENTIALS = credentials('server-credentials')
    }

    stages {
        stage("init") {
            steps {
                script {
                    echo "initialize"
//                     gv = load "script.groovy"
                }
            }
        }
        stage("build jar") {
            steps {
                script {
                    echo "building jar"
                    echo "building version ${NEW_VERSION}"
                    //gv.buildJar()
                }
            }
        }
        stage("build image") {
            steps {
                script {
                    echo "building image"
                    //gv.buildImage()
                }
            }
        }
        stage("deploy") {
            steps {
                script {
                    echo "deploying"
                    ////////////////////////
                    withCredentials([
                        usernamePassword(credentialsId: 'server-credentials', usernameVariable: 'USER', passwordVariable: 'PWD')
                    ]) {
                        sh "some script $USER $PWD"
                    }
                    //gv.deployApp()
                }
            }
        }
    }
}

Tools attribute for Build Tools

  • Only 3 build tools available
    • gradle, maven and jdk

Jenkinsfile

pipeline {
    agent any
    tools {
        maven 'maven-3.9' // Name of build tool installation
    }
}

Parameters in Jenkinsfile

Jenkinsfile

def gv

pipeline {
    agent any
    parameters {
//         string(name: 'VERSION', defaultValue: '', description: 'version to deploy on prod')
        choice(name: 'VERSION', choices: ['1.1.0', '1.2.0', '1.3.0'], description: '')
        booleanParam(name: 'executeTests', defaultValue: true, description: '')
    }
    stages {
        stage("init") {
            steps {
                script {
                    echo "initialize"
//                     gv = load "script.groovy"
                }
            }
        }
        stage("test") {
            when {
                expression {
                    params.executeTests
                }
            }
            steps {
                echo 'testing the application...'
            }
        }
        stage("build jar") {
            steps {
                script {
                    echo "building jar"
                    //gv.buildJar()
                }
            }
        }
        stage("build image") {
            steps {
                script {
                    echo "building image"
                    //gv.buildImage()
                }
            }
        }
        stage("deploy") {
            steps {
                script {
                    echo "deploying"
                    echo "deploying version ${params.VERSION}"
                    //gv.deployApp()
                }
            }
        }
    }
}

Using external Groovy Script

  • Good idea to clean up scripts in Jenkinsfile and put them in their own groovy script file
  • Write scripts in scripts block
  • All environmental variables in Jenkinsfile are available in the groovy script

script.groovy

def buildJar() {
    echo "building the application..."
//    sh 'mvn package'
}

//def buildImage() {
//    echo "building the docker image..."
//    withCredentials([usernamePassword(credentialsId: 'docker-hub-repo', passwordVariable: 'PASS', usernameVariable: 'USER')]) {
//        sh 'docker build -t nanajanashia/demo-app:jma-2.0 .'
//        sh "echo $PASS | docker login -u $USER --password-stdin"
//        sh 'docker push nanajanashia/demo-app:jma-2.0'
//    }
//}

def testApp() {
    echo 'testing the application...'
}

def deployApp() {
    echo "deploying"
    echo "deploying version ${params.VERSION}"
}

return this

Jenkinsfile

def gv

pipeline {
    agent any
    parameters {
//         string(name: 'VERSION', defaultValue: '', description: 'version to deploy on prod')
        choice(name: 'VERSION', choices: ['1.1.0', '1.2.0', '1.3.0'], description: '')
        booleanParam(name: 'executeTests', defaultValue: true, description: '')
    }
    stages {
        stage("init") {
            steps {
                script {
                    echo "initialize"
                    gv = load "script.groovy"
                }
            }
        }
        stage("test") {
            when {
                expression {
                    params.executeTests
                }
            }
            steps {
                script {
                    gv.testApp();
                }
            }
        }
        stage("build jar") {
            steps {
                script {
                    gv.buildJar()
                }
            }
        }
        stage("build image") {
            steps {
                script {
                    echo "building image"
                    //gv.buildImage()
                }
            }
        }
        stage("deploy") {
            steps {
                script {
                    gv.deployApp()
                }
            }
        }
    }
}
  • You can go to Replay mode for a build and make changes to the script to see how the changes affect the build without having to commit any code

Input Parameter in Jenkinsfile

Jenkinsfile

stage("deploy") {
    input {
        message "Select the environment to deploy to"
        ok "Done"
        parameters {
            choice(name: 'ONE', choices: ['dev', 'staging', 'production'], description: '')
            choice(name: 'TWO', choices: ['dev', 'staging', 'production'], description: '')
        }
    }
    steps {
        script {
            gv.deployApp()
            echo "Deploying to ${ONE}"
            echo "Deploying to ${TWO}"
        }
    }
}

Parameters in the script block

stage("deploy") {
    steps {
        script {
            env.ENV = input message: "Select the environment to deploy to", ok: "Done", parameters: [choice(name: 'ONE', choices: ['dev', 'staging', 'production'], description: '')]

            gv.deployApp()
            echo "Deploying to ${ENV}"
        }
    }
}

Create a complete Pipeline

script.groovy

def buildJar() {
    echo "building the application..."
    sh 'mvn package'
}

def buildImage() {
    echo "building the docker image..."
    withCredentials([usernamePassword(credentialsId: 'docker-hub-repo', passwordVariable: 'PASS', usernameVariable: 'USER')]) {
        sh 'docker build -t alfredasare/devops-demo-app:jma-2.0 .'
        sh "echo $PASS | docker login -u $USER --password-stdin"
        sh 'docker push alfredasare/devops-demo-app:jma-2.0'
    }
}

def deployApp() {
    echo "deploying the application"
}

return this

Jenkinsfile

def gv

pipeline {
    agent any
    tools {
        maven 'maven-3.9'
    }
    stages {
        stage("init") {
            steps {
                script {
                    gv = load "script.groovy"
                }
            }
        }
        stage("build jar") {
            steps {
                script {
                    gv.buildJar()
                }
            }
        }
        stage("build image") {
            steps {
                script {
                    gv.buildImage()
                }
            }
        }
        stage("deploy") {
            steps {
                script {
                    gv.deployApp()
                }
            }
        }
    }
}

Introduction to multibranch pipeline

  • Run tests on every branch
  • Execute deployments only on Main/Master Branch
  • When merged into master: test, build and deploy App
  • We'll build pipelines for ALL the branches
  • Need different behaviour based on branch
  • We need to create a multibranch pipeline
    • We need to dynamically create new pipelines for new branches
    • Multibranch pipeline name is usually the name of the app
    • For Behaviours choose Filter by name (with regular expression)
      • .* --> To match all branches
  • Can also specify multiple branches for a regular pipeline but you won't have access to variables and plugins for multiple branch pipelines
    • You won't have an overview of the status of the different branches
  • You would have one Jenkinsfile that all the branches share
    • We need logic to allow deploys only on master and tests on other branches

Branch-based logic for a multibranch pipeline

  • The BRANCH_NAME is only available in the multibranch pipeline jobs

Jenkinsfile

stage("test") {
    steps {
        script {
            echo "Testing the application..."
            echo "Executing pipeline for branch $BRANCH_NAME"
        }
    }
}
stage("build") {
    when {
        expression {
            BRANCH_NAME == 'master'
        }
    }
    steps {
        script {
            echo "Building the application..."
        }
    }
}
stage("deploy") {
    when {
        expression {
            BRANCH_NAME == 'master'
        }
    }
    steps {
        script {
            echo "Deploying the application..."
//                     gv.deployApp()
        }
    }
}

Jenkins Job Overview

  • 3 Types of Jobs
    • Freestyle
      • Executing a single task
    • Pipeline
      • Better solution for CI/CD
      • One job for multiple steps
    • Multibranch pipeline
      • Parent of pipeline jobs
      • One pipeline is meant for a single branch
      • Multibranch pipeline is for multiple branches

Credentials In Jenkins

  • Credentials Plugin helps to store and manage credentials centrally
  • Credentials Scopes
    • System
      • Only available on Jenkins Server
      • e.g. For Jenkins admin.
      • Not visible or accessible by Jenkins Jobs
    • Global
      • Available everywhere
  • Credential Types
    • Github App
    • SSH Username with private key
    • Secret file
    • Secret text
    • Certificate
  • Can have new types based on plugins
  • Credential ID is what you reference in your jobs
  • There is also a Credentials Page for multibranch pipelines
  • That's another scope besides Global and System
  • This scope is limited to the multibranch pipeline or project
  • This comes from a folder plugin
    • Enables you to have credentials scoped to your project
    • It's a good way to seperate credentials for different projects

Jenkins Shared Library

  • Assuming we have microservices
  • In Jenkins
    • We'll need a multistage pipeline for each of those microservices to build them
    • All the projects have their own Jenkins files
    • May have the same logic in those Jenkins files
    • We don't want to copy and paste
  • Shared Library
    • Extension to Pipeline
    • Is its own repository written in Groovy
    • Reference shared logic in Jenkinsfile
  • Another use case
    • A company may have multiple teams working on multiple projects
    • May not use the same tech stack, but might have the same logic
      • e.g. company-wide Nexus Repo
      • e.g. company-wide Slack channel
    • Don't replicate code
  • We'll
    • Create the Shared Library (SL)
    • Make SL available in Jenkins
    • Use the SL in Jenkinsfile

Create Shared Library Project/Repository

  • Create repo

  • Write Groovy code

  • Make SL available globally or for project

  • Use the Shared Library in jenkinsfile to extend the Pipeline

  • Structure of Shared Library

    • vars folder
      • functions that we call from Jenkinsfile
      • Each function/execution step is its own Groovy file
        • e.g. build Jar File
        • build Docker Image
        • push Docker Image
    • src folder
      • helper code
    • resources folder
      • for external libraries
      • non-groovy files

  • You can use this snippet in your Jenkinsfile to let your editor know you're working in a groovy file

#!/usr/bin/env groovy

vars

buildJar.groovy

#!/usr/bin/env groovy

def call() {
    echo "building the application..."
    sh 'mvn package'
}

buildImage.groovy

#!/usr/bin/env groovy

def call () {
    echo "building the docker image..."
    withCredentials([usernamePassword(credentialsId: 'docker-hub-repo', passwordVariable: 'PASS', usernameVariable: 'USER')]) {
        sh 'docker build -t alfredasare/devops-demo-app:jma-2.0 .'
        sh "echo $PASS | docker login -u $USER --password-stdin"
        sh 'docker push alfredasare/devops-demo-app:jma-2.0'
    }
}
  • Create a repository

Make shared library available globally

  • In Jenkins
    • Manage Jenkins > Configure System
    • Add to Global Pipeline Libraries
      • Default version could be a branch or commit hash or tag

Use shared library in Jenkinsfile

Jenkinsfile

#!/usr/bin/env groovy

@Library('jenkins-shared-library')_  // Name we called library in Jenkins config
def gv

pipeline {
    agent any
    tools {
        maven 'maven-3.9'
    }
    stages {
        stage("init") {
            steps {
                script {
                    gv = load "script.groovy"
                }
            }
        }
        stage("build jar") {
            steps {
                script {
                    buildJar()
                }
            }
        }
        stage("build image") {
            steps {
                script {
                    buildImage()
                }
            }
        }
        stage("deploy") {
            when {
                expression {
                    BRANCH_NAME == 'master'
                }
            }
            steps {
                script {
                    echo "Deploying the application..."
                    gv.deployApp()
                }
            }
        }
    }
}
  • @Library('jenkins-shared-library')_
    • The _ is required if we don't have any variable definitions or imports after the library declaration to show the separation.

Using Parameters in Shared Library

buildImage.groovy

#!/usr/bin/env groovy

def call (String imageName) {
    echo "building the docker image..."
    withCredentials([usernamePassword(credentialsId: 'docker-hub-repo', passwordVariable: 'PASS', usernameVariable: 'USER')]) {
        sh "docker build -t $imageName ."
        sh "echo $PASS | docker login -u $USER --password-stdin"
        sh "docker push $imageName"
    }
}
  • Access Global Vars

buildJar.groovy

#!/usr/bin/env groovy

def call() {
    echo "building the application for branch $BRANCH_NAME"
    sh 'mvn package'
}

Extract Logic Into Groovy Classes

  • Can do this in src directory
  • Create a new package com.example

Docker.groovy

#!/usr/bin/env groovy
package com.example

class Docker implements Serializable {
    def script

    Docker(script) {
        this.script = script;
    }

    def buildDockerImage(String imageName) {
        script.echo "building the docker image..."
        script.withCredentials([script.usernamePassword(credentialsId: 'docker-hub-repo', passwordVariable: 'PASS', usernameVariable: 'USER')]) {
            script.sh "docker build -t $imageName ."
            script.sh "echo $script.PASS | docker login -u $script.USER --password-stdin"
            script.sh "docker push $imageName"
        }
    }
}

buildImage.groovy

#!/usr/bin/env groovy

import com.example.Docker

def call (String imageName) {
    return new Docker(this).buildDockerImage(imageName)
}

Split buildDockerImage into separate steps

Docker.groovy

#!/usr/bin/env groovy
package com.example

class Docker implements Serializable {
    def script

    Docker(script) {
        this.script = script;
    }

    def buildDockerImage(String imageName) {
        script.echo "building the docker image..."
        script.sh "docker build -t $imageName ."
    }

    def dockerLogin() {
        script.withCredentials([script.usernamePassword(credentialsId: 'docker-hub-repo', passwordVariable: 'PASS', usernameVariable: 'USER')]) {
            script.sh "echo $script.PASS | docker login -u $script.USER --password-stdin"
        }
    }

    def dockerPush(String imageName) {
        script.sh "docker push $imageName"
    }
}

dockerLogin.groovy

#!/usr/bin/env groovy

import com.example.Docker

def call() {
    return new Docker(this).dockerLogin()
}

dockerPush.groovy

#!/usr/bin/env groovy

import com.example.Docker

def call(String imageName) {
    return new Docker(this).dockerPush(imageName)
}

Jenkinsfile

stage("build and push image") {
    steps {
        script {
            buildImage "alfredasare/devops-demo-app:jma-3.0"
            dockerLogin()
            dockerPush 'alfredasare/devops-demo-app:jma-3.0'
        }
    }
}

Project Scoped Shared Library

  • Overriding the version of a Global shared Library
@Library('jenkins-shared-library@2.0')_
  • Project scoped

Jenkinsfile

#!/usr/bin/env groovy

// @Library('jenkins-shared-library')_

library identifier: 'jenkins-shared-library@main', retriever: modernSCM(
    [
        $class: 'GitSCMSource',
        remote: 'git@github.com:alfredasare/jenkins-shared-library.git',
        credentialsId: 'github-credentials'
    ]
)

def gv

Webhooks trigger pipeline jobs automatically

  • How to trigger Jenkins Build Jobs
    • Manual
      • Use case may be for production pipelines where you want to ensure certain conditions are met before triggering the build for production
    • Automatic
      • Trigger automatically when changes happen in the git repository
      • Jenkins and Gitlab/GitHub need to be configured for that
    • Scheduling
      • Trigger a job on scheduled times
      • For long-running tests. e.g. Selenium tests

Configure Automatic Triggering of Jenkins Job

  • Jenkins config
    • Install Gitlab (for Gitlab)
    • Github
      • In pipeline, check GitHub hook trigger for GITScm polling
  • GitHub config

Configure Automatic triggering of Jenkins Jobs for Multi-Branch Pipeline

  • Use Multibranch Scan Webhook Trigger plugin
  • Go to configuration
    • Tick Scan by webhook
    • Add trigger token
      • Copy URL with token to github: JENKINS_URL/multibranch-webhook-trigger/invoke?token=[Trigger token]

Dynamically increment application version In Jenkins Pipeline - 1

  • Each build tool or package manager tool keeps a version
    • Gradle - build.gradle
    • Maven - pom.xml
    • npm - package.json
  • You decide how to version your app
  • Most common
    • 3 parts of a version e.g. 4.2.1
    • Major
      • Contains big changes
      • Breaking changes
      • Not backward compatible
    • Minor
      • New but backward-compatible changes, API features
    • Patch
      • Minor changes and bug fixes, doesn't change API
    • Can add suffixes to add more information
      • eg. SNAPSHOT: For development versions
      • RC - Release candidate
      • RELEASE
  • How to increment the version
    • Increment the version in e.g. pom.xml file or package.json
    • You should increment the type automatically
    • Automatically increase version inside the build automation
    • Build Tools have commands to increment the version

Increment version locally (Maven)

mvn build-helper:parse-version versions:set -DnewVersion=\${parsedVersion.majorVersion}.\${parsedVersion.minorVersion}.\${parsedVersion.nextIncrementalVersion} versions:commit
  • parsedVersion has six values
    • majorVersion
    • nextMajorVersion
    • minorVersion
    • nextMinorVersion
    • incrementalVersion
    • nextIncrementalVersion
  • Same concept runs through for other package manager tools
  • You run this command in your pipeline and not locally

Increment Application Version on Jenkins (maven Project)

Jenkinsfile

stage("increment version") {
    steps {
        script {
            gv.incrementVersion()
        }
    }
}
stage("build jar") {
    steps {
        script {
            gv.buildJar()
        }
    }
}

script.groovy

def incrementVersion() {
    echo "incrementing app version..."
    sh "mvn build-helper:parse-version versions:set -DnewVersion=\\\${parsedVersion.majorVersion}.\\\${parsedVersion.minorVersion}.\\\${parsedVersion.nextIncrementalVersion} versions:commit"
}

def buildJar() {
    echo "building the application..."
    sh 'mvn package'
}

def buildImage() {
    echo "building the docker image..."
    withCredentials([usernamePassword(credentialsId: 'docker-hub-repo', passwordVariable: 'PASS', usernameVariable: 'USER')]) {
        sh 'docker build -t alfredasare/devops-demo-app:jma-2.0 .'
        sh "echo $PASS | docker login -u $USER --password-stdin"
        sh 'docker push alfredasare/devops-demo-app:jma-2.0'
    }
}

def deployApp() {
    echo "deploying the application to EC2"
}

return this

Docker Image Version

  • Use application version for Docker Image version

Jenkinsfile

stage("increment version") {
    steps {
        script {
            echo "incrementing app version..."
            sh "mvn build-helper:parse-version versions:set -DnewVersion=\\\${parsedVersion.majorVersion}.\\\${parsedVersion.minorVersion}.\\\${parsedVersion.nextIncrementalVersion} versions:commit"
            def matcher = readFile('pom.xml') =~ '<version>(.+)</version>'
            def version = matcher[0][1]
            env.IMAGE_NAME = "$version-$BUILD_NUMBER"
        }
    }
}

script.groovy

def buildJar() {
    echo "building the application..."
    sh 'mvn package'
}

def buildImage() {
    echo "building the docker image..."
    withCredentials([usernamePassword(credentialsId: 'docker-hub-repo', passwordVariable: 'PASS', usernameVariable: 'USER')]) {
        sh "docker build -t alfredasare/devops-demo-app:${IMAGE_NAME} ."
        sh "echo $PASS | docker login -u $USER --password-stdin"
        sh "docker push alfredasare/devops-demo-app:${IMAGE_NAME}"
    }
}

def deployApp() {
    echo "deploying the application to EC2"
}

return this

Replace New Version In Dockerfile

Dockerfile

FROM openjdk:8-jre-alpine

EXPOSE 8080

COPY ./target/java-maven-app-*.jar /usr/app/
WORKDIR /usr/app

CMD java -jar java-maven-app-*.jar

script.groovy

def buildJar() {
    echo "building the application..."
    sh 'mvn clean package'
}
  • Access envs in shell with ${ENV_NAME}

Dynamically increment application version in Jenkinsfile Pipeline 2

Commit version bump from Jenkins

  • The version bump only happened in Jenkins
  • It's never committed to GitHub
  • If a commit is made, devs need to pull changes before committing their code again
  • Install SSH Agent jenkins plugin

Jenkinsfile

stage("commit version update") {
    steps {
        script {
            sshagent(credentials: ['github-credentials']) {
                // Need to set this once
                // Can also ssh into Jenkins server to set it
                sh 'git config --global user.email "jenkins@example.com"'
                sh 'git config --global user.name "jenkins"'

                sh "git status"
                sh "git branch"
                sh "git config --list"

                sh "git remote set-url origin git@github.com:alfredasare/java-maven-app.git"
                sh "git add ."
                sh 'git commit -m "ci: version bump"'
                sh "git push origin HEAD:jenkins-jobs"
            }
        }
    }
}
  • When Jenkins checks out the code from Git, it doesn't checkout the branch, it checks out the commit hash
  • It checks out the last commit in that branch
sh "git push origin HEAD:jenkins-jobs"
  • Push all commits from that branch into jenkins-jobs

Ignore Jenkins Commit for Jenkins Pipeline Trigger

  • We'll end up with a loop of commit + Pipeline Triggering
  • Solution
    • Detect that commit was made from Jenkins
    • Ignore trigger when commit is from Jenkins
  • Install Ignore Committer Strategy
    • Add Build strategies on configure page
  • Add Ignore committer strategy
  • Add jenkins email to blacklist e.g. jenkins@example.com
  • Tick the checkbox to allow commits from all other authors