Configure Multi-Branch Pipeline in Jenkins [Guide]

Jenkins multibranch pipelines automatically discover branches in a Git repository and create a pipeline job for each branch that contains a Jenkinsfile. This removes the need to manually create and manage separate jobs for every feature branch, hotfix, or release branch – Jenkins handles it all through source control.

Original content from computingforgeeks.com - post 96728

This guide covers setting up a Jenkins multibranch pipeline from scratch. We walk through plugin installation, Jenkinsfile creation, branch source configuration for GitHub/GitLab/Bitbucket, webhook triggers, parallel stages, branch-specific deploy logic, and stale branch cleanup.

Prerequisites

Before starting, make sure you have the following in place:

Jenkins 2.x installed and running – if you need to set this up, see our guide on installing Jenkins
Admin access to your Jenkins instance
A Git repository (GitHub, GitLab, or Bitbucket) with at least one branch
Git installed on the Jenkins server
Network access from Jenkins to your Git provider (port 443 for HTTPS, port 22 for SSH)

Step 1: Install Required Jenkins Plugins

Multibranch pipelines need a few core plugins. Most Jenkins installations include these by default, but verify they are installed and up to date.

Navigate to Manage Jenkins > Plugins > Available plugins and install these if missing:

Pipeline – core pipeline engine for Jenkinsfile execution
Pipeline: Multibranch – enables multibranch pipeline job type
Git – Git SCM integration
GitHub Branch Source – for GitHub repositories
GitLab Branch Source – for GitLab repositories
Bitbucket Branch Source – for Bitbucket repositories
Blue Ocean (optional) – modern pipeline visualization UI

You can also install plugins from the CLI. SSH into your Jenkins server and run:

jenkins-plugin-cli --plugins "workflow-multibranch git github-branch-source gitlab-branch-source blueocean"

After installing plugins, restart Jenkins to activate them:

sudo systemctl restart jenkins

Confirm Jenkins is back up:

sudo systemctl status jenkins

The service should show active (running) within a few seconds.

Step 2: Create a Jenkinsfile in Your Repository

The Jenkinsfile defines your pipeline stages and lives in the root of your Git repository. Jenkins reads this file from each branch to determine what to build and how.

Create a file named Jenkinsfile (no extension) in your repository root with this basic structure:

pipeline {
    agent any

    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }

        stage('Build') {
            steps {
                echo 'Building the application...'
                // Replace with your actual build commands
                sh 'make build || echo "No Makefile, skipping build"'
            }
        }

        stage('Test') {
            steps {
                echo 'Running tests...'
                sh 'make test || echo "No tests defined, skipping"'
            }
        }

        stage('Deploy') {
            when {
                branch 'main'
            }
            steps {
                echo 'Deploying to production...'
                // Add your deployment commands here
            }
        }
    }

    post {
        success {
            echo 'Pipeline completed successfully.'
        }
        failure {
            echo 'Pipeline failed. Check the logs above.'
        }
    }
}

This Jenkinsfile runs Checkout, Build, and Test stages on every branch. The Deploy stage only runs on the main branch thanks to the when { branch 'main' } condition. Commit and push this file to your repository.

Step 3: Create a Multibranch Pipeline Job in Jenkins

With the Jenkinsfile in your repo, create the multibranch pipeline job in Jenkins.

From the Jenkins dashboard, click New Item
Enter a name for the job (e.g., my-app-pipeline)
Select Multibranch Pipeline and click OK
You are taken to the job configuration page where you set up branch sources

The job configuration has three key sections: Branch Sources (where to find your code), Build Configuration (how to find the Jenkinsfile), and Scan Triggers (how often to check for new branches).

Step 4: Configure Branch Sources

Branch sources tell Jenkins where your Git repository lives and how to authenticate. The configuration varies slightly depending on your Git provider.

GitHub Repository

In the job configuration, click Add source > GitHub
Under Credentials, add a GitHub Personal Access Token (PAT) with repo scope. Click Add > Jenkins, select Username with password, enter your GitHub username and the PAT as the password
Set Repository HTTPS URL to your repo URL, e.g., https://github.com/your-org/your-repo.git
Under Behaviors, keep the defaults: Discover branches, Discover pull requests from origin, Discover pull requests from forks

GitLab Repository

Click Add source > GitLab Project
Set the Server URL to your GitLab instance (e.g., https://gitlab.com or your self-hosted URL)
Add credentials using a GitLab Personal Access Token with api and read_repository scopes
Select the Owner (namespace) and Project from the dropdowns

If you run a self-hosted GitLab CE instance, make sure Jenkins can reach it over the network and the SSL certificate is trusted by your Jenkins JVM.

Bitbucket Repository

Click Add source > Bitbucket
Add credentials using a Bitbucket App Password with Repositories: Read permission
Enter the Repository Owner and Repository Name

After configuring any branch source, click Save. Jenkins immediately scans the repository and creates pipeline jobs for every branch that contains a Jenkinsfile.

Step 5: Configure Build Triggers

By default, Jenkins only scans for new branches and changes when you manually trigger a scan. For automated CI/CD, configure webhooks or polling.

Webhook Triggers (Recommended)

Webhooks notify Jenkins immediately when code is pushed. This is faster and more efficient than polling.

For GitHub:

Go to your GitHub repo > Settings > Webhooks > Add webhook
Set Payload URL to https://your-jenkins-url/github-webhook/
Set Content type to application/json
Select Just the push event (or add Pull Request events if needed)
Click Add webhook

For GitLab:

Go to your GitLab project > Settings > Webhooks
Set URL to https://your-jenkins-url/project/your-job-name
Check Push events and Merge request events
Click Add webhook

Make sure your Jenkins instance is accessible from the internet (or from your Git provider’s network). If Jenkins is behind a firewall, open port 8080 (or your configured port) for incoming webhook traffic, or use a reverse proxy on port 443.

SCM Polling (Fallback)

If webhooks are not possible (e.g., Jenkins is on a private network), use SCM polling as a fallback. In the multibranch pipeline configuration under Scan Multibranch Pipeline Triggers, check Periodically if not otherwise run and set an interval.

A 2-minute interval works for active development. For less active repos, 5-15 minutes reduces load on your Git server:

1 minute – high-frequency, adds load to Git server
2 minutes – good balance for active repos
5 minutes – reasonable for most teams
15 minutes – low priority repos

Step 6: Jenkinsfile Best Practices

A well-structured Jenkinsfile makes your pipeline readable, maintainable, and fast. Here are patterns that work well in production.

Parallel Stages

Run independent tasks at the same time to cut total pipeline duration. Use parallel inside a stage to execute multiple sub-stages concurrently:

stage('Test') {
    parallel {
        stage('Unit Tests') {
            steps {
                sh 'make unit-test'
            }
        }
        stage('Integration Tests') {
            steps {
                sh 'make integration-test'
            }
        }
        stage('Lint') {
            steps {
                sh 'make lint'
            }
        }
    }
}

Unit tests, integration tests, and linting run simultaneously. If any parallel stage fails, the entire Test stage fails.

When Conditions for Branch Filtering

The when directive controls which branches execute a stage. This is how you prevent feature branches from triggering production deployments:

stage('Deploy to Staging') {
    when {
        branch 'develop'
    }
    steps {
        sh './deploy.sh staging'
    }
}

stage('Deploy to Production') {
    when {
        branch 'main'
    }
    steps {
        sh './deploy.sh production'
    }
}

You can also match branch patterns with glob expressions:

stage('Deploy to QA') {
    when {
        branch 'release/*'
    }
    steps {
        sh './deploy.sh qa'
    }
}

This runs the QA deploy stage only on branches matching the release/* pattern (e.g., release/1.2, release/2.0).

Environment Variables and Credentials

Define environment variables at the pipeline level and use Jenkins credentials for secrets. Never hardcode passwords or API keys in your Jenkinsfile:

pipeline {
    agent any

    environment {
        APP_ENV = 'production'
        DOCKER_REGISTRY = 'registry.example.com'
        DOCKER_CREDS = credentials('docker-registry-creds')
    }

    stages {
        stage('Build Image') {
            steps {
                sh 'docker build -t $DOCKER_REGISTRY/myapp:$BUILD_NUMBER .'
                sh 'echo $DOCKER_CREDS_PSW | docker login $DOCKER_REGISTRY -u $DOCKER_CREDS_USR --password-stdin'
                sh 'docker push $DOCKER_REGISTRY/myapp:$BUILD_NUMBER'
            }
        }
    }
}

The credentials() helper fetches stored credentials from Jenkins. For username/password credentials, Jenkins automatically creates _USR and _PSW variables. If you are building Jenkins in a Docker container, make sure the Docker socket is mounted or use Docker-in-Docker.

Step 7: Branch-Specific Pipeline Behavior

In real projects, different branches serve different purposes. Feature branches need tests, develop needs staging deployment, and main needs production deployment. Here is a complete Jenkinsfile that handles all three:

pipeline {
    agent any

    environment {
        APP_NAME = 'myapp'
        DOCKER_REGISTRY = 'registry.example.com'
    }

    stages {
        stage('Build') {
            steps {
                sh 'docker build -t $DOCKER_REGISTRY/$APP_NAME:$BRANCH_NAME-$BUILD_NUMBER .'
            }
        }

        stage('Test') {
            steps {
                sh 'docker run --rm $DOCKER_REGISTRY/$APP_NAME:$BRANCH_NAME-$BUILD_NUMBER make test'
            }
        }

        stage('Deploy to Staging') {
            when {
                branch 'develop'
            }
            steps {
                echo 'Deploying to staging environment...'
                sh './scripts/deploy.sh staging $BRANCH_NAME-$BUILD_NUMBER'
            }
        }

        stage('Approval') {
            when {
                branch 'main'
            }
            steps {
                input message: 'Deploy to production?', ok: 'Deploy'
            }
        }

        stage('Deploy to Production') {
            when {
                branch 'main'
            }
            steps {
                echo 'Deploying to production...'
                sh './scripts/deploy.sh production $BRANCH_NAME-$BUILD_NUMBER'
            }
        }
    }

    post {
        always {
            cleanWs()
        }
        failure {
            echo "Build failed on branch: ${env.BRANCH_NAME}"
            // Add notification: Slack, email, etc.
        }
    }
}

Key points in this pipeline:

$BRANCH_NAME is a built-in environment variable Jenkins provides for multibranch pipelines
The input step pauses the pipeline and waits for manual approval before production deploy
cleanWs() in the post block removes workspace files after every run to prevent disk bloat
Feature branches only run Build and Test – no deploy stages execute

Step 8: Clean Up Old Branches

Merged and deleted branches leave behind stale pipeline jobs in Jenkins. Over time, these pile up and consume disk space. Configure automatic cleanup in the multibranch pipeline configuration.

Orphaned Item Strategy

In the job configuration under Orphaned Item Strategy, set:

Discard old items – checked
Days to keep old items – 7 (keeps branch jobs for a week after the branch is deleted from Git)
Max # of old items to keep – 20 (hard cap on stale jobs)

When Jenkins scans the repository and no longer finds a branch, it marks the job as orphaned. After the configured retention period, the orphaned job and its build history are automatically deleted.

Build Rotation in Jenkinsfile

Control how many builds Jenkins keeps per branch directly in the Jenkinsfile using the options block:

pipeline {
    agent any

    options {
        buildDiscarder(logRotator(numToKeepStr: '10', daysToKeepStr: '30'))
        disableConcurrentBuilds()
        timeout(time: 30, unit: 'MINUTES')
    }

    stages {
        // your stages here
    }
}

This keeps the last 10 builds or builds from the last 30 days (whichever is more restrictive). The disableConcurrentBuilds() option prevents the same branch from running two builds at the same time, which avoids race conditions during deployment. The timeout option kills stuck builds after 30 minutes.

Jenkins Multibranch Pipeline Jenkinsfile Directives Reference

This table summarizes the most commonly used Jenkinsfile directives for multibranch pipelines.

Directive	Purpose	Example
`pipeline`	Top-level block wrapping the entire pipeline	`pipeline { agent any; stages { ... } }`
`agent`	Where the pipeline runs (any node, specific label, Docker image)	`agent { docker { image 'node:20' } }`
`stages`	Container for all stage blocks	`stages { stage('Build') { ... } }`
`stage`	Named group of steps	`stage('Test') { steps { sh 'make test' } }`
`steps`	Actual commands to execute	`steps { sh 'npm install' }`
`when`	Conditional execution based on branch, environment, or expression	`when { branch 'main' }`
`parallel`	Run multiple stages concurrently	`parallel { stage('A') {...} stage('B') {...} }`
`environment`	Set environment variables for the pipeline or a stage	`environment { DB_HOST = '10.0.1.5' }`
`options`	Pipeline-level settings (timeout, retry, build rotation)	`options { timeout(time: 30, unit: 'MINUTES') }`
`post`	Actions after pipeline completes (always, success, failure)	`post { failure { mail to: '[email protected]' } }`
`input`	Pause for manual approval	`input message: 'Deploy?', ok: 'Yes'`
`credentials()`	Fetch stored Jenkins credentials as environment variables	`CREDS = credentials('my-cred-id')`
`triggers`	Define automated build triggers	`triggers { pollSCM('H/5 * * * *') }`
`parameters`	Define user-input parameters for the pipeline	`parameters { string(name: 'ENV', defaultValue: 'dev') }`

Conclusion

You now have a fully configured Jenkins multibranch pipeline that automatically discovers branches, runs tests, and deploys based on branch name. The Jenkinsfile lives in your repo alongside your code, so pipeline changes go through the same code review process as application changes.

For production use, add SSL/TLS termination in front of Jenkins, enable LDAP or SSO authentication, set up Slack or email notifications in the post block, and back up your Jenkins home directory regularly. If your CI/CD pipeline builds container images, consider integrating with Gitea or a private registry for tighter control over your artifact lifecycle.