Pipeline Architecture¶

Every project repo contains a Jenkinsfile at its root. Jenkins discovers these automatically via the GitHub Organization Folder job and runs them on every push and pull request.

Where things live¶

jenkins-config/                  ← you are here
├── casc.yml                     ← Docker agent template, credentials, shared library registration
├── Dockerfile                   ← Jenkins controller image with all plugins baked in
└── docker-compose.yml           ← Runs the controller + mounts Docker socket

jenkins-shared-library/          ← separate repo
└── vars/
    ├── runPytest.groovy         ← shared Test + Archive stage logic
    ├── deployMkdocs.groovy      ← shared Deploy Docs stage logic
    └── generateChangelog.groovy ← shared Update Changelog stage logic

<any-project-repo>/
├── Jenkinsfile                  ← calls shared library steps; defines what runs
├── cliff.toml                   ← git-cliff config for changelog generation
└── src/ tests/ etc.

Build agent¶

Each build runs inside a fresh Docker container spun up on demand and discarded after the build. The Jenkins controller never runs build code — it only orchestrates. The Docker socket mount (/var/run/docker.sock) allows the controller to launch agent containers via Docker Desktop.

Agent selection¶

There are three ways a project declares its build environment, in order of preference:

Scenario	Agent declaration	casc.yml change?
Single language, standard runtime	`agent { label 'python-3.14' }`	Never
Multi-language, different stages	`agent none` + per-stage labels	Never
Multi-language, same stage	`agent { dockerfile { filename 'Dockerfile.ci' } }`	Never
New commonly-used language	Add template to `casc.yml`	Yes — one time

Pre-defined labels (configured in casc.yml): python-3.14, node-20, java-21, go-1.22, dotnet-8, ruby-3.3

Dockerfile.ci — for projects that need runtimes not covered by a single label, or need a custom combination. Placed in the project repo root, built and cached by Docker on first run:

# Example: Python backend + Node frontend in one image
FROM python:3.14
RUN curl -fsSL https://deb.nodesource.com/setup_20.x | bash - \
    && apt-get install -y nodejs

Auto-detection via shared library¶

When using the shared library, detectAgent() selects the agent automatically based on project files — no agent declaration needed in the Jenkinsfile:

File present	Agent used
`Dockerfile.ci`	`dockerfile { filename 'Dockerfile.ci' }`
`pyproject.toml` / `setup.py`	`label 'python-3.14'`
`package.json`	`label 'node-20'`
`pom.xml` / `build.gradle`	`label 'java-21'`
`go.mod`	`label 'go-1.22'`
`.csproj` / `.sln`	`label 'dotnet-8'`
`Gemfile`	`label 'ruby-3.3'`

Stages¶

Install¶

Lives in: project Jenkinsfile (or runPytest shared step) Agent: python:3.14 container What it does: pip install -e ".[dev]" — installs the project and its dev dependencies into the ephemeral agent

Test¶

Lives in: project Jenkinsfile (or runPytest shared step) Agent: python:3.14 container What it does: Runs pytest, captures test-results.xml and test-output.txt Failure handling: catchError(buildResult: 'UNSTABLE') — build continues even if tests fail so artifacts are still archived Post-stage: JUnit plugin publishes test-results.xml as a trend graph in Jenkins

Key environment variables injected:

Variable	Value	Purpose
`CI`	`true`	Suppresses local archive creation in `conftest.py`
`VERSION`	`v${BUILD_NUMBER}`	Stamped into `test-output.txt` and `test-results.xml`

Rename¶

Lives in: project Jenkinsfile Agent: python:3.14 container Condition: only runs if tmp-test-files/ is non-empty What it does: Runs python -m filename_ingest tmp-test-files draft to apply filename transformations to test-generated files

Archive Artifacts¶

Lives in: project Jenkinsfile (or runPytest shared step) Agent: python:3.14 container Condition: only runs if tmp-test-files/ exists What it does: Archives tmp-test-files/**, test-output.txt, test-results.xml into Jenkins build storage Retention: 30 days (configured via "Discard Old Builds" on the job)

Update Changelog¶

Lives in: project Jenkinsfile (or generateChangelog shared step) Agent: python:3.14 container (uses Docker socket to pull orhunp/git-cliff) Condition: main branch only What it does: 1. Runs git-cliff via Docker to regenerate CHANGELOG.md from Conventional Commits history 2. Commits CHANGELOG.md back to main with [skip ci] in the message to prevent a build loop 3. Skips the commit if CHANGELOG.md has no changes

cliff.toml in each project repo controls the changelog format and filters out noise commits.

Deploy Docs¶

Lives in: project Jenkinsfile (or deployMkdocs shared step) Agent: python:3.14 container Condition: main branch only What it does: 1. pip install -e ".[docs]" 2. Sets git user config 3. Rewrites the remote URL with a Jenkins CI App token via withCredentials([gitHubApp(...)]) 4. Runs mkdocs gh-deploy --force to push built docs to the gh-pages branch

Full Jenkinsfile (initial — before shared library)¶

pipeline {
    agent {
        docker { image 'python:3.14' }
    }
    environment {
        TMP_FILES_DIR = 'tmp-test-files'
        VERSION       = "v${BUILD_NUMBER}"
        CI            = 'true'
    }
    stages {
        stage('Install') {
            steps {
                sh 'pip install -e ".[dev]"'
            }
        }
        stage('Test') {
            steps {
                catchError(buildResult: 'UNSTABLE', stageResult: 'UNSTABLE') {
                    sh 'pytest'
                }
            }
            post {
                always { junit 'test-results.xml' }
            }
        }
        stage('Rename') {
            when {
                expression {
                    return sh(script: 'ls tmp-test-files 2>/dev/null | wc -l',
                              returnStdout: true).trim() != '0'
                }
            }
            steps {
                sh 'python -m filename_ingest tmp-test-files draft'
            }
        }
        stage('Archive Artifacts') {
            when {
                expression { fileExists('tmp-test-files') }
            }
            steps {
                archiveArtifacts artifacts: 'tmp-test-files/**, test-output.txt, test-results.xml',
                                 fingerprint: true
            }
        }
        stage('Update Changelog') {
            when { branch 'main' }
            steps {
                sh 'docker run --rm -v $(pwd):/app orhunp/git-cliff:latest --output CHANGELOG.md'
                withCredentials([gitHubApp(credentialsId: 'jenkins-ci-app', variable: 'GH_TOKEN')]) {
                    sh '''
                        git config user.email "BrettT@aptora.com"
                        git config user.name "Jenkins"
                        git remote set-url origin https://x-access-token:${GH_TOKEN}@github.com/<your-github-username>/<repo>.git
                        if ! git diff --quiet CHANGELOG.md; then
                            git add CHANGELOG.md
                            git commit -m "chore: update changelog [skip ci]"
                            git push origin main
                        fi
                    '''
                }
            }
        }
        stage('Deploy Docs') {
            when { branch 'main' }
            steps {
                sh 'pip install -e ".[docs]"'
                withCredentials([gitHubApp(credentialsId: 'jenkins-ci-app', variable: 'GH_TOKEN')]) {
                    sh '''
                        git config user.email "BrettT@aptora.com"
                        git config user.name "Jenkins"
                        git remote set-url origin https://x-access-token:${GH_TOKEN}@github.com/<your-github-username>/<repo>.git
                        mkdocs gh-deploy --force
                    '''
                }
            }
        }
    }
    post {
        always { cleanWs() }
    }
}

Thin Jenkinsfile (after shared library is set up)¶

@Library('shared') _

pipeline {
    agent { docker { image 'python:3.14' } }
    stages {
        stage('Test')             { steps { runPytest() } }
        stage('Update Changelog') { when { branch 'main' }
                                    steps { generateChangelog() } }
        stage('Deploy Docs')      { when { branch 'main' }
                                    steps { deployMkdocs() } }
    }
}

Trigger flow¶

Push / PR to GitHub
       ↓
Jenkins CI App sends webhook to JENKINS_URL/github-webhook/
       ↓
Jenkins GitHub Branch Source plugin receives it
       ↓
Matching job triggered (branch or PR build)
       ↓
python:3.14 agent container spun up
       ↓
Stages run → results posted back to GitHub commit status
       ↓
Agent container destroyed, workspace cleaned