I have been looking for the opportunity for document deployment of a small application security lab that I created last year in my home. I want to be able to run SAST SCA and other application tools. To see how they behave and how they work with the CI/CD. I will be using Kubernetes for this project. Rationale is that Kubernetes allows IAC to deploy these containers. Which makes the deployment a lot more easier and simpler than having to deploy it from on host, dealing with dependencies and other “FUN” configurations. I will add detailed configuration steps later on how everything connects.

Documentation used for this:

Applications to be use:

  1. Jenkins
  2. GITLeaks
  3. SonarQube
  4. Dependency-Check (OWASP)
  5. JuiceShop (OWASP)
  6. DefectDojo (OWASP)

Deploying Jenkins#

Jenkins is a popular CI/CD automation server that lets you build, test, and deploy code automatically. A lot of software development companies have it but unfortunately they do not have it configured or have not played enough to utilize the full features that the application provides.
First let create 3 yml files. These files will provide the persistent volume, a service NodePort object that will allow us connect to the application and the deployment.

  • jenkins-pvc.yaml -> will contain the configuration for the persistent volume.
  • jenkins-deployment.yaml -> will contain the configuration for the deployment.
  • jenkins-service.yaml -> will contain the NodePort Service to allow connections to the device.
  • jenkins-rbac.yaml -> This will create a service account, Role and Rolebinding that will contain rules to allow Jenkins to connect to kubernetes and deploy pods on demand.

For deploying it we will use the command kubectl apply -f <file.yaml>. Here are my yaml files that I used for the project.

jenkins-pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jenkins-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi

jenkins-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-jenkins
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jenkins
  template:
    metadata:
      labels:
        app: jenkins
    spec:
      serviceAccountName: jenkins
      containers:
      - name: jenkins
        image: jenkins/jenkins:lts
        ports:
        - containerPort: 8080
        volumeMounts:
        - name: jenkins-data
          mountPath: /var/jenkins_home
      volumes:
      - name: jenkins-data
        persistentVolumeClaim:
          claimName: jenkins-pvc

jenkins-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: jenkins
spec:
  type: NodePort
  selector:
    app: jenkins
  ports:
  - name: http
    port: 8080
    targetPort: 8080
    nodePort: 31000
  - name: agent
    port: 50000
    targetPort: 50000

jenkins-rbac.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins
  namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: jenkins
  namespace: default
rules:
- apiGroups: [""]
  resources: ["pods", "pods/log", "pods/exec"]
  verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: jenkins
  namespace: default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: jenkins
subjects:
- kind: ServiceAccount
  name: jenkins
  namespace: default

With this done, you should have a fully functional Jenkins server that will maintain the configuration, as you can see in my screenshot below: Image Description

[tip] To get the initial password for Jenkins just run the following:
kubectl exec pod -it -- cat /var/jenkins_home/secrets/initialAdminPassword

Deploying SonarQube#

SonarQube is a SAST tool that scans source code for quality issues, bugs, code smells, and security vulnerabilities.

In the same fashion on how we deployed Jenkins this is how I will be deploying the application. Create three yaml files to hold the IAC of the application.

  • sonarqube-pvc.yaml -> will contain the configuration for the persistent volume. In this one we will have three volumes to work with.
  • sonarqube-deployment.yaml -> will contain the configuration for the deployment.
  • sonarqube-service.yaml -> will contain the NodePort Service to allow connections to the device.

sonarqube-pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: sonarqube-data
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: sonarqube-extensions
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: sonarqube-logs
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

sonarqube-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-sonarqube
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sonarqube
  template:
    metadata:
      labels:
        app: sonarqube
    spec:
      containers:
      - name: sonarqube
        image: sonarqube:lts-community
        ports:
        - containerPort: 9000
        env:
        - name: SONAR_ES_BOOTSTRAP_CHECKS_DISABLE
          value: "true"
        volumeMounts:
        - name: sonarqube-data
          mountPath: /opt/sonarqube/data
        - name: sonarqube-extensions
          mountPath: /opt/sonarqube/extensions
        - name: sonarqube-logs
          mountPath: /opt/sonarqube/logs
      volumes:
      - name: sonarqube-data
        persistentVolumeClaim:
          claimName: sonarqube-data
      - name: sonarqube-extensions
        persistentVolumeClaim:
          claimName: sonarqube-extensions
      - name: sonarqube-logs
        persistentVolumeClaim:
          claimName: sonarqube-logs

sonarqube-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: sonarqube
spec:
  type: NodePort
  selector:
    app: sonarqube
  ports:
  - port: 9000
    targetPort: 9000
    nodePort: 31001

Creating a Git-Leaks pipeline#

Gitleaks allow the scanning of hardcoded passwords or secrets in the submitted code. I am going to have jenkins create a new Pod in Kubernetes to have the pod run when its needed. This is the main reason we created that Service Account, Role and Role Binding for Jenkins earlier. Before creating the pipeline you must create an Cloud item in Jenkins for Kubernetes.

test-gitleaks pipeline

pipeline {
    agent {
        kubernetes {
            yaml '''
apiVersion: v1
kind: Pod
spec:
  containers:
  - name: gitleaks
    image: zricethezav/gitleaks:latest
    command:
    - sleep
    args:
    - 99d
'''
        }
    }
    stages {
        stage('Secret Scan') {
            steps {
                container('gitleaks') {
                    sh 'gitleaks detect --source=. --report-format=json --report-path=gitleaks-report.json --no-git || true'
                }
                archiveArtifacts artifacts: 'gitleaks-report.json', allowEmptyArchive: true
            }
        }
    }
}

Creating a OWASP Dependency-Check pipeline#

OWASP Dependency-Check is a software component scanner that will let you know of vulnerable dependencies on the code. This is simpler as all you need is to create a pipeline.

[tip] Note that when this runs for the first time it will download the vulnerability database from nist.
You might recieve the warning:

[WARN] An NVD API Key was not provided - it is highly recommended to use an NVD API key as the update can take a VERY long time without an API Key

If you want you can navigate to https://nvd.nist.gov/developers/request-an-api-key fill out the form and get a key. You can add this to Jenkins to make the download faster.

Here is my pipeline script I created to test. As you can see I am also sending my report to my SonarQube. SonarQube will receive the XML read it, import the findings and display them in the dash board.

pipeline {
    agent any
    stages {
        stage('Dependency Check') {
            steps {
                dependencyCheck additionalArguments: '--scan ./ --format HTML --format XML', odcInstallation: 'OWASP SCA'
                dependencyCheckPublisher pattern: 'dependency-check-report.xml'
            }
        }
        stage('SonarQube Analysis') {
            steps {
                script {
                    def scannerHome = tool 'SonarScanner'
                    withSonarQubeEnv('sonarqube') {
                        sh "${scannerHome}/bin/sonar-scanner -Dsonar.projectKey=test-app -Dsonar.sources=. -Dsonar.dependencyCheck.reportPath=dependency-check-report.xml"
                    }
                }
            }
        }
    }
}

Deploying OWASP Juice Shop#

Juice Shop is a vulnerable web application created by OWASP for training and testing security tools. For this we will deploy in Kubernetes for ease. No PVC is needed but we will need to create two yaml files to help for the deployment. We will create a deployment and a service items.

  • juiceshop-deployment.yaml -> will provide the deployment configuration.
  • juiceshop-service.yaml -> will provice the service network configuration to be use

Here are the configurations I am using for my setup:
juiceshop-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: juiceshop
spec:
  replicas: 1
  selector:
    matchLabels:
      app: juiceshop
  template:
    metadata:
      labels:
        app: juiceshop
    spec:
      containers:
      - name: juiceshop
        image: bkimminich/juice-shop
        ports:
        - containerPort: 3000

juiceshop-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: juiceshop
spec:
  type: NodePort
  selector:
    app: juiceshop
  ports:
  - port: 3000
    targetPort: 3000
    nodePort: 31002

Creating a ZAP pipeline#

OWASP ZAP (Zed Attack Proxy) is a free, open-source DAST (Dynamic Application Security Testing) tool. ZAP provides runtime security testing by simulating attacks against deployed applications, finding vulnerabilities that only appear when the app is actually running (like authentication bypasses, session management issues, etc.). This pipeline will run a scan against the OWASP Juice shop and provide the reports.

ZAP Pipeline

pipeline {
    agent {
        kubernetes {
            yaml '''
apiVersion: v1
kind: Pod
spec:
  containers:
  - name: zap
    image: ghcr.io/zaproxy/zaproxy:stable
    command:
    - sleep
    args:
    - 99d
'''
        }
    }
    stages {
        stage('DAST Scan') {
            steps {
                container('zap') {
                    sh '''
                        zap-baseline.sh -t http://juiceshop:3000 -r zap-report.html || true
                    '''
                }
                publishHTML([
                    allowMissing: false,
                    alwaysLinkToLastBuild: true,
                    keepAll: true,
                    reportDir: '.',
                    reportFiles: 'zap-report.html',
                    reportName: 'ZAP DAST Report'
                ])
            }
        }
    }
}

Deploy DefectDojo#

DefectDojo is a security orchestration and vulnerability management tool it provides centralized vulnerability management across your entire security toolchain, giving security and development teams a single source of truth for all security findings and their remediation status. Just the same as before we will create one service and run the helm command to install the rest of the application: helm install defectdojo defectdojo/defectdojo --set redis.auth.enabled=false --set postgresql.auth.postgresPassword=defectdojo --set createSecret=true --set createPostgresqlSecret=true

  • defectdojo-service.yaml -> contains all the configuration to allow the service to be reachable.

Here are my configurations:

defectdojo-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: defectdojo-external
spec:
  type: NodePort
  selector:
    app.kubernetes.io/name: defectdojo
    app.kubernetes.io/component: django
  ports:
  - port: 8080
    targetPort: 8080
    nodePort: 31003

Integrating tools in to Jenkins#

Integrating these tools with Jenkins is quite simple. The first item in the books will be adding the needed credentials. I will be using GitHub as my VCS. For this test we will be using secret text. Please note that is recommended to use GitHub App. But for the sake of training we will be using secret text.

Setting the credentials in GitHub.#

  1. Navigate and log on to GitHub.
  2. Go to Settings.
  3. Go to the bottom of settings and click on “Developer Settings.”
  4. For this scenario click on Personal access tokens and click on Fine-Grained tokens.
  5. Click generate token.
  6. Fill the form with the requested data.
  7. Make sure to add an expiration date prior clicking generate token.
  8. For repository permissions I will be using the “Read access to code, metadata, and pull requests” option and “Read and Write access to commit statuses and repository hooks”.
  9. Click Generate token.
  10. Save the generated token.

Setting the credentials in SonarQube.#

  1. Navigate and log on to SonarQube.
  2. Once in the dashboard click on your profile picture (top right of the Screen) then click “My Account”.
  3. Go to security tab.
  4. For this scenario we will be using a global token.
  5. Generate a new token.
  6. Save the token.

Setting OWASP Dependency-Check on Jenkins.#

  1. Log on to Jenkins
  2. Navigate to Manage Jenkins -> Plugins
  3. Click on Available plugins.
  4. Search for “OWASP Dependency-Check”
  5. Click the check box for install and click installed. Note: There might be need to restart the Jenkins server after the installation.

While recommended not necessary. These keys will allow faster downloads from the NVD database. If not a scan can take up to 30+ minutes.

  1. Navigate to: https://nvd.nist.gov/developers/request-an-api-key
  2. Request free API key
  3. Check email for key
  4. In Jenkins: Manage Jenkins -> System
  5. Find Dependency-Check section
  6. Paste API key
  7. Click Save

Setting the credentials in Jenkins.#

Lets add the GitHub credentials.

  1. Navigate and log on to Jenkins
  2. Go to Manage Jenkins -> Credentials
  3. Click on System (global)
  4. Click on “Add Credentials”
  5. Select Secret Text/
  6. Add the secret token that you save from GitHub.
  7. Add an ID and description for ease of use.

Adding SonarQube.

  1. Navigate and log on to Jenkins
  2. Go to Manage Jenkins -> Credentials
  3. Click on System (global)
  4. Click on “Add Credentials”
  5. Select Secret Text/
  6. Add the secret token that you save from SonarQube.
  7. Add an ID and description for ease of use.

Configuring SonarQube#

  1. Navigate and log on to Jenkins.
  2. Go to Manage Jenkins -> System
  3. Scroll down until you see SonarQube Servers. If the item is not there you will need to download the SonarQube Plugin. From the Jenkins Marketplace.
  4. Add the name of what you want to call this server setup.
  5. Add the server URL.
  6. Lastly Server authentication token.
  7. Click Save.

Sample Jenkinsfile for triggering the builds:#

Here is a sample Jenkinsfile I created for scanning and testing a WebGoat fork. I use this to make sure that the applications are properly funtioning.

pipeline {  
    agent {  
        kubernetes {  
            yaml '''  
apiVersion: v1  
kind: Pod  
spec:  
    containers:        - name: gitleaks          image: zricethezav/gitleaks:latest          command:          - sleep          args:          - 99d        - name: maven          image: maven:3.9-eclipse-temurin-21          command:          - sleep          args:          - 99d'''  
        }  
    }  
  
    environment {  
        APP_NAME = 'WebGoat'  
        SONARQUBE_SERVER = 'sonarqube'  
        BUILD_TIMESTAMP = sh(script: "date '+%Y-%m-%d_%H-%M-%S'", returnStdout: true).trim()  
    }  
  
    stages {  
  
        stage('Checkout') {  
            steps {  
                echo 'STAGE: Checkout Source Code'  
                echo "Branch: ${env.BRANCH_NAME}"  
                echo "Build: ${env.BUILD_NUMBER}"  
  
                sh '''  
                    echo "Working on Directory: $(pwd)"                    echo "Git Status"                    git status                    echo "Git Log (last commit)"                    git log -1                '''            }  
        }  
  
        stage('Secret Scanning') {  
            steps {  
                echo "Stage: Secret Scanning"  
                container('gitleaks') {  
                    script {  
                        def exitCode = sh(  
                            script: 'gitleaks detect --source=. --report-format=json --report-path=gitleaks-report.json --no-git',  
                            returnStatus: true  
                        )  
  
                        if (exitCode == 0) {  
                            echo "No secrets found"  
                        } else if (exitCode == 1) {  
                            error 'SECRET DETECTED! Check gitleaks-report.json'  
                        } else {  
                            error "Gitleaks failed with exit code: ${exitCode}"  
                        }  
                    }  
                }  
            }  
            post {  
                always {  
                    archiveArtifacts artifacts: 'gitleaks-report.json', allowEmptyArchive: true  
                }  
            }  
        }  
  
        stage('Build') {  
            steps {  
                container('maven') {  
                    echo 'Stage: Building Application'  
                    //sh '''  
                    //    mvn clean package -DskipTests                    //'''                      //Trying to fix issue what will not allow the build to continue.                      sh '''  
                          mvn clean package -DskipTests -Dmaven.compiler.release=21                      '''                    echo 'Build completed'  
                }  
            }  
        }  
  
        stage('Unit Test') {  
            steps {  
                container('maven') {  
                    echo 'Stage: Running Unit Test'  
                    sh '''  
                        mvn test                    '''                    echo 'Unit test passed'  
                }  
            }  
            post {  
                always {  
                    junit '**/target/surefire-reports/*.xml'  
                }  
            }  
        }  
  
        stage('Parallel Scans') {  
            parallel {  
                stage('SCA - Dependency Check') {  
                    steps {  
                        container('maven') {  
                            echo "Parallel Stage: SCA"  
  
                            dependencyCheck additionalArguments: '--scan ./ --format HTML --format XML', odcInstallation: 'OWASP SCA'  
                            dependencyCheckPublisher pattern: 'dependency-check-report.xml'  
                        }  
                    }  
                }  
  
                stage('SAST - SonarQube') {  
                    steps {  
                        container('maven') {  
                            echo "Parallel Stage: SAST"  
  
                            script {  
                                withSonarQubeEnv("${SONARQUBE_SERVER}") {  
                                    sh '''  
                                        mvn sonar:sonar \                                        -Dsonar.projectKey=${APP_NAME} \                                        -Dsonar.projectName=${APP_NAME} \                                        -Dsonar.java.binaries=target/classes                                    '''                                }  
  
                                echo "SAST scan completed"  
                            }  
                        }  
                    }  
                }  
            }  
        }  
  
        stage('Quality Gate') {  
            steps {  
                echo "Stage: Waiting for SonarQube Quality Gate"  
  
                script {  
                    timeout(time: 5, unit: 'MINUTES') {  
                        def qg = waitForQualityGate()  
                        if (qg.status != 'OK') {  
                            error "Quality Gate failed: ${qg.status}"  
                        }  
                    }  
                }  
                echo 'Quality Gate passed'  
            }  
        }  
  
        stage('Package') {  
            steps {  
                container('maven') {  
                    echo "Stage: Package Application"  
                    sh '''  
                        echo "Packaging WebGoat Jar...."                        ls -lh target/*.jar                    '''                    archiveArtifacts artifacts: 'target/*.jar', fingerprint: true  
                    echo "Package completed"  
                }  
            }  
        }  
  
        stage('Deploy to Dev') {  
            steps {  
                echo "Stage: Deploy to Development Environment"  
                // Not doing anything just wanted to have this one to look good in the Jenkins job  
            }  
        }  
    }  
  
    post {  
        always {  
            echo "Pipeline completed"  
            cleanWs()  
        }  
        success {  
            echo "Pipeline completed successfully."  
        }  
        failure {  
            echo "Pipeline Failed"  
        }  
    }  
}