How to Set Up Continuous Deployment Pipelines

Continuous Deployment (CD) automates the software release process, enabling teams to deploy code changes to production quickly, reliably, and with minimal manual intervention. A well-designed CD pipeline reduces deployment risk, increases velocity, and improves software quality. This comprehensive guide will walk you through setting up a production-ready continuous deployment pipeline.

Understanding Continuous Deployment

Before building a pipeline, understand key concepts:

Continuous Integration (CI): Automatically building and testing code on every commit
Continuous Delivery (CD): Code is always in a deployable state
Continuous Deployment: Automated deployment to production after passing tests
Pipeline: Series of automated stages from code to production
Deployment Strategy: Method of releasing changes (blue-green, canary, rolling)

Prerequisites

Before setting up your CD pipeline, ensure you have:

Version control system (Git)
CI/CD platform (GitHub Actions, GitLab CI, Jenkins, CircleCI)
Automated test suite
Deployment target (cloud provider, servers, Kubernetes)
Monitoring and logging infrastructure

Step 1: Design Your Pipeline Architecture

Plan your pipeline stages before implementation.

Basic Pipeline Stages

# Typical CD pipeline stages
1. Source (Code Checkout)
2. Build (Compile, Package)
3. Test (Unit, Integration, E2E)
4. Security Scan (SAST, Dependency Check)
5. Deploy to Staging
6. Smoke Tests
7. Deploy to Production
8. Monitor and Verify

Pipeline Design Considerations

Fail Fast: Run quick tests first
Parallel Execution: Run independent stages concurrently
Environment Parity: Make environments similar
Rollback Strategy: Plan for quick rollbacks
Manual Gates: Add approvals for critical stages

Step 2: Set Up GitHub Actions Pipeline

GitHub Actions provides integrated CI/CD for GitHub repositories.

Create Workflow File

## .github/workflows/deploy.yml
name: Continuous Deployment Pipeline

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

env:
  NODE_VERSION: '18.x'
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

jobs:
  # Build and Test
  build-and-test:
    runs-on: ubuntu-latest
    
    steps:
    - name: Checkout code
      uses: actions/checkout@v4
    
    - name: Setup Node.js
      uses: actions/setup-node@v4
      with:
        node-version: ${{ env.NODE_VERSION }}
        cache: 'npm'
    
    - name: Install dependencies
      run: npm ci
    
    - name: Run linter
      run: npm run lint
    
    - name: Run unit tests
      run: npm test -- --coverage
    
    - name: Upload coverage reports
      uses: codecov/codecov-action@v3
      with:
        files: ./coverage/lcov.info
    
    - name: Build application
      run: npm run build
    
    - name: Archive build artifacts
      uses: actions/upload-artifact@v3
      with:
        name: dist
        path: dist/

  # Security Scanning
  security-scan:
    runs-on: ubuntu-latest
    needs: build-and-test
    
    steps:
    - name: Checkout code
      uses: actions/checkout@v4
    
    - name: Run dependency check
      run: npm audit --audit-level=high
    
    - name: Run SAST scan
      uses: github/codeql-action/analyze@v2

  # Build and Push Docker Image
  build-image:
    runs-on: ubuntu-latest
    needs: [build-and-test, security-scan]
    permissions:
      contents: read
      packages: write
    
    steps:
    - name: Checkout code
      uses: actions/checkout@v4
    
    - name: Log in to Container Registry
      uses: docker/login-action@v3
      with:
        registry: ${{ env.REGISTRY }}
        username: ${{ github.actor }}
        password: ${{ secrets.GITHUB_TOKEN }}
    
    - name: Extract metadata
      id: meta
      uses: docker/metadata-action@v5
      with:
        images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
        tags: |
          type=ref,event=branch
          type=sha,prefix={{branch}}-
          type=semver,pattern={{version}}          
    
    - name: Build and push Docker image
      uses: docker/build-push-action@v5
      with:
        context: .
        push: true
        tags: ${{ steps.meta.outputs.tags }}
        labels: ${{ steps.meta.outputs.labels }}

  # Deploy to Staging
  deploy-staging:
    runs-on: ubuntu-latest
    needs: build-image
    environment:
      name: staging
      url: https://staging.example.com
    
    steps:
    - name: Checkout code
      uses: actions/checkout@v4
    
    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v4
      with:
        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        aws-region: us-east-1
    
    - name: Deploy to ECS Staging
      run: |
        aws ecs update-service \
          --cluster staging-cluster \
          --service web-service \
          --force-new-deployment        
    
    - name: Wait for deployment
      run: |
        aws ecs wait services-stable \
          --cluster staging-cluster \
          --services web-service        

  # Integration Tests on Staging
  integration-tests:
    runs-on: ubuntu-latest
    needs: deploy-staging
    
    steps:
    - name: Checkout code
      uses: actions/checkout@v4
    
    - name: Run integration tests
      run: |
        npm install
        npm run test:integration        
      env:
        TEST_URL: https://staging.example.com
        API_KEY: ${{ secrets.STAGING_API_KEY }}
    
    - name: Run smoke tests
      run: |
        curl -f https://staging.example.com/health || exit 1
        curl -f https://staging.example.com/api/status || exit 1        

  # Deploy to Production
  deploy-production:
    runs-on: ubuntu-latest
    needs: integration-tests
    if: github.ref == 'refs/heads/main'
    environment:
      name: production
      url: https://example.com
    
    steps:
    - name: Checkout code
      uses: actions/checkout@v4
    
    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v4
      with:
        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        aws-region: us-east-1
    
    - name: Deploy to ECS Production
      run: |
        aws ecs update-service \
          --cluster production-cluster \
          --service web-service \
          --force-new-deployment        
    
    - name: Wait for deployment
      run: |
        aws ecs wait services-stable \
          --cluster production-cluster \
          --services web-service        
    
    - name: Verify deployment
      run: |
        curl -f https://example.com/health || exit 1
                
    - name: Notify deployment
      uses: 8398a7/action-slack@v3
      with:
        status: ${{ job.status }}
        text: 'Production deployment completed successfully!'
        webhook_url: ${{ secrets.SLACK_WEBHOOK }}
      if: always()

Step 3: Set Up GitLab CI/CD Pipeline

GitLab CI/CD is tightly integrated with GitLab repositories.

Create Pipeline Configuration

## .gitlab-ci.yml
stages:
  - build
  - test
  - security
  - deploy-staging
  - test-staging
  - deploy-production

variables:
  DOCKER_DRIVER: overlay2
  DOCKER_TLS_CERTDIR: "/certs"

## Build Stage
build:
  stage: build
  image: node:18
  script:
    - npm ci
    - npm run build
  artifacts:
    paths:
      - dist/
    expire_in: 1 hour
  cache:
    key: ${CI_COMMIT_REF_SLUG}
    paths:
      - node_modules/

## Unit Tests
test:unit:
  stage: test
  image: node:18
  script:
    - npm ci
    - npm test -- --coverage
  coverage: '/Lines\s*:\s*(\d+\.\d+)%/'
  artifacts:
    reports:
      junit: junit.xml
      coverage_report:
        coverage_format: cobertura
        path: coverage/cobertura-coverage.xml

## Integration Tests
test:integration:
  stage: test
  image: node:18
  services:
    - postgres:14
  variables:
    POSTGRES_DB: testdb
    POSTGRES_USER: testuser
    POSTGRES_PASSWORD: testpass
  script:
    - npm ci
    - npm run test:integration

## Security Scanning
security:sast:
  stage: security
  image: docker:latest
  services:
    - docker:dind
  script:
    - docker run --rm -v $(pwd):/src aquasec/trivy fs --security-checks vuln /src

security:dependency:
  stage: security
  image: node:18
  script:
    - npm audit --audit-level=high
  allow_failure: true

## Build Docker Image
build:docker:
  stage: build
  image: docker:latest
  services:
    - docker:dind
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA .
    - docker tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA $CI_REGISTRY_IMAGE:latest
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
    - docker push $CI_REGISTRY_IMAGE:latest
  only:
    - main
    - develop

## Deploy to Staging
deploy:staging:
  stage: deploy-staging
  image: alpine:latest
  before_script:
    - apk add --no-cache curl
  script:
    - |
      curl -X POST https://api.staging.example.com/deploy \
        -H "Authorization: Bearer $STAGING_DEPLOY_TOKEN" \
        -H "Content-Type: application/json" \
        -d '{"image":"'"$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA"'"}'      
  environment:
    name: staging
    url: https://staging.example.com
  only:
    - main
    - develop

## Smoke Tests on Staging
test:staging:
  stage: test-staging
  image: alpine:latest
  before_script:
    - apk add --no-cache curl
  script:
    - curl -f https://staging.example.com/health || exit 1
    - curl -f https://staging.example.com/api/status || exit 1
  only:
    - main
    - develop

## Deploy to Production
deploy:production:
  stage: deploy-production
  image: alpine:latest
  before_script:
    - apk add --no-cache curl
  script:
    - |
      curl -X POST https://api.example.com/deploy \
        -H "Authorization: Bearer $PRODUCTION_DEPLOY_TOKEN" \
        -H "Content-Type: application/json" \
        -d '{"image":"'"$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA"'"}'      
  environment:
    name: production
    url: https://example.com
  when: manual  # Require manual approval
  only:
    - main

Step 4: Configure Jenkins Pipeline

Jenkins provides flexible pipeline capabilities.

Create Jenkinsfile

// Jenkinsfile
pipeline {
    agent any
    
    environment {
        DOCKER_REGISTRY = 'docker.io'
        IMAGE_NAME = 'myapp'
        STAGING_SERVER = 'staging.example.com'
        PRODUCTION_SERVER = 'example.com'
    }
    
    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }
        
        stage('Install Dependencies') {
            steps {
                sh 'npm ci'
            }
        }
        
        stage('Lint') {
            steps {
                sh 'npm run lint'
            }
        }
        
        stage('Unit Tests') {
            steps {
                sh 'npm test -- --coverage'
            }
            post {
                always {
                    junit 'test-results/*.xml'
                    publishCoverage adapters: [coberturaAdapter('coverage/cobertura-coverage.xml')]
                }
            }
        }
        
        stage('Build') {
            steps {
                sh 'npm run build'
            }
        }
        
        stage('Security Scan') {
            parallel {
                stage('Dependency Check') {
                    steps {
                        sh 'npm audit --audit-level=high'
                    }
                }
                stage('SAST') {
                    steps {
                        sh 'sonar-scanner'
                    }
                }
            }
        }
        
        stage('Build Docker Image') {
            when {
                branch 'main'
            }
            steps {
                script {
                    docker.withRegistry("https://${DOCKER_REGISTRY}", 'docker-credentials') {
                        def image = docker.build("${IMAGE_NAME}:${BUILD_NUMBER}")
                        image.push()
                        image.push('latest')
                    }
                }
            }
        }
        
        stage('Deploy to Staging') {
            when {
                branch 'main'
            }
            steps {
                script {
                    sshagent(['staging-ssh-key']) {
                        sh """
                            ssh user@${STAGING_SERVER} '
                                docker pull ${DOCKER_REGISTRY}/${IMAGE_NAME}:${BUILD_NUMBER}
                                docker stop myapp || true
                                docker rm myapp || true
                                docker run -d --name myapp -p 80:8080 ${DOCKER_REGISTRY}/${IMAGE_NAME}:${BUILD_NUMBER}
                            '
                        """
                    }
                }
            }
        }
        
        stage('Integration Tests') {
            when {
                branch 'main'
            }
            steps {
                sh """
                    export TEST_URL=https://${STAGING_SERVER}
                    npm run test:integration
                """
            }
        }
        
        stage('Deploy to Production') {
            when {
                branch 'main'
            }
            steps {
                input message: 'Deploy to production?', ok: 'Deploy'
                
                script {
                    sshagent(['production-ssh-key']) {
                        sh """
                            ssh user@${PRODUCTION_SERVER} '
                                docker pull ${DOCKER_REGISTRY}/${IMAGE_NAME}:${BUILD_NUMBER}
                                docker stop myapp || true
                                docker rm myapp || true
                                docker run -d --name myapp -p 80:8080 ${DOCKER_REGISTRY}/${IMAGE_NAME}:${BUILD_NUMBER}
                            '
                        """
                    }
                }
            }
        }
        
        stage('Smoke Tests') {
            when {
                branch 'main'
            }
            steps {
                sh """
                    curl -f https://${PRODUCTION_SERVER}/health || exit 1
                """
            }
        }
    }
    
    post {
        success {
            slackSend channel: '#deployments',
                      color: 'good',
                      message: "Deployment successful: ${env.JOB_NAME} ${env.BUILD_NUMBER}"
        }
        failure {
            slackSend channel: '#deployments',
                      color: 'danger',
                      message: "Deployment failed: ${env.JOB_NAME} ${env.BUILD_NUMBER}"
        }
    }
}

Step 5: Implement Deployment Strategies

Choose the right deployment strategy for your application.

Blue-Green Deployment

## GitHub Actions example
- name: Blue-Green Deployment
  run: |
    # Deploy to green environment
    aws ecs update-service \
      --cluster production \
      --service web-green \
      --force-new-deployment
    
    # Wait for green to be healthy
    aws ecs wait services-stable \
      --cluster production \
      --services web-green
    
    # Switch traffic to green
    aws elbv2 modify-rule \
      --rule-arn $RULE_ARN \
      --actions Type=forward,TargetGroupArn=$GREEN_TG_ARN
    
    # Keep blue running for rollback capability
    echo "Blue-green deployment complete"

Canary Deployment

## Kubernetes canary deployment
apiVersion: v1
kind: Service
metadata:
  name: myapp
spec:
  selector:
    app: myapp
  ports:
    - port: 80
      targetPort: 8080

---
## Stable deployment (90% traffic)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-stable
spec:
  replicas: 9
  selector:
    matchLabels:
      app: myapp
      version: stable
  template:
    metadata:
      labels:
        app: myapp
        version: stable
    spec:
      containers:
      - name: myapp
        image: myapp:v1.0.0

---
## Canary deployment (10% traffic)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-canary
spec:
  replicas: 1
  selector:
    matchLabels:
      app: myapp
      version: canary
  template:
    metadata:
      labels:
        app: myapp
        version: canary
    spec:
      containers:
      - name: myapp
        image: myapp:v1.1.0

Rolling Deployment

## Kubernetes rolling update
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
spec:
  replicas: 10
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 2        # Create 2 extra pods during update
      maxUnavailable: 1  # Allow 1 pod to be unavailable
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
      - name: myapp
        image: myapp:v1.1.0
        readinessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 5

Step 6: Implement Rollback Mechanisms

Always have a rollback strategy.

Automated Rollback on Failure

## GitHub Actions with rollback
- name: Deploy and Monitor
  run: |
    # Deploy new version
    kubectl set image deployment/myapp myapp=myapp:${{ github.sha }}
    
    # Wait and monitor
    kubectl rollout status deployment/myapp --timeout=5m || {
      echo "Deployment failed, rolling back"
      kubectl rollout undo deployment/myapp
      exit 1
    }
    
    # Additional health checks
    sleep 30
    curl -f https://example.com/health || {
      echo "Health check failed, rolling back"
      kubectl rollout undo deployment/myapp
      exit 1
    }

Manual Rollback Script

#!/bin/bash
## rollback.sh

DEPLOYMENT="myapp"
NAMESPACE="production"

echo "Current deployment status:"
kubectl rollout history deployment/$DEPLOYMENT -n $NAMESPACE

read -p "Which revision to rollback to? (leave empty for previous): " REVISION

if [ -z "$REVISION" ]; then
    kubectl rollout undo deployment/$DEPLOYMENT -n $NAMESPACE
else
    kubectl rollout undo deployment/$DEPLOYMENT -n $NAMESPACE --to-revision=$REVISION
fi

echo "Monitoring rollback..."
kubectl rollout status deployment/$DEPLOYMENT -n $NAMESPACE

Step 7: Add Monitoring and Notifications

Monitor deployments and notify teams of results.

Deployment Metrics

## Prometheus metrics for deployments
- name: Record deployment metrics
  run: |
    curl -X POST http://prometheus-pushgateway:9091/metrics/job/deployments \
      -d "deployment_count{environment=\"production\",status=\"success\"} 1"
    
    curl -X POST http://prometheus-pushgateway:9091/metrics/job/deployments \
      -d "deployment_duration_seconds{environment=\"production\"} $DURATION"

Slack Notifications

- name: Notify Slack
  uses: 8398a7/action-slack@v3
  with:
    status: custom
    custom_payload: |
      {
        "blocks": [
          {
            "type": "section",
            "text": {
              "type": "mrkdwn",
              "text": "*Deployment Status: ${{ job.status }}*\n*Environment:* Production\n*Version:* ${{ github.sha }}\n*Author:* ${{ github.actor }}"
            }
          }
        ]
      }      
    webhook_url: ${{ secrets.SLACK_WEBHOOK }}
  if: always()

Email Notifications

- name: Send Email Notification
  uses: dawidd6/action-send-mail@v3
  with:
    server_address: smtp.gmail.com
    server_port: 465
    username: ${{ secrets.EMAIL_USERNAME }}
    password: ${{ secrets.EMAIL_PASSWORD }}
    subject: "Deployment ${{ job.status }}: ${{ github.repository }}"
    body: |
      Deployment to production completed with status: ${{ job.status }}
      
      Commit: ${{ github.sha }}
      Author: ${{ github.actor }}
      Workflow: ${{ github.workflow }}      
    to: [email protected]
  if: always()

Step 8: Secure Your Pipeline

Implement security best practices in your CD pipeline.

Secrets Management

## Use environment-specific secrets
- name: Deploy with secrets
  env:
    API_KEY: ${{ secrets.PRODUCTION_API_KEY }}
    DATABASE_URL: ${{ secrets.PRODUCTION_DB_URL }}
  run: |
    kubectl create secret generic app-secrets \
      --from-literal=api-key=$API_KEY \
      --from-literal=database-url=$DATABASE_URL \
      --dry-run=client -o yaml | kubectl apply -f -

Image Signing and Verification

- name: Sign container image
  run: |
    cosign sign --key cosign.key ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}    

- name: Verify image signature
  run: |
    cosign verify --key cosign.pub ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}

SBOM Generation

- name: Generate SBOM
  run: |
    syft ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }} -o spdx-json > sbom.json    

- name: Upload SBOM
  uses: actions/upload-artifact@v3
  with:
    name: sbom
    path: sbom.json

Best Practices

1. Keep Pipelines Fast

Parallelize independent stages
Use caching effectively
Run fastest tests first
Optimize build processes

2. Make Pipelines Reliable

Retry flaky tests
Use health checks
Implement proper timeouts
Handle failures gracefully

3. Maintain Visibility

Log all pipeline steps
Provide clear error messages
Track deployment metrics
Send meaningful notifications

4. Version Everything

Pipeline configuration
Infrastructure as code
Deployment manifests
Environment configurations

5. Test Your Pipeline

Test on feature branches
Use staging environments
Validate rollback procedures
Conduct disaster recovery drills

Common Pitfalls to Avoid

Not testing deployments: Always test in staging first
Missing rollback plans: Every deployment should be reversible
Ignoring monitoring: Deploy and verify, don’t deploy and hope
Hardcoded secrets: Use proper secrets management
No approval gates: Add manual approvals for production
Skipping security scans: Security should be automated
Poor error handling: Pipeline failures should be clear

Conclusion

A well-designed continuous deployment pipeline:

✅ Automates the entire deployment process
✅ Includes comprehensive testing at every stage
✅ Implements security scanning and best practices
✅ Uses appropriate deployment strategies
✅ Provides quick and reliable rollback mechanisms
✅ Monitors deployments and notifies teams
✅ Maintains secrets securely
✅ Enables rapid, confident releases

Start with a simple pipeline and iterate based on your needs. The goal is to make deployments routine, boring, and safe—allowing your team to focus on building features rather than managing releases.

Remember: The best CD pipeline is one that your team trusts and uses confidently every day. Invest time in making it reliable, fast, and maintainable.