Setup and Manage Azure CI/CD Pipelines

Setting up your Azure DevOps Pipeline is the major step towards automating your Continuous Integration and Continuous Deployment process. Azure DevOps offers a very user friendly yet powerful way to manage your CI/CD pipelines to initiate and trigger upstream processes starting with a code deployment to Azure Repo.

Create your Azure Pipeline

To create a Pipeline, click on ‘New Pipeline’ on top left –

Select the Repo you want to connect to –

Select the Repo that you created at the beginning of your project creation –

Select ‘Starter Pipeline’ –

Run/Test you Pipeline

You will see that your Pipeline is created showing your default yml file created, with an option to Save and Run the Pipeline. Yaml file is where you configure everything that you would like to run when Pipeline is triggered (eg: when code is pushed to the Repo). Click ‘Save and run’ as shown below –

Add a commit message as the .yml file will be added to the Repo –

You can now see the Pipeline running –

..and getting finished –

Trigger you Pipeline by local Repo push

Perform Git pull on your local repo to pull the newly created .yml file and verify that you are able to modify it and push it back to the Azure repository and trigger the Pipeline –

Make a minor change and commit, push it back to the Azure Repo –

Once you push, you can see the Pipeline running and getting completed –

Understanding Pipeline YAML file and various setups

Pipeline YAML file can be used configure various stages, jobs, steps, strategies, dependencies, approval process and agent pools.

Below is a general structure of a CI/CD pipeline YAML file –

trigger:
- main

pool:
  vmImage: ubuntu-latest

steps:
- script: echo Hello, world test!
  displayName: 'Run a one-line script'

- script: |
    echo Add other tasks to build, test, and deploy your project.
    echo See https://aka.ms/yaml
  displayName: 'Run a multi-line script'

Above means that the Pipeline will be triggered on changes to the ‘main’ branch of the Repo, its build over the machine specified under the ‘pool’ definition and it perform the steps mentioned under the ‘steps’ section.

Generally, Pipeline YAML config allows you to split the operation across – Stages à Jobs à Steps.

Stages are analogous to various Deployment levels. Eg: Build, Dev, UAT, Prod. Within a stage, you can have multiple Jobs that you would want to run. Within a Job, you can have multiple tasks or steps you would want to perform. You can also create dependencies between Jobs under a specific stage or dependency between various stages as well. You can also restrict a stage’s execution by putting an approval process in front of it eg: requiring QA review and approval before production deployment.

Below is a more extensive YAML configuration covering above scenarios –

trigger:
- main

pool:
  vmImage: windows-latest

stages:
- stage: Build
  jobs:
  - job: Job1
    steps:
    - bash: echo Build Job1
    - bash: echo Do Something
  - job: Job2
    dependsOn: Job1
    steps:
    - bash: echo Build Job2
- stage: Dev
  dependsOn: Build
  jobs:
  - job: Job1
    steps:
    - bash: echo Build Job1
  - job: Job2    
    steps:
    - bash: echo Build Job2
- stage: UAT
  dependsOn: Build
  jobs:
  - job: Job1
    steps:
    - bash: echo Build Job1
    - bash: echo Do Something
  - job: Job2
    dependsOn: Job1
    steps:
    - bash: echo Build Job2
- stage: Prod
  dependsOn: Dev
  dependsOn: UAT
  jobs:
  - job: Job1
    steps:
    - bash: echo Build Job1
    - bash: echo Do Something
  - job: Job2
    dependsOn: Job1
    steps:
    - bash: echo Build Job2

    Once you save the YML file, it will kick off the various stages, Jobs and steps defined as below –

See how the various ‘deploy’ stages for Dev and UAT kicks off in parallel after Build is completed. Also note that Prod deploy is initiated only after other 3 are completed. If you click on the job details, you can see below –

Note that Job 2 in Build stage is only initiated after Job1 is completed.

Build, Deploy stages with deployment step, environments and approval step

A typical CI/CD pipeline involves building the code, deploying it to multiple environments involving approval needed at some stage – eg: QA requiring to approve before QA deployment can be completed. Below is a sample YAML file for this scenario –

trigger:
- main

pool:
  vmImage: windows-latest
  
stages:
- stage: Build
  jobs:
  - job: Job1
    steps:
    - bash: echo Build Job1
    - bash: echo Do Something
- stage: Dev
  dependsOn:
  - Build
  jobs:
  - deployment: DevDeployJob
    environment: Dev
    strategy:
      runOnce:
       deploy:
        steps:
        - script: echo Dev deploy
- stage: QA
  dependsOn:
  - Build
  jobs:
  - deployment: QADeployJob
    environment: QA
    strategy:
      runOnce:
       deploy:
        steps:
        - script: echo QA deploy
- stage: Prod
  dependsOn:
  - QA   
  jobs:
  - deployment: ProdDeployJob
    environment: Prod
    strategy:
      runOnce:
       deploy:
        steps:
        - script: echo Prod deploy

To set approval, go to environment tab and add the approval step eg: in this scenario, QA need approval before deployment can be made –