HashiCorp Terraform is a popular tool for managing infrastructure as code (IaC). By defining your IaC using Terraform, you can use version control with your infrastructure configuration and also automate infrastructure deployment in a consistent and repeatable way. Azure DevOps Pipelines can be used to setup YAML pipelines to instrument the Terraform infrastructure deployments using the traditional Terraform Plan and Apply workflow. This article will show you how to set up two separate YAML pipelines in Azure DevOps to implement the Plan and Apply workflow for your Terraform infrastructure deployments.
How the Terraform Plan and Apply Workflow will be set up in Azure DevOps
With the usual Terraform workflow, the plan
command is first run, then the apply
command is run. This is the normal Terraform workflow process. However, here’s a process flow outline of the steps necessary for the Terraform workflow along with the places where the YAML pipelines shown in this article fit in.
- The DevOps Engineer or Site Reliability Engineer (SRE) commits Terraform code changes to the git repository within Azure DevOps.
- The
Terraform Plan
(defined in this article) will trigger automatically to run theterraform plan
command and generate a Terraform Plan (.tfplan
) file that will get stored in the build artifacts for the pipeline. - The DevOps Engineer or SRE will then perform the manual task of inspecting the Azure DevOps Pipeline output and review the generated Terraform plan.
- Once the proposed Terraform plan of infrastructure changes to make has been approved by the DevOps Engineer or SRE, they will manually trigger the
Terraform Apply
pipeline to run. - The
Terraform Apply
pipeline will pull in the Terraform plan file (.tfplan
) stored within the build artifacts of the most recent run of theTerraform Plan
pipeline, then it will execute the plan and make all the necessary infrastructure changes. - When ever Terraform configuration changes are needed to update / modify the infrastructure deployment, this process repeats again.
This Terraform workflow of the plan and apply steps could be implemented using a single Azure DevOps YAML Pipeline. However, then the pipeline would automatically apply after performing the plan step. This would not be ideal as it’s very important to inspect the Terraform plan before applying. If you skip this review step then it’s possible for unintended or breaking changes to be made to the infrastructure deployment. It’s best practice to always review the Terraform plan before applying it.
Now, let’s look at the YAML code for the Azure DevOps Pipelines to implement multi-pipeline strategy for managing HashiCorp Terraform deployments using Azure DevOps!
Define Terraform Plan Azure DevOps YAML Pipeline
The following is the YAML for the Azure DevOps Pipeline for performing the Terraform Plan step of the workflow:
name: "Terraform Plan"
trigger:
branches:
include:
- main
pool:
vmImage: 'ubuntu-latest'
steps:
- task: TerraformInstaller@0
displayName: "Install Terraform"
inputs:
terraformVersion: '1.3.9'
terraformDownloadLocation: 'https://releases.hashicorp.com/terraform'
- script: |
terraform init
terraform plan -out=terraform.tfplan
displayName: 'Terraform Plan'
- task: PublishBuildArtifacts@1
inputs:
pathtoPublish: '$(Build.SourcesDirectory)/terraform.tfplan'
artifactName: 'terraformPlan'
publishLocation: 'Container'
To implement these pipelines according to what’s laid out in this article, be sure to name this first YAML pipeline Terraform Plan
. This name is important, as the next YAML pipeline will reference it by name.
Notice the trigger
defined on this pipeline. It is configured to automatically trigger this pipeline on pushes to the main
branch of the repository. This will help to automatically generate a Terraform Plan each time changes are pushed to the branch. Since the terraform plan
command doesn’t make any environment changes, this is perfectly save to do. However, you may want to configure a different trigger if that fits better with your teams project release goals.
Also notice that the Terraform Plan file is named terraform.tfplan
and the artifact name is set to terraformPlan
. These are important values, as they will be used in the Terraform apply pipeline later to reference the correct artifacts when downloading / loading the plan file in that step of the Terraform workflow.
This Terraform Plan
YAML pipeline contains the following steps to build the Terraform plan and save it in the build artifacts for the pipeline:
- The
pool.vmImage
for the YAML defines the use of Ubuntu Linux for the operating system of the build agent. This is done with the value ofubuntu-latest
to specify the latest Ubuntu image version available. - The
TerraformInstaller@0
task is used to download and setup HashiCorp Terraform for use on the Azure DevOps build agent machine. It is also specifying the Terraform version required for the Terraform code in the repository. Be sure to set this to the version you’re standardizing your Terraform deployments on. - The
script
task is used to call both theterraform init
andterraform plan
commands to generate the Terraform plan file (.tfplan
) for the infrastructure configuration in the repository. - The
PublishBuildArtifacts@1
task is used to publish the Terraform plan file (.tfplan
) to the build artifacts for this pipeline. This is important, as this is how the Terraform apply pipeline will be able to access the previously generated plan from this pipeline.
Once the Terraform Plan
YAML pipeline is set up in your Azure DevOps project, the next item to set up is the Terraform Apply
pipeline.
Define Terraform Apply Azure DevOps YAML Pipeline
The following is the YAML for the Azure DevOps Pipeline for performing the Terraform Apply step of the workflow:
name: "Terraform Apply"
trigger: none
pool:
vmImage: 'ubuntu-latest'
steps:
- task: DownloadPipelineArtifact@2
displayName: 'Download Terraform Plan'
inputs:
buildType: specific
buildVersionToDownload: 'latest'
project: 'TFYAML' # replace with the name of your Azure DevOps Project
definition: 'Terraform Plan'
artifactName: 'terraformPlan'
path: '$(Pipeline.Workspace)'
- task: TerraformInstaller@0
displayName: "Install Terraform"
inputs:
terraformVersion: '1.3.9'
terraformDownloadLocation: 'https://releases.hashicorp.com/terraform'
- script: |
terraform init
terraform apply $(Pipeline.Workspace)/terraform.tfplan
displayName: 'Terraform Apply'
The Terraform Apply
pipeline is setup similarly to the Terraform Plan
pipeline, but has a couple differences necessary for applying the Terraform infrastructure changes based on the previously generated Terraform plan.
This Terraform Apply
YAML pipeline contains the following steps to pull in the previously generated Terraform Plan file (.tfplan
) and then use terraform apply
command to make the planned changes to the infrastructure deployment:
- The
DownloadPipelineArtifact@2
task is used to download the Terraform Plan file (.tfplan
) stored in the build artifacts of theTerraform Plan
pipeline so it can be used within this pipeline.
Be sure the following arguments of theDownloadPipelineArtifact@2
task are set correctly as follows:- project: This needs to be set to the name of the Azure DevOps Project where the
Terraform Plan
pipeline is run. In my example, it’sTFYAML
but will be different for your configuration of Azure DevOps. - definition: This is the name of the
Terraform Plan
pipeline. If you named your instance of the pipeline differently, then this must be set to the name you used.. - artifactName: The
Terraform Plan
pipeline code example stores the plan file (.tfplan
) in a build artifact namedterraformPlan
. If you change the code to use a different artifact name, then this will need to be set accordingly.
- project: This needs to be set to the name of the Azure DevOps Project where the
- The
TerraformInstaller@0
task is used to download and setup HashiCorp Terraform for use on the Azure DevOps build agent machine. It is also specifying the Terraform version required for the Terraform code in the repository. Be sure to set this to the Terraform version in this pipeline to the same version used in theTerraform Plan
pipeline. - The
script
task runs both theterraform init
andterraform apply
commands to perform the proposed changes from the Terraform plan and modify the infrastructure deployment to match.
Happy Terraforming! I hope this article helps you get your Azure DevOps Pipelines set up correctly to be able to perform Terraform infrastructure as code (IaC) deployments efficiently.
Very nice.
I do the plan and apply in one pipeline, with conditions to determine if apply will run, and approvals required if it will.
I add this in the plan script – I set a VSO variable to indicate if drift is detected
# -detailed error code gives 2 as rc if plan needs an apply
set +e
terraform plan -detailed-exitcode “${@}”
rc=$?
set -e
if [[ $rc -eq 0 ]]; then
echo “No diff, no apply needed”
echo “##vso[task.setvariable variable=result;isOutput=true]unchanged”
elif [[ $rc -eq 1 ]]; then
echo “Error running plan”
exit 1
elif [[ $rc -eq 2 ]]; then
echo “Diff, apply needed”
echo “##vso[task.setvariable variable=result;isOutput=true]changed”
else
echo “Unexpected rc $rc – erroring”
exit $rc
fi
Then in the apply stage, I put a condition on the stage based on the VSO variable
stages:
– stage: terraform_apply
# note the syntax for conditions is different to setting variables
# https://learn.microsoft.com/en-us/azure/devops/pipelines/process/deployment-jobs?view=azure-devops
condition: and(succeeded(), eq(dependencies.terraform_plan.outputs[‘terraform_plan.plan.result’], ‘changed’))
jobs:
– deployment: terraform_apply
environment: ${{ parameters.azdo_release_env }}
….
If there is no change detected, the Apply stage skips.
Note I also use a deployment rather than a job. This way I can add permissions to approve the deployment to the environment, so the apply stage, if there is a change detected, pauses until someone, in the approval list for environment, clicks the button.
Thanx again for this high-quality post
regards
Leif