Terraform state management is extremely important with all Terraform projects. State management is the feature that allows you to manage, store, and retrieve the current state of the resources managed by your Terraform infrastructure as code (IaC) deployments. In this article, we will explore different types of Terraform state management, best practices, and security concerns to keep in mind while managing the state.

Let’s get started looking at how this insanely important feature of HashiCorp Terraform works!

What is Terraform State?

HashiCorp Terraform state is a representation of the current state of your infrastructure resources. This is how Terraform tracks the resources managed by the Terraform project. This includes resource attributes such as resource names, resource IDs, and any metadata associated with the deployed resources.

Terraform state is stored in a state file, using a .tfstate file extension, which is created and maintained by Terraform. The state file is not manually edited. This state file is essential because it allows Terraform to track the resources it manages and is used to determine the actions required to bring the resources into the desired state when Terraform configuration is changed.

Why is Terraform State Used?

Terraform state is used to effectively manage the infrastructure resources that are deployed and managed by the Terraform project. Infrastructure resources can be complex and require multiple steps to create, configure, and manage. Terraform allows you to declare the desired state of your resources in code and Terraform uses the state file to keep track of the state of those resources.

Terraform state is also used to enforce idempotency. Idempotency is the ability to apply a configuration multiple times and achieve the same result every time. Terraform uses state to ensure that the resources’ current state matches the declared state. If the current state does not match the declared state, Terraform will take the necessary actions to bring the resources into the desired state.

Another essential use of Terraform state is to enable collaboration. Terraform state is shared among team members to ensure that everyone is working on the same version of the infrastructure deployed to the environment. This allows for better collaboration, easier debugging, and a more transparent workflow while implementing HashiCorp Terraform as code on your projects.

How does Terraform State Work?

Terraform state works by storing a record of the resources deployed and managed by Terraform and their current configuration. When you run the terraform plan and terraform apply commands, it uses the contents of the state file to determine the required actions to modify the infrastructure resources to bring them into the desired state that is configured in the Terraform code. The state is also used to determine if a resource needs to be created, updated, or destroyed in order to meet the desired configuration.

Terraform state can be managed locally or remotely. Local state management stores the state file on the local file system, while remote state management stores the state file in a remote data store such as Microsoft Azure Blob Storage or AWS S3. Remote state management provides better collaboration and security features than local state management.

Terraform state management can be divided into two categories:

• local state management
• remote state management

Let’s look at each of these a little more closely.

Local and Remote Terraform State Management

Local state management stores the state on the local file system of the machine running Terraform. Local state management is suitable for small teams or single-person projects because it is easy to set up and manage. Local state management is the default and makes it easy to get up and running with HashiCorp Terraform infrastructure as code deployments.

Remote state management stores the state in a remote data store such as Microsoft Azure Blob Storage, AWS S3, or even the HashiCorp Terraform Cloud. Remote state management is suitable for larger teams or projects with multiple contributors because it allows multiple team members to share and manage the state.

How to Use Local State Management

Terraform uses a local state file named terraform.tfstate to store the current state of resource configuration. The local state file is the default Terraform when creating a new Terraform project, but you can configure it using the terraform block in the Terraform configuration file.

To initialize local state management, run the terraform init command. Terraform will create a new state file named terraform.tfstate in the current directory of the Terraform project. Once initialized, Terraform will use this local state file to store the metadata of your infrastructure resources when terraform apply is run to deploy to the environment.

Terraform local state does, however, have some disadvantages. It is vulnerable to data loss, and is challenging for team collaboration. Therefore, it is recommended to use remote state management with production Terraform deployments.

Disadvantages of Local State Management in Terraform

Local state management for Terraform projects does have some disadvantages. Local state management stores the state file (.tfstate) on the local file system. There are some disadvantages to relying on local state management. While it’s the default and easy to use, it is important to understand the limitations and disadvantages of relying on local state management for your Terraform projects.

The following are some disadvantages to relying on Terraform local state management:

Lack of Collaboration

One of the main disadvantages of using local state in Terraform is the lack of collaboration. Local state management stores the state file on the local file system, making it challenging to share the state file among team members. This can lead to conflicts when multiple team members are working on the same infrastructure. For example, if two team members apply changes to the infrastructure at the same time, it can result in conflicting state changes.

Security Risks

Another disadvantage of using local state in Terraform is security risks. Local state management stores the state file on the local file system, which can be vulnerable to security risks such as data loss, corruption, and unauthorized access. Additionally, local state management can make it difficult to manage access controls, making it challenging to ensure that only authorized users have access to the state file.

Limited Backup and Recovery Options

When using local state management, there are limited backup and recovery options available. If the state file becomes corrupted or lost, it can be challenging to restore the state file to its previous state. Additionally, local state management does not provide version control, making it difficult to track changes made to the infrastructure.

You might be inclined to commit the state file (.tfstate) to a Git or other source control repository. After all, it’s just JSON so that would work, right? Wrong! Do not ever commit the Terraform state file to source control. There are secrets, keys and other information stored in the Terraform state file, and committing this to source control would open you up to being vulnerable to unintended access issues to those secrets and keys.

How to Use Remote State Management with Azure, AWS or Other Cloud Providers

Remote state management allows you to store the state in a remote data store. Terraform supports several remote data stores, including Microsoft Azure Blob Storage, AWS S3, Google Cloud Storage, and the HashiCorp Terraform Cloud.

The following is an example of how to configure remote state management using Microsoft Azure Blob Storage:

terraform {
  backend "azurerm" {
    resource_group_name  = "tf-state-rg
    storage_account_name = "tfstatesa"
    container_name       = "tfstate"
    key                  = "prod.terraform.tfstate"
  }
}

The above code block configures Terraform to store the state in the Azure Blob Storage within the container named tfstate within the tfstatesa storage account. The state file name is prod.terraform.tfstate. You need to create the storage account and container manually in Azure before running the terraform init command.

Terraform also supports using the HashiCorp Terraform Cloud as a remote state backend. Terraform Cloud provides a secure and scalable way to store and manage your state files.

Related: For further explanation and example of setting up remote state management using a cloud storage account, please refer to the “Terraform: Store Backend State in Azure Storage Account” article written by Chris Pietschmann.

Advantages of Remote State Management in Terraform

Terraform state management is a critical aspect of managing infrastructure as code. Remote state management stores the state file in a remote data store such as Microsoft Azure Blob Storage, AWS S3, or even the HashiCorp Terraform Cloud.

The following are some advantages of using remote state management for managing the state of Terraform projects:

Improved Collaboration

One of the significant advantages of using remote state management in Terraform is improved collaboration. Remote state management allows team members to access and update the state file from a central location. This helps to avoid conflicts that may arise when multiple team members work on the same infrastructure. Additionally, remote state management provides better version control, making it easy to track changes and revert to previous versions.

Increased Security

Remote state management provides increased security for Terraform state (.tfstate) files. By storing the state file in a remote data store, such as Microsoft Azure Blob Storage or AWS S3, you can ensure that only authorized team members have access to the state file. Additionally, remote state management provides better access control, making it easier to manage permissions and roles.

Better Backup and Recovery Options

Remote state management provides better backup and recovery options for Terraform state files. By storing the state file in a remote data store, you can easily back up the state file, or utilize the cloud data replication features, to prevent data loss. Additionally, cloud storage services provide version control features which make it easy to restore the state file to a previous version if necessary.

Integration with Cloud Providers

Remote state management in Terraform integrates seamlessly with cloud providers such as Microsoft Azure and Amazon AWS. This integration allows Terraform to create and manage resources in the cloud, while also storing state files in a remote data store in the same cloud.

How to Use Terraform Cloud to Manage Terraform State

HashiCorp Terraform Cloud is a SaaS-based (Software as a Service) platform provided by HashiCorp to manage Terraform state files remotely. Terraform Cloud provides a secure and scalable way run Terraform deployments, and to store and manage the state files for these deployments.

To use Terraform Cloud, you need to sign up for a HashiCorp Terraform Cloud account and create a new workspace. After creating a workspace, you can configure your Terraform configuration file to use Terraform Cloud as the backend.

The following is a simple example of configuring Terraform to use Terraform Cloud as the backend:

terraform {
  backend "remote" {
    organization = "myorganization"
    workspaces {
      name = "myworkspace"
    }
  }
}

The above code block configures Terraform to use the remote backend provided by Terraform Cloud. You need to replace the organization and workspace name with your Terraform Cloud organization and workspace.

Once you have configured the backend, you can initialize the Terraform workspace by running the terraform init command. Terraform will prompt you to log in to your Terraform Cloud account, and it will automatically create a new state file in your workspace.

Terraform Cloud provides several advantages over other remote state backends. It provides a user-friendly web interface to view and manage the state files. It also provides collaboration features, such as role-based access control and workspace sharing.

Securely Managing Terraform State

Terraform state management involves storing sensitive information such as access keys, passwords, and tokens. Therefore, it is essential to keep the state file secure and prevent unauthorized access.

The following are some best practices for managing the security of Terraform state files:

• Encrypt the state file
  Encrypt the state file using a strong encryption algorithm to prevent unauthorized access when stored at rest or in transit with the cloud storage service when using remote state management. Most cloud storage services will actually implement this by default.
• Limit access to the state file
  It is important to restrict access to the Terraform state file to authorized users only. Use role-based access control to grant access to specific team members as necessary. Often times, the service account (or Service Principal) that the Terraform project uses in CI/CD pipelines may be the only user needing access directly to the Terraform state file, so restricting access accordingly will keep the secrets and keys stored in the state file safe.
• Use secure storage
  Store the state file in a secure and encrypted data store. Avoid storing the state file in plain text in a public or broadly accessible repository. In general, it’s best practice to never commit Terraform state files to a git or other source control repository. Instead, keep the state file in a secure file store instead.
• Rotate access keys and credentials
  Rotate access keys and credentials regularly to prevent unauthorized access. Use a key management system such as HashiCorp Vault or Azure Key Vault to manage the access keys and credentials as necessary. Key rotation is a general security best practice for any data storage, and this includes the Terraform state file.

Conclusion

In this article, we explored the different types of state management that is supported by HashiCorp Terraform, as well as the best practices for securely managing Terraform state files. We covered how to use local and remote state management, and how to use HashiCorp Terraform Cloud for state management. Following the best practices mentioned will help ensure that the Terraform state management files are always available and secured appropriately.

Happy Terraform-ing!