Terraform Security & Secrets Management: Protecting State and Credentials

Terraform state files store every resource attribute in plaintext - including database passwords, API keys, and TLS private keys. One leaked tfstate file can hand an attacker the keys to your entire cloud account. In this lesson you will learn how to keep secrets out of state, redact them from plan output, encrypt state at rest, inject dynamic credentials from HashiCorp Vault, and scope least-privilege IAM policies for your Terraform service accounts.

1. Learning Objectives

By the end of this lesson, you will be able to:

• Explain why terraform.tfstate is a high-value security target and which secrets it stores in plaintext
• Mark variables and outputs as sensitive to keep secrets out of logs and plan output
• Keep secrets out of version control using .gitignore, tfvars conventions, and TF_VAR_ environment variables
• Encrypt state at rest with remote backends and cloud KMS
• Inject dynamic credentials from HashiCorp Vault instead of hardcoding them
• Apply least-privilege IAM policies to Terraform service accounts and CI/CD roles
• Detect leaked secrets in state files and plans, and remediate them safely

2. Why This Matters

You provision an RDS database with Terraform. The master password you pass through a tfvars file is written, in plaintext, into terraform.tfstate alongside every other resource attribute. A teammate pushes that state file to a public GitHub repository, or your S3 backend bucket is accidentally left world-readable. Within hours, automated scanners that sweep GitHub and S3 for files named terraform.tfstate extract every password, access key, and private key - and use them to take over your cloud account.

This is not hypothetical. Public state-file leaks are among the most common causes of cloud account compromise reported in incident reviews. A state file is the crown jewels of your infrastructure: it contains the full inventory of every resource Terraform manages and the credentials needed to re-create or destroy it. Securing Terraform is therefore not an optional hardening step - it is the foundation of your cloud security posture.

3. Core Concepts

3.1 Why State Files Are a Security Risk

Terraform state is a JSON document that records the real-world attributes of every resource it manages. It stores what the provider observed - including values you thought of as inputs: database passwords, API keys, TLS private keys, and connection strings. Marking a variable sensitive does not remove it from state; it only changes how the value is displayed in the CLI. That is the single most important fact in this lesson.

3.2 Sensitive Variables and Outputs

The sensitive = true argument on a variable or output block tells Terraform to redact the value in plan output, apply output, and terraform output. It does not encrypt the value in state. Sensitive is a display-layer control; backend encryption is the storage-layer control. You need both.

variable "db_password" {
  type      = string
  sensitive = true
}

output "db_password" {
  value     = aws_db_instance.main.password
  sensitive = true
}

# Use nonsensitive() ONLY when you are certain the value
# contains no secrets (for example, a hostname or a port).
output "db_endpoint" {
  value = nonsensitive("${aws_db_instance.main.address}:${aws_db_instance.main.port}")
}

3.3 Secret Sources and the TF_VAR Convention

Never hardcode secrets in .tf files and never commit tfvars files that contain real values. Terraform resolves variables in this order: CLI -var flags, tfvars files, environment variables prefixed with TF_VAR_, then defaults. For secrets, prefer environment variables or a secret manager, and keep only example tfvars files containing placeholders.

export TF_VAR_db_password="$(openssl rand -base64 24)"
export TF_VAR_api_key="$(aws secretsmanager get-secret-value --secret-id my-app/api-key --query SecretString --output text)"
terraform plan

3.4 Encrypted Remote State

A local tfstate file on a laptop is a single point of failure and a leak waiting to happen. Store state in a remote backend, enable server-side encryption, and enable locking so two engineers cannot apply conflicting changes. On AWS that means S3 with a KMS key plus a DynamoDB lock table; on Azure, Blob Storage with encryption and blob leases; on GCP, Cloud Storage with a KMS key and object locking.

terraform {
  backend "s3" {
    bucket         = "my-org-terraform-state"
    key            = "prod/network/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    kms_key_id     = "arn:aws:kms:us-east-1:123456789012:key/abcd-1234"
    dynamodb_table = "terraform-locks"
  }
}

3.5 Least-Privilege Service Accounts

The identity that runs terraform plan and terraform apply should have exactly the permissions the configuration needs - nothing more. Create a dedicated service account or CI/CD role per environment, grant only the actions the resources require, and separate state access (S3 read/write, DynamoDB locking, KMS decrypt) from infrastructure permissions. Never run Terraform with a human admin role or a standing root key.

3.6 HashiCorp Vault for Dynamic Secrets

Instead of storing static secrets in variables at all, let Vault issue short-lived credentials at apply time. The Vault provider can read KV secrets, generate database passwords, or mint ephemeral cloud credentials. The credentials exist only for the duration of the run - and because state still records what the provider used, always pair Vault with backend encryption.

# Vault issues short-lived AWS credentials for the apply run.
data "vault_aws_access_credentials" "creds" {
  backend = "aws"
  role    = "terraform-role"
}

provider "aws" {
  region     = "us-east-1"
  access_key = data.vault_aws_access_credentials.creds.access_key
  secret_key = data.vault_aws_access_credentials.creds.secret_key
}

4. Hands-On Practice

4.1 Build a Secret-Safe Project Skeleton

Start every Terraform project with a .gitignore that makes it hard to commit state or secrets by accident.

# Never commit state files - they contain secrets in plaintext.
*.tfstate
*.tfstate.*

# Local provider plugin cache and module downloads.
.terraform/

# Variable files may contain secrets. Commit examples only.
*.tfvars
.terraform.tfvars
!example.tfvars

# Terraform crash logs.
crash.log

4.2 Mark Variables and Outputs Sensitive

Declare every secret input with sensitive = true and propagate the flag through module outputs. A common leak: a module returns a password in an output that is not marked sensitive, and the parent configuration prints it in plaintext.

# variables.tf
variable "db_password" {
  type      = string
  sensitive = true
}

# outputs.tf - propagate sensitivity from the module
output "db_password" {
  value     = module.database.db_password
  sensitive = true
}

4.3 Pass Secrets via Environment Variables

Set secrets in the shell or in your CI/CD secret store and let Terraform pick them up through TF_VAR_. Never echo the values into logs.

export TF_VAR_db_password="$(openssl rand -base64 24)"
terraform plan -out=tfplan
terraform apply tfplan

4.4 Enable an Encrypted Remote Backend

Convert the project to a remote backend so state lives in S3 with KMS encryption and DynamoDB locking.

terraform {
  backend "s3" {
    bucket         = "my-org-terraform-state"
    key            = "prod/network/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    kms_key_id     = "arn:aws:kms:us-east-1:123456789012:key/abcd-1234"
    dynamodb_table = "terraform-locks"
  }
}

Run terraform init again after editing the backend block - Terraform will prompt you to migrate the local state into the remote backend. From then on, every read and write of state is encrypted at rest, and concurrent applies are blocked by the lock table.

4.5 Inject Database Credentials from Vault

Use the Vault provider to fetch credentials at apply time instead of storing them in variables.

data "vault_generic_secret" "db_creds" {
  path = "secret/data/team/database"
}

resource "aws_db_instance" "main" {
  identifier = "app-db"
  engine     = "postgres"
  username   = data.vault_generic_secret.db_creds.data["username"]
  password   = data.vault_generic_secret.db_creds.data["password"]
}

Point the provider at your Vault with VAULT_ADDR and VAULT_TOKEN environment variables, and give the token a policy that allows read on exactly the paths it needs - nothing more.

4.6 Scan State and Plans for Leaked Secrets

Make secret scanning part of your workflow. The state file and plan file will contain secret values - that is how Terraform works - so the goal is to catch secrets in places they should not be: logs, diffs, and version control.

The first command confirms a hard truth: the password is sitting in state in plaintext. That is exactly why backend encryption and credential rotation are non-negotiable, and why the diff scan runs as a pre-push check.

4.7 Scope a Least-Privilege Policy

Attach a policy like this to the Terraform service account instead of a broad admin policy. It can describe resources, write state, and decrypt the KMS key - but nothing else.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowStateAccess",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "dynamodb:GetItem",
        "dynamodb:PutItem",
        "dynamodb:DeleteItem"
      ],
      "Resource": [
        "arn:aws:s3:::my-org-terraform-state/*",
        "arn:aws:dynamodb:us-east-1:123456789012:table/terraform-locks"
      ]
    },
    {
      "Sid": "AllowKmsDecrypt",
      "Effect": "Allow",
      "Action": ["kms:Decrypt", "kms:GenerateDataKey"],
      "Resource": "arn:aws:kms:us-east-1:123456789012:key/abcd-1234"
    },
    {
      "Sid": "AllowDescribe",
      "Effect": "Allow",
      "Action": ["ec2:Describe*", "rds:Describe*"],
      "Resource": "*"
    }
  ]
}

5. Common Errors & Solutions

1. "Output refers to sensitive values" - Terraform refuses to render a value derived from a sensitive input into a non-sensitive output. Mark the output sensitive, or wrap only the provably-safe part in nonsensitive() (for example, a hostname or port). Never use nonsensitive() on a password.

2. Secrets still appear in logs and plan JSON - sensitive = true redacts the human-readable plan, but terraform plan -json and the state file still contain the values. If a child module does not mark its outputs sensitive, the parent sees plaintext. Mark outputs in every module, and do not pipe -json output into logs or CI artifacts.

3. State file or tfvars accidentally committed - Deleting the file does NOT remove it from git history. Treat the credentials as compromised: rotate every key and password in the file, then purge history with git filter-repo or BFG Repo-Cleaner. Finally, add the .gitignore rules from Section 4.1 so it cannot happen again.

4. Error: AccessDenied when Terraform reads or locks state - The runner can usually reach S3 but lacks a supporting permission. Grant s3:GetObject/s3:PutObject, the DynamoDB lock actions, and kms:Decrypt on the state key, and verify the KMS key policy itself allows the service account. All three pieces must agree: IAM policy, key policy, and bucket policy.

5. Vault provider returns 403 Forbidden or "missing required argument" - VAULT_ADDR and VAULT_TOKEN are not set in the runner, or the token's policy cannot read the secrets path. Set both environment variables, create a policy that grants read on exactly the paths Terraform uses, and never store the Vault token in a tfvars file.

6. Summary Checklist

• [ ] State files and tfvars are covered by .gitignore and never committed
• [ ] Every secret variable and output is marked sensitive, including module outputs
• [ ] Secrets are injected via TF_VAR_ environment variables or a secret manager, never hardcoded
• [ ] State is stored in a remote backend with encryption at rest and locking enabled
• [ ] The Terraform service account follows least privilege, with separate roles per environment
• [ ] Vault (or a cloud secret manager) issues credentials where possible instead of static values
• [ ] Plan and apply output is scanned for secrets, and any exposure triggers immediate rotation

7. Practice Exercise

Build the secret-safe lab from Section 4 end to end:

1. Create a project with the .gitignore from 4.1 and a variables.tf declaring db_password as sensitive.
2. Set the value with a TF_VAR_db_password environment variable and run terraform plan. Confirm the password is redacted in the plan output.
3. Add the S3 + DynamoDB backend from 4.4 and run terraform init to migrate state.
4. Fetch the same password from Vault (or your cloud secret manager) in a data source and point an aws_ssm_parameter resource at it.
5. Run terraform state pull and grep for the password. You will find it - document why backend encryption and rotation are therefore mandatory.
6. Commit only .tf files, then run the scanning commands from 4.6 as a pre-push check.

When you are done, the only places the secret should exist are the source system (Vault or a secret manager), the encrypted state file, and the resource itself - never in git history, logs, or a plaintext tfvars file.

8. Next Steps

You now have a security-hardened Terraform workflow: sensitive variables, encrypted remote state, least-privilege identities, and dynamic secrets from Vault. In the next lesson, we will explore Terraform Best Practices & Production Hardening - code organization and naming conventions, state workflows for teams, drift detection, and quality gates that keep a growing codebase reliable. Until then, apply what you have learned: mark every secret variable sensitive, enable backend encryption, and scan your state and plan output for leaked credentials before every push.