Terraform drift: cloud resource created manually, detect and reconcile

TL;DR — You run terraform plan or apply and discover a resource already exists, has changed unexpectedly, or Terraform wants to recreate something that was created manually in the cloud console/CLI. The fastest safe path is: refresh state, identify the exact remote object, then either terraform import it into the correct address or delete the out-of-band resource if Terraform should own a fresh one. Reading time: ~6 min

The scenario

It is Tuesday afternoon, you are trying to ship a small networking change, and terraform apply suddenly fails on a resource your team swears "already exists." Someone added a DNS record, security group rule, bucket, IAM role, or database instance directly in the cloud dashboard during an incident last week and never backfilled Terraform. Now your plan is noisy, your CI job is red, and the risky part is that Terraform may try to replace something production is actively using. You need to determine whether this is harmless drift, a state problem, or a naming collision before you touch anything.

Symptoms

• terraform plan shows unexpected create/update/destroy actions for resources you thought Terraform already managed.
• terraform apply fails with provider errors like:

Error: creating S3 Bucket (my-app-assets-prod): BucketAlreadyOwnedByYou: Your previous request to create the named bucket succeeded and you already own it.
	status code: 409

• Or:

Error: creating Security Group (web-prod): InvalidGroup.Duplicate: The security group 'web-prod' already exists for VPC 'vpc-0123456789abcdef0'
	status code: 400

• Or:

Error: A resource with the ID "/subscriptions/.../resourceGroups/rg-prod/providers/.../my-resource" already exists - to be managed via Terraform this resource needs to be imported into the State.

• terraform plan -refresh-only shows drift on attributes nobody changed in code.
• terraform state list does not contain the resource address you expect, but the cloud console/CLI shows the object exists.
• CI exits non-zero on plan:

Plan: 1 to add, 0 to change, 0 to destroy.
Error: ... already exists

• A teammate says "I fixed it in the console" and there is no corresponding Git commit changing Terraform.

Likely causes

Step-by-step diagnosis

1. Check you are in the right directory, backend, and workspace.

terraform init -reconfigure
terraform workspace show
terraform providers

This is your problem if the workspace is not the expected one (default instead of prod, for example), or the backend points at the wrong state object. Jump to Fixes → Wrong workspace/state/backend.

1. Refresh-only plan to separate real remote drift from stale local assumptions.

terraform plan -refresh-only -out=tfrefresh.plan

This is your problem if Terraform reports changes without any code changes, for example:

  # aws_security_group.web has changed
  ~ resource "aws_security_group" "web" {
        id = "sg-0abc1234"
      ~ ingress = [ ... ]
    }

Plan: 0 to add, 1 to change, 0 to destroy.

If the resource is already in state and only attributes changed, jump to Fixes → Resource was changed manually after Terraform created it or Provider defaults/API normalization differ from config.

1. Check whether the resource exists in state at all.

terraform state list | sort

If the expected address is missing, inspect the cloud side by exact identifier using your provider CLI. Examples:

aws s3api head-bucket --bucket my-app-assets-prod
aws ec2 describe-security-groups --filters Name=group-name,Values=web-prod
az resource show --ids "/subscriptions/.../resourceGroups/rg-prod/providers/.../my-resource"
gcloud compute addresses describe web-prod-ip --region us-central1

This is your problem if the cloud object exists but terraform state list has no matching address. Jump to Fixes → Resource exists remotely but is not in Terraform state.

1. Check whether Terraform is trying to destroy one address and create the same thing at another address.

terraform plan

Look for output shaped like this:

  # module.network.aws_security_group.web[0] will be destroyed
  # aws_security_group.web["prod"] will be created

This is your problem if the remote object is the same logical thing but the Terraform address changed because of a refactor, module move, or count to for_each migration. Jump to Fixes → Resource address changed in code.

1. Confirm whether the plan noise is just provider normalization/defaults.

terraform plan -refresh-only

This is your problem if the same attributes flip every run, often empty vs null, reordered rules, provider-populated defaults, or API-normalized casing. Jump to Fixes → Provider defaults/API normalization differ from config.

1. Decide ownership before changing anything. If the resource was created manually during an incident, ask one question: should Terraform own this exact object long-term? If yes, import it. If no, either rename your Terraform resource to avoid collision or delete the manual object after confirming it is unused. Jump to Fixes → Manual resource should not be Terraform-managed and is colliding by name.

[Code] ---- 1. plan/apply fails ----> [Terraform]
  ^                                   |
  | 4. import/state mv                | 2. refresh state
  |                                   v
[State] <---- 3. compare IDs ---- [Cloud Resource]

Fixes

Resource exists remotely but is not in Terraform state

Add the resource block to code first, matching the real object as closely as possible, then import it.

Example resource and import:

resource "aws_s3_bucket" "assets" {
  bucket = "my-app-assets-prod"
}

terraform import aws_s3_bucket.assets my-app-assets-prod
terraform plan

For module resources or for_each, import to the full address:

terraform import 'module.storage.aws_s3_bucket.assets["prod"]' my-app-assets-prod

If import succeeds, Terraform prints output like:

aws_s3_bucket.assets: Importing from ID "my-app-assets-prod"...
aws_s3_bucket.assets: Import prepared!
  Prepared aws_s3_bucket for import
aws_s3_bucket.assets: Refreshing state... [id=my-app-assets-prod]

Import successful!

Then update config until terraform plan is clean or only shows intentional changes.

Verify it worked:

terraform plan -detailed-exitcode; echo $?

Exit code 0 means no diff; 2 means remaining changes still need review.

Resource was changed manually after Terraform created it

First inspect the exact drift:

terraform plan -refresh-only

Then choose one path:

• Keep the manual change: update Terraform code to match the remote object.
• Revert the manual change: run normal terraform apply from reviewed code.

For emergency-only fields you intentionally allow operators to change, use lifecycle.ignore_changes narrowly:

resource "aws_autoscaling_group" "web" {
  # ...
  lifecycle {
    ignore_changes = [desired_capacity]
  }
}

Do not use broad ignore_changes = all; it hides real drift.

Verify it worked:

terraform plan -refresh-only

It should show No changes. Your infrastructure still matches the configuration.

Wrong workspace/state/backend

Point Terraform at the correct backend and workspace before doing anything else.

terraform init -reconfigure
terraform workspace list
terraform workspace select prod

If you recently changed backend config, migrate state explicitly:

terraform init -migrate-state

Inspect a known object in state:

terraform state show module.network.aws_vpc.main | sed -n '1,20p'

If the IDs belong to the wrong account/region/environment, stop and fix credentials/backend config.

Verify it worked:

terraform workspace show && terraform state list | head

The workspace and resource inventory should match the target environment.

Resource address changed in code

If the remote object is correct and only the Terraform address changed, move state instead of recreating infrastructure.

terraform state mv 'module.network.aws_security_group.web[0]' 'aws_security_group.web["prod"]'
terraform plan

For module renames:

terraform state mv 'module.old.aws_db_instance.main' 'module.database.aws_db_instance.main'

⚠️ terraform state mv changes state immediately. It does not touch the cloud resource, but a wrong move can make Terraform manage the wrong object and cause destructive actions on the next apply.

If you are on a recent Terraform version, prefer declarative moved blocks for long-term maintainability:

moved {
  from = module.network.aws_security_group.web[0]
  to   = aws_security_group.web["prod"]
}

Verify it worked:

terraform plan

The destroy/create pair should disappear.

Provider defaults/API normalization differ from config

Normalize your config to what the API returns, or ignore only the unstable attribute if there is no better option.

Common examples:

• Explicitly set booleans/defaults the API populates.
• Sort or split rule resources instead of inline unordered blocks.
• Use provider-recommended standalone resources for repeated nested items.

Example of replacing inline security group rules with standalone resources to reduce reordering drift:

resource "aws_security_group" "web" {
  name   = "web-prod"
  vpc_id = aws_vpc.main.id
}

resource "aws_security_group_rule" "https_ingress" {
  type              = "ingress"
  from_port         = 443
  to_port           = 443
  protocol          = "tcp"
  cidr_blocks       = ["0.0.0.0/0"]
  security_group_id = aws_security_group.web.id
}

If the provider documents an attribute as computed or unstable, ignore just that field:

lifecycle {
  ignore_changes = [tags_all]
}

Verify it worked:

terraform plan -refresh-only

Repeated no-op drift should stop.

Manual resource should not be Terraform-managed and is colliding by name

If Terraform should create a different object, remove the collision.

Option A: delete the manual object if it is unused.

⚠️ Deleting a cloud resource can cause downtime or data loss. Confirm attachments/dependencies first with your provider CLI and maintenance window rules.

Examples:

aws s3 rb s3://my-app-assets-prod --force
aws ec2 delete-security-group --group-id sg-0abc1234
az resource delete --ids "/subscriptions/.../resourceGroups/rg-prod/providers/.../my-resource"

Option B: rename your Terraform-managed resource to a non-colliding name.

resource "aws_s3_bucket" "assets" {
  bucket = "my-app-assets-prod-tf"
}

Option C: if the manual object must remain and should be managed, import it instead of creating a new one.

Verify it worked:

terraform apply -target=aws_s3_bucket.assets

It should create/import cleanly without "already exists" errors.

Prevention

• Add a drift check job in CI for every main branch change and nightly on production workspaces:

terraform init -input=false
terraform plan -refresh-only -detailed-exitcode -no-color
case $? in
  0) echo "no drift" ;;
  2) echo "drift detected"; exit 1 ;;
  *) echo "plan failed"; exit 2 ;;
esac

• Lock down manual mutation paths with least-privilege IAM. Give most engineers read-only cloud console access in production and require break-glass roles for writes. The practical check is to remove broad *:* write permissions and keep Terraform runner/service-account as the normal writer.
• Require import/state moves in code review when refactoring resource addresses. Put moved blocks in the same PR as module renames or count/for_each migrations:

moved {
  from = aws_instance.web[0]
  to   = aws_instance.web["prod"]
}

• Persist plan artifacts and review them before apply. In CI:

terraform plan -out=tfplan
terraform show -json tfplan > tfplan.json
jq '.resource_changes[] | {address, actions: .change.actions}' tfplan.json

This catches unexpected destroy/create pairs caused by drift or address mistakes.

• Tag or label Terraform-owned resources consistently and alert on untagged production resources. Example tag pattern in code:

tags = {
  ManagedBy = "terraform"
  Service   = "my-app"
  Env       = "prod"
}

Then query your provider inventory for prod resources missing ManagedBy=terraform and review them weekly.

• Store backend config and workspace selection in automation, not human memory. Example wrapper:

export TF_WORKSPACE=prod
terraform init -backend-config=backend-prod.hcl -input=false
terraform workspace select "$TF_WORKSPACE" || terraform workspace new "$TF_WORKSPACE"

This removes a common source of "drift" that is really just the wrong state file.

This article was written by an AI system and published pending human review. Verify anything you intend to act on.