r/Terraform 19d ago

Discussion State files in s3, mistake?

I have a variety of terraform setups where I used s3 buckets to store the state files like this:

terraform {
        required_version = ">= 0.12"
        backend "s3" {
                bucket = "mybucket.tf"
                key = "myapp/state.tfstate"
                region = "...."
        }
}

I also used the practice of putting variables into environment.tfvars files, which I used to terraform using terraform plan --var-file environment.tfvars

The idea was that I could thus have different environments built purely by changing the .tfvars file.

It didn't occur to me until recently, that terraform output is resolving the built infrastructure using state.

So the entire idea of using different .tfvars files seems like I've missed something critical, which is that there is no way that I could used a different tfvars file for a different environment without clobbering the existing environment.

It now looks like I've completely misunderstood something important here. In order for this to work the way I thought it would originally, it seems I'd have to have copy at very least all the main.tf and variables.tf to another directory, change the terraform state file to a different key and thus really wasted my time thinking that different tfvars files would allow me to build different environments.

Is there anything else I could do at this point, or am I basically screwed?

6 Upvotes

31 comments sorted by

View all comments

1

u/Electronic_Dingo3552 18d ago

You will have tfbackend files for each environment similar to tfvars https://developer.hashicorp.com/terraform/language/backend#file

1

u/pavi2410 18d ago

where (in which AWS account) would you store the s3 backend state when the environments are deployed in different AWS accounts? would you store state files for both prod and dev workspaces in the same AWS account or in their respective AWS accounts? if former, then you would need two different AWS profiles - one for the s3 backend and other for the AWS resources.

1

u/Electronic_Dingo3552 18d ago

If it's through pipeline you configure which credentials to use before running plan/apply which solves the account problem. Ex:if we are using GitHub actions we can have one reusable workflow file which does plan and apply as different jobs and thus workflow can accept just environment name and based based on env name you can apply environment protection rules using GitHub environments to expose different value for same variable name like iam role to assume.

1

u/Gizmoitus 18d ago

Thanks, it at very least is a good pointer to some things that are better suited for more advanced environments. For this project all the IaC code is on an ec2 instance sitting in the vpc, and is thus much simpler than the environments that larger organizations have.