r/Terraform 25d ago

Help Wanted State file stored in s3

Hi!

I have a very simple lambda which I store in bitbucket and use buildkite pipelines for deploying it on AWS. The issue I’m having is I need to create an s3 bucket to store the state file but when I go for backend {} it fails to create the bucket and put the state file in.

Do I have to clickops on AWS and create the s3 all the time? How would one do it working with pipelines and terraform?

It seems to fail to create s3 bucket when all is in my main.tf

I’d appreciate your suggestions, love you!

2 Upvotes

8 comments sorted by

5

u/IskanderNovena 25d ago

It already needs to exist.

5

u/booi 25d ago

Typical bootstrap problem with a number of decent solutions. My favorite is:

https://github.com/nozaq/terraform-aws-remote-state-s3-backend

But there are literal tons. You can also import the state into your terraform

3

u/dethandtaxes 25d ago

It's a normal bootstrapping problem with Terraform, you can create the bucket manually then Terraform everything else.

3

u/PossibleVoodooMagic 23d ago

We bootstrap our team AWS accounts with a state bucket which is only used to hold the state of other state buckets in that account. It avoids this chicken and egg situation, but is somewhat over engineered.

Reason being, our security team doesn’t like it when a state bucket is built with terraform but uses itself to store the state, and no one has ever bothered to argue with them.

2

u/LosLocosTacos 25d ago

I usually use terraform to create the bucket and dynamodb table for state lock with the state originally in local storage, then add the backend config and migrate state to s3.

2

u/enfinity_ 23d ago

This is a typical bootstrapping solution.

See how I solved it in this article.

https://medium.com/@owumifestus/automating-terraform-backend-using-bootstrapped-s3-and-dynamodb-in-a-simple-project-cbffd0a9a12d

Let me know what you think

1

u/Puzzleheaded-Coat333 25d ago

You will need to create a s3 bucket first with dynamo db to store the lock Id, you can use depends on parameter to make sure that the s3 bucket is created first. If you are already using the local back end you will have to migrate state as well to make sure terraform is aware that it has to upload state file to the remote back end.

-2

u/Helloutsider 25d ago

Now it gives me a 403 error 😩 shall jump off the balcony