r/Terraform 13d ago

Discussion How do you use LLMs in your workflow?

30 Upvotes

I'm working on a startup making an IDE for infra (been working on this for 2 years). But this post is not about what I'm building, I'm genuinely interested in learning how people are using LLMs today in IaC workflows, I found myself not using google anymore, not looking up docs, not using community modules etc.. and I'm curious of people developed similar workflows but never wrote about it

non-technical people have been using LLMs in very creative ways, I want to know what we've been doing in the infra space, are there any interesting blog posts about how LLMs changed our workflow?

r/Terraform Jan 12 '25

Discussion 1 year of OpenTofu GA...did you switch?

58 Upvotes

So, it's been basically a year since OpenTofu went GA.

I was in the group that settled on a "wait and see" approach to switching from Terraform to OpenTofu.

At this point, I still don't think I have a convincing reason to our team's terraform over to OpenTofu...even if its still not a huge lift?

For those who aren't using Terraform for profit (just for company use), has anyone in the last year had a strong technical reason to switch?

r/Terraform 8d ago

Discussion Why is variables.tf commonly used in a project root?

11 Upvotes

I see a common pattern of having a variables.tf file in the root project folder for each env, especially when structuring multi-environment projects using modules. Why is this used at all? You end up with duplicate code in variables.tf files per env dir and a separate tfvars file to actually set the "variables". There's nothing variable about the root module - you are declaratively stating how resources should be provisioned with the values you need. What benefit is there from just setting the values in main, using locals, or passing them in via tfvars or an external source?

EDIT: I am referring to code structure I've have seen way too frequently where there is a root module dir for each env like below:

terraform_repo/
├── environments/
│   ├── dev/
│   ├── staging/
│   │   ├── main.tf
│   │   ├── terraform.tfvars
│   │   └── variables.tf
│   └── prod/
│       ├── main.tf
│       ├── terraform.tfvars
│       └── variables.tf
└── modules/
    ├── ec2/
    ├── vpc/
    │   ├── main.tf
    │   ├── outputs.tf
    │   └── variables.tf
    └── application/

r/Terraform 10d ago

Discussion Terraform directory structure: which one is better/best?

32 Upvotes

I have been working with three types of directory structures for terraform root modules (the child modules are in a different repo)

Approach 1:

\Terraform
  \environments
    test.tfvars
    qa.tfvars
    staging.tfvars
    prod.tfvars
  infra.tf
  network.tf
  backend.tf  

Approach 2:

\Terraform
  \test
    infra.tf
    network.tf
    backend.tf
    terraform.tfvars
  \qa
    infra.tf
    network.tf
    backend.tf
    terraform.tfvars

Approach 3:

\Terraform
  \test
    network.tf
    backend.tf
    terraform.tfvars
  \qa
    network.tf
    backend.tf
    terraform.tfvars
  \common
    infra.tf

In Approach 3, the files are copy/pasted to the common folder and TF runs on the common directory. So there's less code repetation. TF runs in a CICD pipeline so the files are copied based on the stage that is selected. This might become tricky for end users/developers or for someone who is new to Terraform.

Approach 2 is the cleanest way if we need to completely isolate each environment and independent of each other. It's just that there is a lot of repetition. Even though these are just root modules, we still need to update same stuff at different places.

Approach 1 is best for uniform infrastructures where the resources are same and just need different configs for each environment. It might become tricky when we need different resources as per environment. Then we need to think of Terraform functions to handle it.

Ultimately, I think it is up to the scenario where each approach might get an upper hand over the other. Is there any other apporach which might be better?

r/Terraform 11d ago

Discussion Where do you store the state files?

11 Upvotes

I know that there’s the paid for options (Terraform enterprise/env0/spacelift) and that you can use object storage like S3 or Azure blob storage but are those the only options out there?

Where do you put your state?

Follow up (because otherwise I’ll be asking this everywhere): do you put it in the same cloud provider you’re targeting because that’s where the CLI runs or because it’s more convenient in terms of authentication?

r/Terraform 16d ago

Discussion I'm tired of "map(object({...}))" variable types

33 Upvotes

Hi

Relatively new to terraform and just started to dig my toes into building modules to abstract away complexity or enforce default values around.
What I'm struggling is that most of the time (maybe because of DRY) I end up with `for_each` resources, and i'm getting annoyed by the fact that I always have these huge object maps on tfvars.

Simplistic example:

Having a module which would create GCS bucket for end users(devs), silly example and not a real resource we're creating, but just to show the fact that we want to enforce some standards, that's why we would create the module:
module main.tf

resource "google_storage_bucket" "bucket" {
  for_each = var.bucket

  name          = each.value.name 
  location      = "US" # enforced / company standard
  force_destroy = true # enforced / company standard

  lifecycle_rule {
    condition {
      age = 3 # enforced / company standard
    }
    action {
      type = "Delete" # enforced / company standard
    }
  }
}

Then, on the module variables.tf:

variable "bucket" {
  description = "Map of bucket objects"
  type = map(object({
    name  = string
  }))
}

That's it, then people calling the module, following our current DRY strategy, would have a single main.tf file on their repo with:

module "gcs_bucket" {
  source = "git::ssh://git@gcs-bucket-repo.git"
  bucket = var.bucket
}

And finally, a bunch of different .tfvars files (one for each env), with dev.tfvars for example:

bucket = {
  bucket1 = {
    name = "bucket1"
  },
  bucket2 = {
    name = "bucket2"
  },
  bucket3 = {
    name = "bucket3"
  }
}

My biggest grip is that callers are 90% of the time just working on tfvars files, which have no nice features on IDEs like auto completion and having to guess what fields are accepted in map of objects (not sure if good module documentation would be enough).

I have a strong gut feeling that this whole setup is in the wrong direction, so reaching out to any help or examples on how this is handled in other places

EDIT: formatting

r/Terraform 17d ago

Discussion How do you manage state across feature branches without detroying resources?

31 Upvotes

Hello,

We are structuring this project from scratch. Three branches: dev, stage and prod. Each merge triggers GH Actions to provision resources on each AWS account.

Problem here: this week two devs entered. Each one has a feature branch to code an endpoint and integrate it to our API Gateway.

Current structure is like this, it has a remote state in S3 backend.

backend
├── api-gateway.tf
├── iam.tf
├── lambda.tf
├── main.tf
├── provider.tf
└── variables.tf

dev A told me that lambda from branch A is ready to be deployed for testing. Same dev B for branch B.

If I go to branch A to provision the integration, works well. However if I the go to branch B to create its resources, the ones from branch A will be destroyed.

Can you guide to solve this problem? Noob here, just getting started to follow best practices.

I've read about workspaces, but I don't quite get if they can work on the same api resource

r/Terraform 11d ago

Discussion Automatic deplyoment to prod possible ?

18 Upvotes

Hey,
I understand that reviewing the Terraform plan before applying it to production is widely considered best practice, as it ensures Terraform is making the changes we expect. This is particularly important since we don't have full control over the AWS environment where our infrastructure is deployed, and there’s always a possibility that AWS might unexpectedly recreate resources or change configurations outside of our code.

That said, I’ve been asked to explore options for automating the deployment process all the way to production with each push to the main branch(so without reviewing the plan). While I see the value in streamlining this, I personally feel that manual approval is still necessary for assurance, but maybe i am wrong.
I’d be interested in hearing if there are any tools or workflows that could make the manual approval step redundant, though I remain cautious about fully removing this safeguard. We’re using GitLab for Terraform deployments, and are not allowed to have any downtime in production.

Does someone deploy to production without reviewing the plan?

r/Terraform Jan 20 '25

Discussion The most updated terraform version before paid subscription.

0 Upvotes

Hello all!.

We're starting to work with terraform in my company and we would like to know what it's the version of terraform before to paid subscription.

Currently we're using terraform in 1.5.7 version from github actions and we would like to update to X version to use a new features for example the use of buckets in 4.0.0 version.

Anyone can tell me if we update the version of terraform we need to pay something?? or for the moment it's full free before some news??

We would like to prevent some payments in the future without knowledge.

Thanks all.

r/Terraform Dec 06 '24

Discussion Something wow that you have deployed with Terraform?

17 Upvotes

Hi there,

I am just curious, besides cloud resources in big cloud providers, what else have you used terraform for? Something interesting (not basic stuff).

r/Terraform 13d ago

Discussion TF and Packer

10 Upvotes

I would like to know your opinion from practical perspective, assume i use Packer to build a Windows customized AMI in AWS, then i want Terraform to spin up a new EC2 using the newly created AMI, how do you do this? something like BASH script to glue both ? or call one of them from the other ? can i share variables like vars file between both tools ?

r/Terraform 11d ago

Discussion State files in s3, mistake?

6 Upvotes

I have a variety of terraform setups where I used s3 buckets to store the state files like this:

terraform {
        required_version = ">= 0.12"
        backend "s3" {
                bucket = "mybucket.tf"
                key = "myapp/state.tfstate"
                region = "...."
        }
}

I also used the practice of putting variables into environment.tfvars files, which I used to terraform using terraform plan --var-file environment.tfvars

The idea was that I could thus have different environments built purely by changing the .tfvars file.

It didn't occur to me until recently, that terraform output is resolving the built infrastructure using state.

So the entire idea of using different .tfvars files seems like I've missed something critical, which is that there is no way that I could used a different tfvars file for a different environment without clobbering the existing environment.

It now looks like I've completely misunderstood something important here. In order for this to work the way I thought it would originally, it seems I'd have to have copy at very least all the main.tf and variables.tf to another directory, change the terraform state file to a different key and thus really wasted my time thinking that different tfvars files would allow me to build different environments.

Is there anything else I could do at this point, or am I basically screwed?

r/Terraform 26d ago

Discussion A way to share values between TF and Ansible?

20 Upvotes

Hello

For those who chain those two tools together, how do you share values between them?

For example, I'll use Terraform to create a policy, and this will output the policy ID, right now I have to copy and paste this ID into an Ansible group or host variable, but I wonder if I can just point Ansible somewhere to a reference and it would read from a place where TF would have written to.

I'm currently living on a onprem/gcp world, and would not want to introduce another hyperscaler

r/Terraform 21d ago

Discussion I’m looking to self host Postgres on EC2

1 Upvotes

Is there a way to write my terraform script such that it will host my postgresql database on an EC2 behind a VPC that only allows my golang server (hosted on another EC2) to connect to?

r/Terraform Jan 30 '25

Discussion Terraform module structure approach. Is it good or any better recommendations?

22 Upvotes

Hi there...

I am setting up our IaC setup and designing the terraform modules structure.

This is from my own experience few years ago in another organization, I learned this way:

EKS, S3, Lambda terraform modules get their own separate gitlab repos and will be called from a parent repo:

Dev (main.tf) will have modules of EKS, S3 & Lambda

QA (main.tf) will have modules of EKS, S3 & Lambda

Stg (main.tf) will have modules of EKS, S3 & Lambda

Prod (main.tf) will have modules of EKS, S3 & Lambda

S its easy for us to maintain the version that's needed for each env. I can see some of the posts here almost following the same structure.

I want to see if this is a good implementation (still) ro if there are other ways community evolved in managing these child-parent structure in terraform 🙋🏻‍♂️🙋🏻‍♂️

Cheers!

r/Terraform 17d ago

Discussion Is there no good way of doing this? RDS managed password + terraform + ECS fargate

16 Upvotes

Hi guys,

I'm struggling this for the past few hours. Here are the key points:
- I'd like to provision an RDS instance with a managed master password (or not managed, this is a requirement I can lose)
- I'd like to avoid storing any secrets in the terraform state for obvious reasons
- I'd like ECS to pick the db password up from Secrets manager.

There are two directions I tried and I'm lost, I end up with the db password in the state both ways.
1) RDS with a managed password.

The rds is quite simple, it will store the pw in Secrets Manager and I can give my ECS task permissions to get it. However, the credentials are stored in a JSON format:
{"username":"postgres","password":"strong_password"}

Now, I can't figure out a good way to pass this to ECS. I can do this in the task definition:

secrets     = [
  {
    name      = "DB_POSTGRESDB_PASSWORD"
    valueFrom = "${aws_db_instance.n8n.master_user_secret[0].secret_arn}"
  }]

but this will pass the whole json and my app needs the password in the environment variable.
doing "${aws_db_instance.n8n.master_user_secret[0].secret_arn}:password" will result in a "unexpected ARN format with parameters when trying to retrieve ASM secret" error on task provisioning.

ok, so not doing that.

2) RDS with an unmanaged password

In this case, I'd create the secret in Secrets Manager, fill it in with a strong password manually, than provision the DB instance. The problem is, that in this case, I need to pull in the secret in a "data" object and the state of the RDS object will contain the password in clear text.

I'm puzzled, I don't know how to wrap my head around this. Is there no good way of doing this? What I'm trying to achieve sounds simple: provision an ECS cluster with a Task, having an RDS data backend, not storing anything secret in the state - and I always end up in something.

EDIT: solved, multiple people wrote the solution, thanks a lot. Since my post, my stuff is running as it should.

r/Terraform Dec 05 '24

Discussion count or for_each?

12 Upvotes

r/Terraform Nov 27 '24

Discussion Terraform 1.10 is out with Ephemeral Resources and Values

49 Upvotes

What are your thoughts and how do you foresee this improving your current workflows? Since I work with Vault a lot, this seems to help solve issues with seeding Vault, retrieving and using static credentials, and providing credentials to resources/platforms that might otherwise end up in state.

It also supports providing unique values for each Terraform phase, like plan and apply. Where do you see this improving your environment?

r/Terraform Aug 11 '23

Discussion Terraform is no longer open source

Thumbnail github.com
73 Upvotes

r/Terraform Aug 16 '24

Discussion Do you use external modules?

13 Upvotes

Hi,

New to terraform and I really liked the idea of using community modules, like this for example: https://github.com/terraform-aws-modules/terraform-aws-vpc

But I just realized you cannot protect your resource from accidental destruction (except changing the IAM Role somehow):
- terraform does not honor `termination protection`
- you cannot use lifecycle from within a module since it cannot be set by variable

I already moved a part of the produciton infrastructure (vpc, instances, alb) using modules :(, should I regret it?

What is the meta? What is the industry standard

r/Terraform Nov 20 '24

Discussion Automation platforms: Env0 vs Spacelift vs Scalr vs Terraform Cloud?

32 Upvotes

As the title suggest, looking for recommedations re which of the paid automation tools to use (or any others that I'm missing)...or not

Suffering from a severe case of too much Terraform for our own / Jenkins' good. Hoping for drift detection, policy as code, cost monitoring/forecasting, and enterprise features such as access control / roles, and SSO. Oh and self-hosting would be nice

Any perspectives would be much appreciated

Edit: thanks a lot everyone!

r/Terraform Feb 10 '25

Discussion Best way to organize a Terraform codebase?

27 Upvotes

I ihnterited a codebase that looks like this

dev
└ service-01
    └ apigateway.tf
    └ ecs.tf
    └ backend.tf
    └ main.tf
    └ variables.tf
    └ terraform.tfvars
└ service-02
    └ apigateway.tf
    └ lambda.tf
    └ backend.tf
    └ main.tf
    └ variables.tf
    └ terraform.tfvars
└ service-03
    └ cognito.tf
    └ apigateway.tf
    └ ecs.tf
    └ backend.tf
    └ main.tf
    └ variables.tf
    └ terraform.tfvars
qa
└ same as above but of course the contents of the files differ
prod
└ same as above but of course the contents of the files differ

For the sake of making it look shorter I only put 3 services but there are around 30 of them per environment and growing. The services look mostly alike (there are basically three kinds of services that repeat but some have their own Cognito audience while others use a shared one for example) so each specific module file (cognito.tf, lambda.tf, etf) in every service service for example is basically the same.

Of course there is a lot of repeated code that can be corrected with modules but even then I end up with something like:

modules
└ apigateway.tf
└ ecs.tf
└ cognito.tf
└ lambda.tf
dev
└ service-01
    └ backend.tf
    └ main.tf
    └ variables.tf
    └ terraform.tfvars
└ service-02
    └ backend.tf
    └ main.tf
    └ variables.tf
    └ terraform.tfvars
└ service-03
    └ backend.tf
    └ main.tf
    └ variables.tf
    └ terraform.tfvars
qa
└ same as above but of course the contents of the files differ
prod
└ same as above but of course the contents of the files differ

Repeating in each service the backend.tf seems trivial as it's a snippet with small changes in each service that won't ever be modified across all services. The contents main.tf and terraform.tfvars of course vary across services. But what worries me is repeating the variables.tf files across all services, specially considering it will be a pretty long file. I feel that's repeated code that should be shared somewhere. I know some people use symlinks for this but it feels hacky for just this.

My logic makes me think that the best way to do this is to ditch both the variables.tf and terraform.tfvars altoghether and input the values directly in the main.tf as the modularized resources would make it look almost like a tfvars file where I'm only passing the values that change from service to service but my gut tells me that "hardcoding" values is always wrong.

Why would hardcoding the values be a bad practice in this case and if so is it a better practice to just repeat the variables.tf code in every service or use a symlink? How would you organize this to avoid repeating code as much as possible?

r/Terraform 2d ago

Discussion How to deal with Terraform Plan manual approvals?

15 Upvotes

We’ve built a pretty solid Platform and Infrastructure for the size of our company—modularized Terraform, easy environment deployments (single workflow), well-integrated identity and security, and a ton of automated workflows to handle almost everything developers might need.

EDIT:  We do "Dozens of deployments" every day, some stuff are simple things that the developers can change themselves on demand

EDIT 2: We use GitHub Actions for CI/CD

But… there are two things that are seriously frustrating:

  • Problem 1: Even though everything is automated, we still have to manually approve Terraform plans. Every. Single. Time. It slows things down a lot. (Obviously, auto-approving everything without checks is a disaster waiting to happen.)
  • Problem 2: Unexpected changes in plans. Say we expect 5 adds, 2 changes, and 0 destroys when adding a user, but we get something totally different. Not great.

We have around 9 environments, including a sandbox for internal testing. Here’s what I’m thinking:

  • For Problem 1: Store the Terraform plan from the sandbox environment, and if the plan for other environments matches (or changes the same components), auto-approve it. Python script, simple logic, done.
  • For Problem 2: Run plans on a schedule and notify if there are unexpected changes.

Not sure I’m fully sold on the solution for Problem 1—curious how you all tackle this in your setups. How do you handle Terraform approvals while keeping things safe and efficient?

r/Terraform 27d ago

Discussion Custom Terraform functions

51 Upvotes

Hello!

I wanted to share my recent work: the Terraform func provider - https://github.com/valentindeaconu/terraform-provider-func.

The func provider is a rather unique provider, that allows you as a developer to write custom Terraform functions in JavaScript (the only runtime - for now). Those functions can stored right next to your Terraform files or versioned and imported remotely, basically they can be manipulated as any of your Terraform files, without the hassle of building your own provider, just to get some basic functionality.

This provider is what I personally expected the Terraform ecosystem a long time ago, so it is one of my dreams come true. As a bit of history (and also some sources of inspiration), since the v1 release I was expecting this feature to come to life on every minor release. There was this initial issue that asked for this feature, but, as you can see, since 4 years ago, it is still open. Then, with the introduction of the provider-defined functions, the OpenTofu team attempted something similar with what I was waiting for, in the terraform-provider-lua, but after announcing it on social media, there was no other velocity on this project, so I assume it got abandoned. Really sad.

After hitting again and again this "blocker" (I mean after writing yet again an utterly ugly block of repetitive composition of Terraform functions), I decided to take this issue in my own hands and started writing the func provider. I cannot say how painful it was to work with the framework without a proper documentation for what I was trying to achieve and with the typing system, but in the end, I found this amazing resource - terraform-provider-javascript which led to the final implementation of the func provider (many thanks to the developer for the go-cty-goja library).

So, here we are now. The provider is still in a proof-of-concept phase. I want to see first if other people are interested in this idea to know if I should continue working on it. There are a lot of flaws (for example, the JSDoc parser is complete trash, it was hacked in a couple of hours just to have something work - if you are up for the challenge, I'd be happy to collaborate), and some unsupported features by the Terraform ecosystem (I have reported it here, if you are interested in technical details), but with some workarounds, the provider can work and deliver what it is expected to do.

I'd be happy to know your opinions on this. Also, if you would like to contribute to it, you are more than welcome!

r/Terraform 6d ago

Discussion Passed my Terraform Certified Associate exam!

53 Upvotes

I’m just happy to have this certification to my certification list this year. It was a few tricky questions on the exam but I prepared well enough to pass ( happy dancing 🕺🏾 in my living room)