Organizing With Multiple States

We'll be seeing what it looks like to move to a more advanced way of managing infrastructure in Terraform - using multiple states (state files) to manage multiple "areas" of infrastructure.

My example is a bit contrived but you should be able to see why this might be useful in multi-team environments that have responsibilities in different areas.

How will we divide our states?

1. Compute - ec2, autoscale, lambda, server stuff
2. Network - VPC, security groups, gateways, peering, etc
3. Data - RDS, Elasticache, S3, etc

State Files vs Workspaces

To divide up our infrastructure, we can make each one a different state.

We won't use workspaces for this. As Terraform docs suggest:

Named workspaces allow conveniently switching between multiple instances of a single configuration within its single backend. They are convenient in a number of situations, but cannot solve all problems.

Why Multiple States?

Out-of-band changes: There are scenarios where you want to make changes to one area of Terraform, but get "blocked" by changes in another.

For example, you may have been forced to update the underlying database engine in your RDS instance. If terraform is out of date, it will try to destroy and re-create your RDS database (bad!).

If you're running Terraform locally, you can update your RDS data, or use the -target option. This can be painful if you have to target a LOT of things (you can't currently exclude).

If you've automated Terraform runs, then this becomes even more painful.

It's something you can work around, but human error can create issues.

A similar reason here is that things that don't change much can be in their own state.

More Control: You may want one team to control the compute tier, but lock down important tiers like those controlling your databases to another team (in addition to using lifecycle rules to prevent deletions, and AWS options to prevent deletion).

Shared Resource: You may want multiple environments/copies of infrastructure to share a common resource, such as an RDS database or VPC.

Multiple state files for each of these can help you organize those.

Directory Structure

Here's the directory structure we'll use for managing multiple state "areas":

 1- root
 2  |- modules
 3    |- vpc
 4    |- ec2
 5    |- rds
 6  |- production
 7    |- compute
 8    |- network
 9    |- data
10  |- staging
11    |- compute
12    |- network
13    |- data

A few things to note:

1. Each item - compute, network, and data - are their own separate state files.
2. We'll make a new module to create an RDS database.
3. We deleted the backend configurations, and will replace them with alternative arguments in the terraform command.