https://developer.hashicorp.com/terraform/language/backend/s3
https://developer.hashicorp.com/terraform/cli/commands/init
https://registry.terraform.io/providers/hashicorp/aws/latest/docs
https://github.com/hashicorp/terraform-provider-aws/blob/main/website/docs/index.html.markdown
https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucketnamingrules.html

Terraform AWS S3 backend:
    S3 bucket:
        stores terraform.tfstate
        versioning should be enabled for recovery from accidental state overwrite/delete

    state key:
        path inside the bucket, for example order/prod/default/ap-east-1/network/terraform.tfstate

    use_lockfile:
        enables S3-native state locking
        creates a .tflock object while Terraform is running
        deletes the .tflock object when the run finishes
        correct option is use_lockfile, not user_lockfile

scope:
    this page covers AWS provider guardrails and S3 backend config
    includes manual S3 backend bucket bootstrap
    AWS resource tag design is covered in Terraform AWS Tags
    AWS resource naming pattern is covered in Terraform AWS Naming
    S3 bucket naming and bucket policy are covered in Terraform AWS S3 Bucket
    IAM role design and AssumeRole usage are covered in Terraform AWS IAM Role
    IAM user manual Terraform migration is covered in Terraform AWS IAM User Manual Terraform
    GCP and Azure should have their own backend pages

region guardrails:
    provider region must be explicit
    aws_region variable must have validation
    provider alias must bind to a specific region/account
    backend key must include workload region
    CI should run aws sts get-caller-identity and aws configure get region / AWS_REGION check

provider "aws" {
  region = var.aws_region

  allowed_account_ids = [
    var.aws_account_id
  ]

  default_tags {
    tags = local.common_tags
  }
}

variable "aws_region" {
  type        = string
  description = "AWS region for this root module"

  validation {
    condition = contains([
      "ap-east-1",
      "ap-southeast-1"
    ], var.aws_region)

    error_message = "aws_region must be one of: ap-east-1, ap-southeast-1."
  }
}

variable "aws_account_id" {
  type        = string
  description = "Expected AWS account ID for this environment"
}

aws_region     = "ap-east-1"
aws_account_id = "123456789012"

why use allowed_account_ids:
    prevents using wrong AWS_PROFILE / credentials
    catches accidental dev credentials while applying prod
    catches CI role misconfiguration early
    makes account boundary explicit in code review

why validate region:
    prevents typo or unapproved region such as ap-east-2
    prevents accidental deploy to a region not approved for this system
    makes multi-region expansion explicit in code review

what it is not:
    not IAM authorization
    not SCP / permission boundary
    not a replacement for backend bucket/key isolation
    not a replacement for reviewing terraform plan

provider "aws" {
  region = "ap-east-1"

  allowed_account_ids = [
    "123456789012"
  ]

  assume_role {
    role_arn     = "arn:aws:iam::123456789012:role/TerraformProvisionRole"
    session_name = "terraform-prod-default"
  }

  default_tags {
    tags = {
      Environment = "prod"
      ManagedBy   = "terraform"
    }
  }
}

good cases:
    same environment, same account, multiple workload regions
    same environment, cross-account dependency such as shared DNS / shared network
    bootstrap module that intentionally wires two tightly related accounts

avoid:
    dev and prod in the same root module
    uat and prod in the same root module
    many unrelated accounts in one root module
    using aliases to avoid proper state/workspace/root-module separation

provider "aws" {
  region = "ap-east-1"

  allowed_account_ids = ["123456789012"]
}

provider "aws" {
  alias  = "singapore"
  region = "ap-southeast-1"

  allowed_account_ids = ["123456789012"]
}

module "prod_hong_kong_ecs" {
  source = "./modules/ecs-service"

  providers = {
    aws = aws
  }
}

module "prod_singapore_ecs" {
  source = "./modules/ecs-service"

  providers = {
    aws = aws.singapore
  }
}

provider "aws" {
  region = "ap-east-1"

  allowed_account_ids = ["123456789012"]
}

provider "aws" {
  alias  = "shared_dns"
  region = "ap-east-1"

  allowed_account_ids = ["999999999999"]

  assume_role {
    role_arn     = "arn:aws:iam::999999999999:role/terraform-shared-dns"
    session_name = "terraform-prod-order"
  }
}

module "service_dns" {
  source = "./modules/route53-record"

  providers = {
    aws = aws.shared_dns
  }
}

rule:
    provider aliases should still be part of one coherent deployment boundary
    if a plan can change dev and prod together, the root module boundary is wrong
    split different environments into different root modules and different backend keys

aws sts get-caller-identity
aws configure get region

terraform init
terraform plan -var-file=envs/prod.tfvars

failure pattern:
    terraform uses credentials for account 111111111111
    provider allowed_account_ids only contains 123456789012
    plan/apply fails before managing resources

    terraform uses aws_region = eu-west-1
    variable validation only allows ap-east-1 / ap-southeast-1
    validate/plan fails before provider manages resources

related option:
    forbidden_account_ids is the inverse guardrail
    allowed_account_ids conflicts with forbidden_account_ids
    prefer allowed_account_ids for production root modules
    there is no matching allowed_regions option in the AWS provider

production rule:
    every AWS root module should set allowed_account_ids
    every AWS root module should validate aws_region
    each env tfvars should include expected aws_account_id
    each env tfvars should include expected aws_region
    CI should pass the same env tfvars used for the reviewed plan
    do not set skip_requesting_account_id in normal AWS accounts
    do not set skip_region_validation in normal AWS regions

recommended:
    one state bucket per AWS account
    bucket lives in one fixed home region
    state key carries env / env_variant / workload region / component

example buckets:
    acme-tfstate-111111111111-ap-east-1   # dev account
    acme-tfstate-222222222222-ap-east-1   # uat account
    acme-tfstate-333333333333-ap-east-1   # prod account

why:
    account boundary is clear
    IAM and bucket policy are simpler
    blast radius is smaller than one central bucket
    bucket count stays manageable
    workload resources can still be multi-region because region is in the key

default decision:
    use per account + fixed home region
    put account ID and bucket region in bucket name
    put environment, environment variant, workload region, and component in state key

recommended pattern:
    <company>-aws-terraform-<account_alias>-<account_id>-tfstate-<state_bucket_region>

example:
    acme-aws-terraform-prod-123456789012-tfstate-ap-east-1

bucket rules:
    use lowercase letters, numbers, and hyphen
    include AWS account ID
    include state bucket region
    do not include secret/project codename
    do not include environment if one bucket stores many env/variant keys
    do not include workload region if state bucket region is fixed

principle:
    bucket boundary first
    key pattern second
    do not repeat account/env/region in key if bucket already provides that boundary
    include workload_region in key when one bucket stores states for multiple workload regions

account:
    account alias or account ID

env_group:
    dev / uat / prod

env_variant:
    default / market / hotfix / perf

workload_region:
    ap-east-1 / ap-southeast-1 / global

system:
    order / payment / platform

component:
    network / ecs / rds / iam / route53

bucket:
    acme-tfstate-central-999999999999-ap-east-1

key pattern:
    <account_alias>/<env_group>/<env_variant>/<workload_region>/<system>/<component>/terraform.tfstate

examples:
    dev-app/dev/default/ap-east-1/order/network/terraform.tfstate
    uat-app/uat/market/ap-east-1/order/ecs/terraform.tfstate
    prod-app/prod/default/ap-southeast-1/order/ecs/terraform.tfstate

key must include:
    account
    env_group
    env_variant
    workload_region

because bucket does not provide those boundaries

bucket:
    acme-tfstate-111111111111-ap-east-1     # dev account
    acme-tfstate-222222222222-ap-east-1     # uat account
    acme-tfstate-333333333333-ap-east-1     # prod account

key pattern:
    <system>/<env_group>/<env_variant>/<workload_region>/<component>/terraform.tfstate

examples:
    order/dev/default/ap-east-1/network/terraform.tfstate
    order/uat/default/ap-east-1/network/terraform.tfstate
    order/uat/market/ap-east-1/network/terraform.tfstate
    order/uat/hotfix/ap-east-1/network/terraform.tfstate
    order/prod/default/ap-east-1/ecs/terraform.tfstate

key does not need account:
    account boundary is already in bucket name and AWS account policy

keep env_group:
    useful when one account contains multiple env groups or variants
    can be omitted only if the account is strictly one environment

bucket:
    acme-tfstate-333333333333-ap-east-1

bucket region:
    ap-east-1

workload regions:
    ap-east-1
    ap-southeast-1
    global

key pattern:
    <system>/<env_group>/<env_variant>/<workload_region>/<component>/terraform.tfstate

examples:
    order/prod/default/ap-east-1/network/terraform.tfstate
    order/prod/default/ap-southeast-1/ecs/terraform.tfstate
    platform/prod/default/global/iam/terraform.tfstate

important:
    backend region = S3 bucket region
    workload_region in key = region where resources are deployed
    these two can be different

bucket:
    acme-tfstate-333333333333-ap-east-1
    acme-tfstate-333333333333-ap-southeast-1

key pattern:
    <system>/<env_group>/<env_variant>/<component>/terraform.tfstate

examples:
    order/prod/default/network/terraform.tfstate
    order/prod/default/ecs/terraform.tfstate

key can omit workload_region:
    bucket already represents account + state bucket region
    use only if workload resources in that state are in the same region as the bucket

rules:
    one key = one Terraform root module state
    split by ownership boundary and blast radius
    keep workload_region in key when bucket stores multiple workload regions
    use global as workload_region for global resources
    keep env_variant if variants such as uat-market / uat-hotfix exist
    include account only when one bucket stores multiple accounts
    do not put every AWS resource into one giant state file

terraform {
  backend "s3" {
    bucket       = "acme-tfstate-123456789012-ap-east-1"
    key          = "order/prod/default/ap-east-1/network/terraform.tfstate"
    region       = "ap-east-1"
    encrypt      = true
    use_lockfile = true
  }
}

fields:
    bucket:
        S3 bucket that stores state
        enable versioning on this bucket

    key:
        object path for this root module state

    region:
        S3 bucket region

    encrypt:
        asks S3 backend to use server-side encryption

    use_lockfile:
        enables S3-native locking with a .tflock object

required:
    versioning enabled
    server-side encryption enabled
    public access blocked
    access limited to Terraform roles/admins
    lifecycle policy reviewed, but do not expire current state
    use_lockfile = true for S3-native state locking

avoid:
    unversioned state bucket

manual bootstrap order:
    create bucket
    enable versioning
    enable default encryption
    block public access
    enforce bucket owner object ownership
    attach bucket policy
    configure backend.hcl
    run terraform init -backend-config=backend.hcl

export AWS_REGION=ap-east-1
export STATE_BUCKET=acme-tfstate-123456789012-ap-east-1

aws s3api create-bucket \
  --bucket "$STATE_BUCKET" \
  --region "$AWS_REGION" \
  --create-bucket-configuration LocationConstraint="$AWS_REGION"

note:
    us-east-1 is special and does not use LocationConstraint
    for ap-east-1 / ap-southeast-1 / other regions, include LocationConstraint

aws s3api put-bucket-versioning \
  --bucket "$STATE_BUCKET" \
  --versioning-configuration Status=Enabled

aws s3api put-bucket-encryption \
  --bucket "$STATE_BUCKET" \
  --server-side-encryption-configuration '{
    "Rules": [
      {
        "ApplyServerSideEncryptionByDefault": {
          "SSEAlgorithm": "AES256"
        }
      }
    ]
  }'

SSE-KMS:
    use customer managed KMS key when central audit/key control is required
    Terraform roles then also need kms:Decrypt and kms:GenerateDataKey

aws s3api put-public-access-block \
  --bucket "$STATE_BUCKET" \
  --public-access-block-configuration \
  BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true

aws s3api put-bucket-ownership-controls \
  --bucket "$STATE_BUCKET" \
  --ownership-controls '{
    "Rules": [
      {
        "ObjectOwnership": "BucketOwnerEnforced"
      }
    ]
  }'

why:
    disables ACL-based ownership complexity
    bucket owner owns state and lockfile objects
    simpler for centralized state bucket operations

aws s3api get-bucket-versioning \
  --bucket "$STATE_BUCKET"

aws s3api get-bucket-encryption \
  --bucket "$STATE_BUCKET"

aws s3api get-public-access-block \
  --bucket "$STATE_BUCKET"

aws s3api get-bucket-policy \
  --bucket "$STATE_BUCKET"

terraform init -backend-config=backend.hcl

bucket       = "acme-tfstate-123456789012-ap-east-1"
key          = "order/prod/default/ap-east-1/network/terraform.tfstate"
region       = "ap-east-1"
encrypt      = true
use_lockfile = true

terraform init -backend-config=backend.hcl

why backend.hcl:
    backend block does not support normal Terraform variables
    backend.hcl keeps env-specific backend values outside reusable module code
    do not put AWS credentials in backend.hcl

bucket       = "acme-tfstate-111111111111-ap-east-1"
key          = "order/dev/default/ap-east-1/network/terraform.tfstate"
region       = "ap-east-1"
encrypt      = true
use_lockfile = true

bucket       = "acme-tfstate-222222222222-ap-east-1"
key          = "order/uat/market/ap-east-1/ecs/terraform.tfstate"
region       = "ap-east-1"
encrypt      = true
use_lockfile = true

bucket       = "acme-tfstate-333333333333-ap-east-1"
key          = "order/prod/default/ap-east-1/ecs/terraform.tfstate"
region       = "ap-east-1"
encrypt      = true
use_lockfile = true

bucket       = "acme-tfstate-333333333333-ap-east-1"
key          = "order/prod/default/ap-southeast-1/ecs/terraform.tfstate"
region       = "ap-east-1"
encrypt      = true
use_lockfile = true

note:
    region above is the S3 state bucket region
    ap-southeast-1 in key is the workload region

bucket       = "acme-tfstate-central-999999999999-ap-east-1"
key          = "prod-app/prod/default/ap-east-1/order/network/terraform.tfstate"
region       = "ap-east-1"
encrypt      = true
use_lockfile = true

bucket       = "acme-tfstate-333333333333-ap-southeast-1"
key          = "order/prod/default/ecs/terraform.tfstate"
region       = "ap-southeast-1"
encrypt      = true
use_lockfile = true

terraform plan/apply starts:
    create lock object:
        order/prod/default/ap-east-1/network/terraform.tfstate.tflock

another terraform process starts:
    sees the lock object
    fails/waits instead of writing state at the same time

terraform plan/apply finishes:
    deletes the .tflock object

notes:
    use_lockfile is the Terraform backend option
    user_lockfile is not a Terraform backend option
    lockfile is stored in the same S3 bucket as state
    S3 Object Lock is not required for use_lockfile
    do not apply Object Lock retention/legal hold to .tflock objects
    lockfile name is derived from the state key:
        <key>.tflock

    example:
        key:
            order/prod/default/ap-east-1/network/terraform.tfstate

        lockfile:
            order/prod/default/ap-east-1/network/terraform.tfstate.tflock

    this is Terraform S3 backend behavior, not a project-specific convention

terraform init -backend-config=backend.hcl -migrate-state

terraform state list

aws s3api list-object-versions \
  --bucket acme-tfstate-123456789012-ap-east-1 \
  --prefix order/prod/default/ap-east-1/network/

rollback:
    keep a local backup of terraform.tfstate before migration
    do not delete the local backup until remote state is verified
    if migration fails, fix backend config and rerun terraform init -migrate-state