Go Auto Scaling

Links#

1. Scaling Strategy#

Go 服务通常 CPU signal 很干净，但 HTTP 服务仍优先用 request rate per task，因为它更接近业务 demand。

Workload	Primary Scaling Metric	Target
HTTP API behind ALB	`ALBRequestCountPerTarget`	load test safe RPS per task 的 60-75%
CPU-bound API / gRPC	`ECSServiceAverageCPUUtilization`	60-70%
SQS / stream worker	custom `backlog_per_task`	backlog 在 SLO 内下降
memory-sensitive service	memory guard + Go runtime metrics	不用 RSS spike 直接推导 demand

Go guardrails:

p95 latency:
    <= service SLO

CPU:
    target load <= 70%
    sustained > 80% means scale-out or profile

goroutine:
    count stable under stable traffic
    no unbounded goroutine growth

GC:
    pause stays inside latency budget
    heap goal / live heap trend stable

memory:
    use GOMEMLIMIT below ECS memory limit
    keep container memory headroom >= 20%

Decision:

web API:
    use ALBRequestCountPerTarget for predictable routing capacity
    use CPU policy when request cost varies heavily

worker:
    use backlog_per_task
    add oldest-message-age alert

runtime:
    set GOMAXPROCS intentionally when task vCPU is small
    set GOMEMLIMIT to avoid container OOM

2. ECS Policy#

aws application-autoscaling register-scalable-target \
  --service-namespace ecs \
  --scalable-dimension ecs:service:DesiredCount \
  --resource-id service/prod-apps/order-api \
  --min-capacity 2 \
  --max-capacity 25 \
  --region ap-east-1

ALB request count target tracking:

aws application-autoscaling put-scaling-policy \
  --service-namespace ecs \
  --scalable-dimension ecs:service:DesiredCount \
  --resource-id service/prod-apps/order-api \
  --policy-name order-api-go-rps-per-task \
  --policy-type TargetTrackingScaling \
  --target-tracking-scaling-policy-configuration '{
    "TargetValue": 600,
    "PredefinedMetricSpecification": {
      "PredefinedMetricType": "ALBRequestCountPerTarget",
      "ResourceLabel": "app/prod-public-alb/abc123/targetgroup/prod-order-api/def456"
    },
    "ScaleOutCooldown": 60,
    "ScaleInCooldown": 300
  }' \
  --region ap-east-1

CPU target tracking:

aws application-autoscaling put-scaling-policy \
  --service-namespace ecs \
  --scalable-dimension ecs:service:DesiredCount \
  --resource-id service/prod-apps/order-api \
  --policy-name order-api-go-cpu \
  --policy-type TargetTrackingScaling \
  --target-tracking-scaling-policy-configuration '{
    "TargetValue": 65,
    "PredefinedMetricSpecification": {
      "PredefinedMetricType": "ECSServiceAverageCPUUtilization"
    },
    "ScaleOutCooldown": 60,
    "ScaleInCooldown": 300
  }' \
  --region ap-east-1

3. Load Test For Threshold#

Goal: 找到 latency 还稳定时的 per-task throughput。

test setup:
    desiredCount = 1
    same task CPU/memory as production
    same GOMAXPROCS / GOMEMLIMIT as production
    expose pprof or runtime metrics in test env

increase load:
    100 RPS -> 300 -> 600 -> 900 -> 1200
    each step >= 10m

stop point:
    p95 latency > SLO
    5xx >= 1%
    CPU avg >= 80%
    goroutine count keeps growing
    GC pause or heap growth breaks baseline

Example result:

task size:
    1 vCPU / 512MB

observed:
    800 RPS: p95 160ms, CPU 62%, memory stable
    1000 RPS: p95 420ms, CPU 83%, goroutine count rising

safe RPS per task:
    800

autoscaling target:
    800 * 0.75 = 600
    choose ALBRequestCountPerTarget target = 600

Useful commands:

hey -z 10m -c 300 https://api.example.com/orders

go tool pprof -top http://localhost:6060/debug/pprof/profile?seconds=30

4. Readiness Checklist#

scaling:
    minCapacity >= 2
    maxCapacity checked against DB and downstream quotas
    scale-in cooldown slower than scale-out

go runtime:
    GOMAXPROCS matches task CPU allocation
    GOMEMLIMIT lower than ECS memory limit
    SIGTERM graceful shutdown implemented

observability:
    ALB RPS / latency / 5xx
    ECS CPU / memory
    goroutine count
    GC pause / heap allocation
    task restart count