AWS ECS Java Auto Scaling

验证：未记录

适用版本: 未记录
验证范围: 尚未登记验证记录。

本文缩写（英文全称与中文含义）

缩写	英文全称／名称	中文含义
ALB	Application Load Balancer	应用负载均衡器
API	Application Programming Interface	应用程序编程接口
CPU	Central Processing Unit	中央处理器
DB	Database	数据库
ECS	Elastic Container Service	AWS 容器编排服务
GC	Garbage Collection	垃圾回收
HTTP	Hypertext Transfer Protocol	超文本传输协议
JVM	Java Virtual Machine	Java 虚拟机
RPS	Requests Per Second	每秒请求数
SLO	Service Level Objective	服务等级目标
vCPU	Virtual CPU	虚拟处理器

Links#

1. Scaling Strategy#

Java on ECS 的主扩缩容指标通常选 request rate 或 CPU。JVM memory 更适合做 guardrail，因为 heap 预留、GC 行为和 actual demand 不一定线性对应。

Workload	Primary Scaling Metric	Target
HTTP API behind ALB	`ALBRequestCountPerTarget`	load test safe RPS per task 的 60-70%
CPU-bound service	`ECSServiceAverageCPUUtilization`	55-65%
worker / consumer	custom `backlog_per_task`	backlog 在业务 SLO 内清空
memory-bound batch	custom business metric + memory guard	不把 JVM heap usage 单独作为默认主指标

Java guardrails:

p95 latency:
    <= service SLO

CPU:
    average <= 70% at target load
    sustained > 80% means scale-out or optimize

GC:
    p95 pause stays inside latency budget
    GC time ratio should not keep rising under stable RPS

heap:
    old gen after full GC should stay stable
    container memory headroom >= 20-25%

threads:
    servlet / reactive worker pool not saturated
    DB connection pool wait time near zero at target load

Decision:

JVM 内存占用不直接等于业务需求；扩容同时看请求/队列压力、CPU、GC 和数据库连接预算，避免只按 memory 扩容。

Spring Boot / HTTP service:
    use ALBRequestCountPerTarget when traffic drives load
    use CPU when per-request cost varies heavily

async worker:
    use queue backlog per task
    alert on oldest message age

avoid:
    memory-only scaling for normal JVM services
    high maxCapacity without checking DB pool and downstream quotas

2. ECS Policy#

aws application-autoscaling register-scalable-target --service-namespace ecs --scalable-dimension ecs:service:DesiredCount --resource-id service/prod-apps/order-api --min-capacity 2 --max-capacity 30 --region ap-east-1

CPU target tracking:

aws application-autoscaling put-scaling-policy --service-namespace ecs --scalable-dimension ecs:service:DesiredCount --resource-id service/prod-apps/order-api --policy-name order-api-java-cpu --policy-type TargetTrackingScaling --target-tracking-scaling-policy-configuration '{
    "TargetValue": 60,
    "PredefinedMetricSpecification": {
      "PredefinedMetricType": "ECSServiceAverageCPUUtilization"
    },
    "ScaleOutCooldown": 90,
    "ScaleInCooldown": 300
  }' --region ap-east-1

ALB request count target tracking:

aws application-autoscaling put-scaling-policy --service-namespace ecs --scalable-dimension ecs:service:DesiredCount --resource-id service/prod-apps/order-api --policy-name order-api-java-rps-per-task --policy-type TargetTrackingScaling --target-tracking-scaling-policy-configuration '{
    "TargetValue": 180,
    "PredefinedMetricSpecification": {
      "PredefinedMetricType": "ALBRequestCountPerTarget",
      "ResourceLabel": "app/prod-public-alb/abc123/targetgroup/prod-order-api/def456"
    },
    "ScaleOutCooldown": 90,
    "ScaleInCooldown": 300
  }' --region ap-east-1

3. Load Test For Threshold#

Goal: 找到 JVM warmup 后的 steady-state capacity。

test setup:
    desiredCount = 1
    same task CPU/memory as production
    same JAVA_TOOL_OPTIONS as production
    enable GC logs or export JVM metrics
    run warmup traffic before measuring

increase load:
    warmup 10-20m
    50 RPS -> 100 -> 150 -> 200 -> 250
    each measured step >= 15m

stop point:
    p95 latency > SLO
    5xx >= 1%
    CPU avg >= 75%
    GC pause breaks latency budget
    connection pool wait appears
    old gen after GC keeps growing

Example result:

task size:
    1 vCPU / 2GB

observed:
    250 RPS: p95 220ms, CPU 64%, GC p95 pause 35ms
    320 RPS: p95 530ms, CPU 82%, DB pool wait appears

safe RPS per task:
    250

autoscaling target:
    250 * 0.70 = 175
    choose ALBRequestCountPerTarget target = 175-180

Useful commands:

hey -z 15m -c 150 https://api.example.com/orders

aws logs tail /ecs/order-api --since 30m --filter-pattern '"gc" "pause"' --region ap-east-1

4. Readiness Checklist#

scaling:
    minCapacity >= 2
    maxCapacity checked against DB pool, Redis, downstream API quota
    request-count and CPU policies do not fight each other

jvm:
    heap limit is container-aware
    MaxRAMPercentage / InitialRAMPercentage are explicit
    graceful shutdown waits for in-flight requests

observability:
    ALB RPS / latency / 5xx
    ECS CPU / memory
    JVM heap / non-heap
    GC pause / GC time ratio
    thread pool and DB pool metrics