Node.js Auto Scaling

Links#

1. Scaling Strategy#

Node.js ECS service 不要只看 memory。先按 workload 找 demand metric，再用 CPU / memory / event loop 做 guardrail。

Workload	Primary Scaling Metric	Target
HTTP API behind ALB	`ALBRequestCountPerTarget`	load test 得到的 safe RPS per task 的 60-70%
CPU-heavy API	`ECSServiceAverageCPUUtilization`	55-65%
SQS / async worker	custom `backlog_per_task`	`queue_visible_messages / running_tasks` <= 单 task 在 SLO 内可处理量
memory-heavy stable workload	`ECSServiceAverageMemoryUtilization`	65-75%，只在确认不是 leak 后使用

Node.js guardrails:

p95 latency:
    <= service SLO, for example 300ms / 500ms

5xx rate:
    < 1% during load test

event loop lag p95:
    < 100ms preferred
    >= 200ms means CPU/blocking code risk

CPU:
    p70-p90 should not stay above 70% before scale-out

memory:
    heap_used / heap_limit < 75%
    RSS stable after warmup

Decision:

web API:
    use ALBRequestCountPerTarget first
    add CPU high alarm
    add event loop lag app metric

worker:
    use backlog_per_task first
    alert on ApproximateAgeOfOldestMessage

do not:
    use memory as default primary metric
    scale out to hide memory leak
    set target from one short benchmark run

2. ECS Policy#

aws application-autoscaling register-scalable-target \
  --service-namespace ecs \
  --scalable-dimension ecs:service:DesiredCount \
  --resource-id service/prod-apps/order-api \
  --min-capacity 2 \
  --max-capacity 20 \
  --region ap-east-1

HTTP API with ALB request count:

aws application-autoscaling put-scaling-policy \
  --service-namespace ecs \
  --scalable-dimension ecs:service:DesiredCount \
  --resource-id service/prod-apps/order-api \
  --policy-name order-api-rps-per-task \
  --policy-type TargetTrackingScaling \
  --target-tracking-scaling-policy-configuration '{
    "TargetValue": 300,
    "PredefinedMetricSpecification": {
      "PredefinedMetricType": "ALBRequestCountPerTarget",
      "ResourceLabel": "app/prod-public-alb/abc123/targetgroup/prod-order-api/def456"
    },
    "ScaleOutCooldown": 60,
    "ScaleInCooldown": 300
  }' \
  --region ap-east-1

CPU fallback:

aws application-autoscaling put-scaling-policy \
  --service-namespace ecs \
  --scalable-dimension ecs:service:DesiredCount \
  --resource-id service/prod-apps/order-api \
  --policy-name order-api-cpu \
  --policy-type TargetTrackingScaling \
  --target-tracking-scaling-policy-configuration '{
    "TargetValue": 60,
    "PredefinedMetricSpecification": {
      "PredefinedMetricType": "ECSServiceAverageCPUUtilization"
    },
    "ScaleOutCooldown": 60,
    "ScaleInCooldown": 300
  }' \
  --region ap-east-1

3. Load Test For Threshold#

Goal: 找到 safe RPS per task，不是找极限峰值。

test setup:
    desiredCount = 1
    same task cpu/memory as production
    same container image and NODE_OPTIONS
    hit ALB / real health path / main business API
    downstream dependency use staging-size or controlled mock

increase load:
    50 RPS -> 100 -> 200 -> 300 -> 400
    each step >= 10m
    warmup 3-5m ignored

stop point:
    p95 latency > SLO
    5xx >= 1%
    event loop lag p95 >= 200ms
    CPU avg >= 75%
    memory grows continuously after traffic stabilizes

Example result:

task size:
    0.5 vCPU / 1GB

observed:
    400 RPS: p95 280ms, CPU 68%, event_loop_lag_p95 80ms
    500 RPS: p95 620ms, CPU 86%, event_loop_lag_p95 240ms

safe RPS per task:
    400

autoscaling target:
    400 * 0.70 = 280
    choose ALBRequestCountPerTarget target = 280-300

Useful commands:

hey -z 10m -c 200 https://api.example.com/orders

aws cloudwatch get-metric-statistics \
  --namespace AWS/ECS \
  --metric-name CPUUtilization \
  --dimensions Name=ClusterName,Value=prod-apps Name=ServiceName,Value=order-api \
  --statistics Average Maximum \
  --period 60 \
  --start-time 2026-06-17T02:00:00Z \
  --end-time 2026-06-17T02:30:00Z \
  --region ap-east-1

4. Readiness Checklist#

scaling:
    minCapacity >= 2 for production API
    maxCapacity matches DB / Redis / external API capacity
    scale-out cooldown short, scale-in cooldown slower

node runtime:
    NODE_ENV=production
    NODE_OPTIONS sets heap/diagnostic flags intentionally
    graceful shutdown handles SIGTERM

observability:
    ALB request count / latency / 5xx
    ECS CPU / memory
    event loop lag
    heap usage
    task restart count