Links#
1. Dashboard Rule#
Folder: Compute
Dashboard: ECS Service - Overview
Rows:
Service Health:
desired 是否满足,CPU / memory 是否接近瓶颈,container 是否频繁 restart
Scaling Signals:
running / desired / pending 的关系,CPU / memory 长期趋势,deployment / capacity gap 症状为什么不保留四层:
ECS service 原生 CloudWatch metrics 不多。
如果没有接入 ALB metrics 和应用 Prometheus metrics,拆成 Overview / Drilldown / Capacity / Debug 很容易重复。
所以 ECS service 这类 dashboard 更适合压成 1 到 2 个 row:
第一行回答服务当前是否健康
第二行回答是否需要扩缩容,或者 deployment / capacity provider 是否有问题2. Overview#
| Panel | Metrics | Why |
|---|---|---|
| Service running tasks | RunningTaskCount, DesiredTaskCount |
running 小于 desired 是服务容量不足或部署异常 |
| CPU usage | CPUUtilization |
ECS service 的 compute saturation |
| Memory usage | MemoryUtilization |
ECS service 最常见 scaling / OOM signal |
| Request rate | app http_requests_total or ALB RequestCount |
没有流量上下文时 CPU / memory 很难判断 |
| Error rate | app 5xx or ALB HTTPCode_Target_5XX_Count |
用户侧失败信号 |
| Latency | app latency p95/p99 or ALB TargetResponseTime |
用户侧慢请求信号 |
request / error / latency 来源:
ECS namespace:
不提供完整 request / error / latency。
ECS metrics 主要回答 service / task 资源和运行状态。
ALB, if service is HTTP behind ALB:
RequestCount
HTTPCode_Target_5XX_Count
HTTPCode_ELB_5XX_Count
TargetResponseTime
App metrics, recommended:
http_requests_total
http_request_duration_seconds
error counter by route / status / exception3. Dashboard Panels#
| Row | Panel | Metrics | Why |
|---|---|---|---|
| Service Health | Running vs desired tasks | RunningTaskCount, DesiredTaskCount |
第一优先级先看服务有没有满足 desired |
| Service Health | CPU utilization | CPUUtilization |
判断 CPU 是否接近 target tracking 或 service saturation |
| Service Health | Memory utilization | MemoryUtilization |
最常见的 ECS autoscaling / OOM 风险信号 |
| Service Health | Pending task count | PendingTaskCount |
发现部署卡住、容量不足、任务无法及时拉起 |
| Scaling Signals | Task count by service | RunningTaskCount, DesiredTaskCount, PendingTaskCount |
区分扩容中、部署卡住、capacity 不足 |
| Scaling Signals | CPU / memory trend | CPUUtilization, MemoryUtilization |
判断现有 scaling target 是否合适 |
| Scaling Signals | Scaling / deployment symptoms | DesiredTaskCount - RunningTaskCount, PendingTaskCount |
快速识别 desired-running gap 和 pending 压力 |
4. YACE Metrics#
如果 ECS dashboard 使用 aws_ecs_* metrics,需要 YACE 采集 AWS/ECS。应用请求量、错误率、延迟仍然建议由应用 /metrics 或 ALB metrics 提供。
YACE config:
apiVersion: v1alpha1
sts-region: ap-northeast-1
discovery:
jobs:
- type: AWS/ECS
regions:
- ap-northeast-1
customTags:
- key: environment
value: uat
period: 300
length: 600
nilToZero: true
metrics:
- name: CPUUtilization
statistics: [Average]
- name: MemoryUtilization
statistics: [Average]
- type: ECS/ContainerInsights
regions:
- ap-northeast-1
customTags:
- key: environment
value: uat
period: 300
length: 600
nilToZero: true
metrics:
- name: RunningTaskCount
statistics: [Average]
- name: DesiredTaskCount
statistics: [Average]
- name: PendingTaskCount
statistics: [Average]Container Insights note:
下面这些指标不属于标准 AWS/ECS 这段 YACE 配置:
RunningTaskCount
DesiredTaskCount
PendingTaskCount
它们来自:
ECS/ContainerInsights
如果 dashboard 要展示 task count / pending,
需要同时采:
AWS/ECS
ECS/ContainerInsights如果 ECS service behind ALB,并且 dashboard 要显示 request / error / latency,YACE 还需要采集 AWS/ApplicationELB。ALB 的详细 YACE 配置见 ALB YACE Dashboard。
确认 metric name:
{__name__=~"aws_ecs_.*"}5. Do Not Put In Overview#
不要放:
每个 container 的所有 cgroup counters
runtime GC / heap / goroutine / event loop 全量明细
每个 deployment event 的长文本
原因:
ECS overview 应该优先回答 desired 是否满足、资源是否饱和、用户体验是否变差