1. Dashboard Rule#

Folder: Messaging
Dashboard: SQS - Overview

Rows:
    Overview:
        backlog 是否增长，oldest message 是否变老，DLQ 是否有消息

    Drilldown:
        按 queue / producer / consumer 拆开看

    Capacity:
        incoming / processed rate、drain time、consumer capacity

    Debug:
        visibility timeout、empty receive、delayed message、in-flight 细节，不放第一屏；排查消费异常时查看

2. Overview#

Panel Metrics Why
Oldest message age ApproximateAgeOfOldestMessage 队列最重要的用户影响信号
Visible backlog ApproximateNumberOfMessagesVisible 等待消费的消息量
In-flight messages ApproximateNumberOfMessagesNotVisible 正在被 consumer 处理的消息量
Incoming rate NumberOfMessagesSent producer 写入速度
Processed rate NumberOfMessagesDeleted consumer 完成速度
DLQ depth DLQ ApproximateNumberOfMessagesVisible 失败消息是否堆积

Unit: ops/s#

ops/s means operations per second. In SQS dashboard, one operation usually means one message-related metric event, not a business transaction.

NumberOfMessagesSent:
    messages sent per second

NumberOfMessagesReceived:
    messages received per second

NumberOfMessagesDeleted:
    messages deleted per second

NumberOfEmptyReceives:
    empty receive calls per second

YACE collects these SQS CloudWatch metrics with Statistic: Sum. Sum is the total count inside one CloudWatch period. To show ops/s, divide by the period seconds:

ops/s = CloudWatch Sum / period_seconds

example:
    period = 300s
    NumberOfMessagesDeleted Sum = 1500
    ops/s = 1500 / 300 = 5

So:

raw Sum:
    count per CloudWatch period

ops/s:
    normalized per-second rate

3. Drilldown#

Panel Metrics Why
Queue backlog by queue ApproximateNumberOfMessagesVisible 找到具体积压 queue
Age by queue ApproximateAgeOfOldestMessage 优先处理影响最大的 queue
Sent vs deleted NumberOfMessagesSent, NumberOfMessagesDeleted 判断是 producer 激增还是 consumer 变慢
Receive behavior NumberOfMessagesReceived, NumberOfEmptyReceives 判断 polling 是否低效
In-flight saturation ApproximateNumberOfMessagesNotVisible visibility timeout / consumer 并发问题

4. Capacity#

Panel Metrics Why
Backlog trend ApproximateNumberOfMessagesVisible 判断是否持续积压
Drain time backlog / processed rate 估算清空队列需要多久
Producer trend NumberOfMessagesSent 识别业务峰值
Consumer throughput trend NumberOfMessagesDeleted 估算 worker 扩容需求
DLQ trend DLQ visible messages 识别失败率是否长期上升

5. Debug#

Panel Metrics Why
Delayed messages ApproximateNumberOfMessagesDelayed 排查 delay queue 或 message timer
Empty receives NumberOfEmptyReceives polling 参数或 consumer 空转问题
Receive count gap NumberOfMessagesReceived vs NumberOfMessagesDeleted 识别重复处理或处理失败
FIFO group pressure FIFO group / dedup related app metrics FIFO 场景下需要业务侧指标补足

6. YACE Metrics#

如果 SQS dashboard 使用 aws_sqs_* metrics，需要 YACE 采集 AWS/SQS。DLQ 也是一个普通 queue，需要把 DLQ 一起纳入 discovery。

YACE config:

apiVersion: v1alpha1
sts-region: ap-northeast-1
discovery:
  jobs:
    - type: AWS/SQS
      regions:
        - ap-northeast-1
      customTags:
        - key: environment
          value: uat
      period: 300
      length: 600
      nilToZero: true
      metrics:
        - name: ApproximateAgeOfOldestMessage
          statistics: [Average, Maximum]
        - name: ApproximateNumberOfMessagesVisible
          statistics: [Average, Maximum]
        - name: ApproximateNumberOfMessagesNotVisible
          statistics: [Average, Maximum]
        - name: ApproximateNumberOfMessagesDelayed
          statistics: [Average, Maximum]
        - name: NumberOfMessagesSent
          statistics: [Sum]
        - name: NumberOfMessagesDeleted
          statistics: [Sum]
        - name: NumberOfMessagesReceived
          statistics: [Sum]
        - name: NumberOfEmptyReceives
          statistics: [Sum]

同一个配置文件也放在:

static/Grafana/yace-sqs-config.yml

确认 metric name:

{__name__=~"aws_sqs_.*"}

7. Do Not Put In Overview#

不要放:
    每个 worker 的内部指标
    empty receive 明细
    FIFO dedup 低频指标

原因:
    SQS overview 重点是 age、backlog、throughput、DLQ
Backward CloudWatch Dashboard YACE Dashboard Forward