VictoriaMetrics vmauth

验证：未记录

适用版本: 未记录
验证范围: 尚未登记验证记录。

本文缩写（英文全称与中文含义）

缩写	英文全称／名称	中文含义
API	Application Programming Interface	应用程序编程接口
CA	Certificate Authority	证书颁发机构
HTTP	Hypertext Transfer Protocol	超文本传输协议
HTTPS	Hypertext Transfer Protocol Secure	使用 TLS 加密的 HTTP
IAM	Identity and Access Management	身份与访问管理
JWT	JSON Web Token	一种携带声明的令牌格式
P1	Priority 1	优先级编号；具体严重程度按团队规范定义
P2	Priority 2	优先级编号；具体严重程度按团队规范定义
RBAC	Role-Based Access Control	基于角色的访问控制
SLO	Service Level Objective	服务等级目标
TLS	Transport Layer Security	传输层安全协议
UI	User Interface	用户界面
URL	Uniform Resource Locator	统一资源定位符，通常指访问地址
VMUI	VictoriaMetrics User Interface	VictoriaMetrics 查询界面
VPN	Virtual Private Network	虚拟专用网络

Links#

https://docs.victoriametrics.com/victoriametrics/vmauth/
https://docs.victoriametrics.com/victoriametrics/vmauth/#authorization
https://docs.victoriametrics.com/victoriametrics/vmauth/#routing
https://docs.victoriametrics.com/victoriametrics/vmauth/#load-balancing
https://docs.victoriametrics.com/victoriametrics/vmauth/#config-reload
https://docs.victoriametrics.com/victoriametrics/vmauth/#security

1. Important Points#

vmauth 是 VictoriaMetrics 的 HTTP proxy，用来做 authorization、routing 和 backend load balancing。

use for:
    protect VictoriaMetrics / VMUI / vmselect / vminsert endpoints
    expose one stable endpoint to Grafana / vmagent / users
    route different users or tokens to different backends
    route query and write traffic to cluster paths
    load balance across vmselect / vminsert replicas

do not use as:
    full IAM / RBAC system
    public endpoint without TLS and strong auth
    replacement for network isolation

Default:

listen port:
    8427

required flag:
    -auth.config=/path/to/auth.yml

production default:
    expose vmauth, not raw :8428 / :8481 / :8480
    put TLS at vmauth or at an authenticated edge proxy

2. Minimal Basic Auth#

auth.yml:

users:
  - username: "grafana"
    password: "change-me"
    url_prefix: "http://victoria-metrics:8428/"

Run:

docker run -d --name vmauth -p 8427:8427 -v "$PWD/auth.yml:/etc/vmauth/auth.yml:ro" victoriametrics/vmauth:latest -auth.config=/etc/vmauth/auth.yml

Verify:

curl -u grafana:change-me 'http://localhost:8427/api/v1/query?query=up'
curl -u grafana:change-me 'http://localhost:8427/vmui/'

3. Bearer Token#

适合 vmagent remote_write、automation job、Grafana datasource token。

users:
  - bearer_token: "replace-with-long-random-token"
    url_prefix: "http://victoria-metrics:8428/"

Verify:

curl -H 'Authorization: Bearer replace-with-long-random-token' 'http://localhost:8427/api/v1/query?query=up'

Security:

token:
    use long random token
    store in secret manager / Kubernetes Secret / Vault
    rotate periodically

transport:
    use HTTPS when crossing host or network boundary

4. Protect VMUI#

VMUI lives behind the same VictoriaMetrics HTTP endpoint, so it should go through vmauth.

users:
  - username: "platform"
    password: "change-me"
    url_prefix: "http://victoria-metrics:8428/"

Access:

http://vmauth.example.com/vmui/

Rule:

do:
    expose VMUI only through vmauth / VPN / private network / authenticated proxy

do not:
    expose victoria-metrics:8428/vmui/ directly to public internet

5. Cluster Routing#

VictoriaMetrics cluster uses different backends for query and write.

query:
    vmselect
    /select/<accountID>/prometheus/

write:
    vminsert
    /insert/<accountID>/prometheus/

Example for account 0:

users:
  - username: "cluster-query"
    password: "change-me"
    url_prefix:
      - "http://vmselect-1:8481/select/0/prometheus/"
      - "http://vmselect-2:8481/select/0/prometheus/"
    retry_status_codes: [500, 502, 503]

  - bearer_token: "replace-with-vmagent-token"
    url_prefix:
      - "http://vminsert-1:8480/insert/0/prometheus/"
      - "http://vminsert-2:8480/insert/0/prometheus/"
    retry_status_codes: [500, 502, 503]

Grafana datasource:

url:
    http://vmauth:8427/

auth:
    Basic Auth with cluster-query

vmagent remote_write:

remote_write:
  - url: http://vmauth:8427/api/v1/write
    authorization:
      type: Bearer
      credentials: replace-with-vmagent-token

6. Routing By Path#

Use url_map when one vmauth endpoint serves multiple components.

Same user for VictoriaLogs and VictoriaMetrics:

users:
  - username: "logs-reader"
    password: "change-me"
    url_map:
      - src_paths:
          - "/select/.*"
        url_prefix: "http://victoria-logs:9428/"

      - src_paths:
          - "/vmui/.*"
          - "/prometheus/.*"
          - "/api/v1/.*"
        url_prefix: "http://victoria-metrics:8428/"

Why:

VMUI path:
    /vmui/.*

VMUI backend API calls:
    /prometheus/api/v1/export
    /api/v1/query
    /api/v1/query_range

VictoriaLogs query path:
    /select/.*

If only /vmui/.* is configured, VMUI can load but query/export requests fail with:
    user <name> missing route for "/prometheus/api/v1/export..."

Generic multi-component route:

unauthorized_user:
  url_map:
    - src_paths:
        - "/vmui/.*"
        - "/prometheus/.*"
        - "/api/v1/query.*"
        - "/api/v1/query_range.*"
      url_prefix: "http://victoria-metrics:8428/"

    - src_paths:
        - "/vmalert/.*"
      url_prefix: "http://vmalert:8880/"

  url_prefix: "http://blackhole.local/404"

Use unauthorized_user only when auth is handled by an upstream proxy or the endpoint is private.

7. Protect vmalert And Alertmanager UI#

只想给 vmalert 和 Alertmanager 的 UI 加认证时，可以把 UI 入口放到 vmauth 后面，服务间通信继续走内网直连。

human access:
    https://monitor.example.com/vmalert/
        -> vmauth
        -> vmalert:8880

    https://monitor.example.com/alertmanager/
        -> vmauth
        -> alertmanager:9093

service traffic:
    vmalert
        -> http://alertmanager:9093/api/v2/alerts

不要让 vmalert 发送告警也绕 vmauth，除非专门给 vmalert 配好认证。UI 调试访问和告警投递链路分开，排障时更简单。

Component Flags#

vmalert UI 挂在 /vmalert：

vmalert -datasource.url=http://victoriametrics:8428 -notifier.url=http://alertmanager:9093 -rule=/etc/vmalert/rules/*.yml -http.pathPrefix=/vmalert

Alertmanager UI 挂在 /alertmanager：

alertmanager --config.file=/etc/alertmanager/alertmanager.yml --web.route-prefix=/alertmanager --web.external-url=https://monitor.example.com/alertmanager

--web.external-url 要使用用户浏览器看到的外部地址，否则 Alertmanager UI 生成的链接、跳转地址可能不正确。

Where To Put Flags#

这些不是 vmauth 配置项，而是 vmalert 和 Alertmanager 进程自己的启动参数。放在哪里取决于你的部署方式。

Docker run:

docker run -d --name vmalert -p 8880:8880 -v ./rules:/etc/vmalert/rules:ro victoriametrics/vmalert:latest -datasource.url=http://victoriametrics:8428 -notifier.url=http://alertmanager:9093 -rule=/etc/vmalert/rules/*.yml -http.pathPrefix=/vmalert

docker run -d --name alertmanager -p 9093:9093 -v ./alertmanager.yml:/etc/alertmanager/alertmanager.yml:ro prom/alertmanager:latest --config.file=/etc/alertmanager/alertmanager.yml --web.route-prefix=/alertmanager --web.external-url=https://monitor.example.com/alertmanager

Docker Compose:

services:
  vmalert:
    image: victoriametrics/vmalert:latest
    command:
      - -datasource.url=http://victoriametrics:8428
      - -notifier.url=http://alertmanager:9093
      - -rule=/etc/vmalert/rules/*.yml
      - -http.pathPrefix=/vmalert
    volumes:
      - ./rules:/etc/vmalert/rules:ro

  alertmanager:
    image: prom/alertmanager:latest
    command:
      - --config.file=/etc/alertmanager/alertmanager.yml
      - --web.route-prefix=/alertmanager
      - --web.external-url=https://monitor.example.com/alertmanager
    volumes:
      - ./alertmanager.yml:/etc/alertmanager/alertmanager.yml:ro

systemd:

[Service]
ExecStart=/usr/local/bin/vmalert -datasource.url=http://victoriametrics:8428 -notifier.url=http://alertmanager:9093 -rule=/etc/vmalert/rules/*.yml -http.pathPrefix=/vmalert

[Service]
ExecStart=/usr/local/bin/alertmanager --config.file=/etc/alertmanager/alertmanager.yml --web.route-prefix=/alertmanager --web.external-url=https://monitor.example.com/alertmanager

Kubernetes container args:

containers:
  - name: vmalert
    image: victoriametrics/vmalert:latest
    args:
      - -datasource.url=http://victoriametrics:8428
      - -notifier.url=http://alertmanager:9093
      - -rule=/etc/vmalert/rules/*.yml
      - -http.pathPrefix=/vmalert

  - name: alertmanager
    image: prom/alertmanager:latest
    args:
      - --config.file=/etc/alertmanager/alertmanager.yml
      - --web.route-prefix=/alertmanager
      - --web.external-url=https://monitor.example.com/alertmanager

Verify:

curl -u admin:change-me -I https://monitor.example.com/vmalert/
curl -u admin:change-me -I https://monitor.example.com/alertmanager/
curl -s http://alertmanager:9093/-/ready
curl -s http://vmalert:8880/metrics

Expected:

browser:
    /vmalert/ 打开 vmalert UI
    /alertmanager/ 打开 Alertmanager UI

service traffic:
    vmalert -notifier.url 仍然指向 http://alertmanager:9093
    告警投递不依赖 https://monitor.example.com/alertmanager/

vmauth Config#

同时匹配带 / 和不带 / 的路径，避免访问 /vmalert 或 /alertmanager 时找不到 route。

users:
  - username: "admin"
    password: "change-me"
    url_map:
      - src_paths:
          - "/vmalert"
          - "/vmalert/.*"
        url_prefix: "http://vmalert:8880/"

      - src_paths:
          - "/alertmanager"
          - "/alertmanager/.*"
        url_prefix: "http://alertmanager:9093/"

Access:

https://monitor.example.com/vmalert/
https://monitor.example.com/alertmanager/

Debug:

vmalert UI:
    查看 rule 是否 loaded
    查看 alert 是否 firing
    查看 query / datasource 是否正常

Alertmanager UI:
    查看 alert 是否收到
    查看 alert 是否被 silence / inhibit
    查看 labels 是否匹配预期 route

Checklist:

do:
    raw vmalert:8880 和 alertmanager:9093 只暴露在内网
    人访问 UI 走 vmauth / VPN / authenticated edge proxy
    vmalert -notifier.url 使用内网 Alertmanager 地址
    Alertmanager external-url 使用浏览器访问的外部 URL

do not:
    不要把 vmalert -> Alertmanager 的告警投递链路强行绕到 vmauth
    不要只配置 /vmalert/.*，漏掉 /vmalert
    不要只配置 /alertmanager/.*，漏掉 /alertmanager

8. systemd#

Directory:

sudo useradd --system --home /var/lib/vmauth --shell /usr/sbin/nologin vmauth
sudo mkdir -p /etc/vmauth /var/lib/vmauth
sudo chown -R vmauth:vmauth /var/lib/vmauth
sudo chmod 0755 /etc/vmauth

Install binary:

sudo install -o root -g root -m 0755 vmauth /usr/local/bin/vmauth
/usr/local/bin/vmauth --version

Config:

sudo tee /etc/vmauth/auth.yml >/dev/null <<'EOF'
users:
  - username: "logs-reader"
    password: "change-me"
    url_map:
      - src_paths:
          - "/select/.*"
        url_prefix: "http://127.0.0.1:9428/"

      - src_paths:
          - "/vmui/.*"
          - "/prometheus/.*"
          - "/api/v1/.*"
        url_prefix: "http://127.0.0.1:8428/"
EOF

sudo chown root:vmauth /etc/vmauth/auth.yml
sudo chmod 0640 /etc/vmauth/auth.yml

Unit:

[Unit]
Description=VictoriaMetrics vmauth
Documentation=https://docs.victoriametrics.com/victoriametrics/vmauth/
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=vmauth
Group=vmauth
ExecStart=/usr/local/bin/vmauth -auth.config=/etc/vmauth/auth.yml -httpListenAddr=0.0.0.0:8427
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartSec=5s
LimitNOFILE=65536
NoNewPrivileges=true
ProtectSystem=full
ProtectHome=true
PrivateTmp=true

[Install]
WantedBy=multi-user.target

Install unit:

sudo tee /etc/systemd/system/vmauth.service >/dev/null <<'EOF'
[Unit]
Description=VictoriaMetrics vmauth
Documentation=https://docs.victoriametrics.com/victoriametrics/vmauth/
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=vmauth
Group=vmauth
ExecStart=/usr/local/bin/vmauth -auth.config=/etc/vmauth/auth.yml -httpListenAddr=0.0.0.0:8427
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartSec=5s
LimitNOFILE=65536
NoNewPrivileges=true
ProtectSystem=full
ProtectHome=true
PrivateTmp=true

[Install]
WantedBy=multi-user.target
EOF

Start:

sudo systemctl daemon-reload
sudo systemctl enable --now vmauth
sudo systemctl status vmauth --no-pager

Verify:

curl -u logs-reader:change-me 'http://127.0.0.1:8427/vmui/'
curl -u logs-reader:change-me 'http://127.0.0.1:8427/api/v1/query?query=up'
curl -u logs-reader:change-me 'http://127.0.0.1:8427/select/logsql/query?query=*'

Reload after config change:

sudo vmauth -auth.config=/etc/vmauth/auth.yml -dryRun
sudo systemctl reload vmauth
sudo journalctl -u vmauth -n 100 --no-pager

Operations:

sudo systemctl restart vmauth
sudo systemctl stop vmauth
sudo journalctl -u vmauth -f

9. TLS#

Terminate TLS at vmauth:

vmauth -auth.config=/etc/vmauth/auth.yml -httpListenAddr=0.0.0.0:443 -tls -tlsCertFile=/etc/vmauth/tls/server.crt -tlsKeyFile=/etc/vmauth/tls/server.key -tlsMinVersion=TLS12

Backend TLS:

users:
  - username: "grafana"
    password: "change-me"
    url_prefix: "https://victoria-metrics:8428/"

If backend uses private CA, install that CA into the vmauth host/container trust store.

Avoid in production:

tls_insecure_skip_verify: true

10. Reload#

Recommended reload:

kill -HUP "$(pidof vmauth)"

Alternative:

curl -X POST 'http://localhost:8427/-/reload?authKey=change-me'

When using /-/reload, protect it:

vmauth -auth.config=/etc/vmauth/auth.yml -reloadAuthKey=change-me

11. Monitoring#

endpoints:
    /metrics

watch:
    request rate by status code
    backend 5xx / timeout
    vmauth process restarts
    config reload errors
    p95 / p99 proxy latency

Scrape example:

scrape_configs:
  - job_name: vmauth
    static_configs:
      - targets:
          - vmauth:8427

Alert defaults:

P1:
    backend 5xx or timeout > 0 for 5m on production query/write path
    vmauth down for 2m

P2:
    p95 latency above dashboard SLO for 10m
    config reload failed

12. Production Checklist#

access:
    raw VictoriaMetrics endpoints are private
    vmauth is the only exposed VictoriaMetrics entrypoint
    TLS enabled when crossing network boundary

auth:
    Basic Auth / Bearer token / JWT configured
    tokens stored in secret manager
    no weak demo password in production

routing:
    query traffic goes to vmselect
    write traffic goes to vminsert
    accountID path is explicit for cluster

reliability:
    multiple vmselect / vminsert backends configured when available
    retry_status_codes configured for transient backend errors
    config reload procedure tested

observability:
    /metrics scraped
    5xx / timeout / latency / restarts alerted