vmauth 


https://docs.victoriametrics.com/victoriametrics/vmauth/
https://docs.victoriametrics.com/victoriametrics/vmauth/#authorization
https://docs.victoriametrics.com/victoriametrics/vmauth/#routing
https://docs.victoriametrics.com/victoriametrics/vmauth/#load-balancing
https://docs.victoriametrics.com/victoriametrics/vmauth/#config-reload
https://docs.victoriametrics.com/victoriametrics/vmauth/#security

1. Important Points#

vmauth 是 VictoriaMetrics 的 HTTP proxy，用来做 authorization、routing 和 backend load balancing。

use for:
    protect VictoriaMetrics / VMUI / vmselect / vminsert endpoints
    expose one stable endpoint to Grafana / vmagent / users
    route different users or tokens to different backends
    route query and write traffic to cluster paths
    load balance across vmselect / vminsert replicas

do not use as:
    full IAM / RBAC system
    public endpoint without TLS and strong auth
    replacement for network isolation

Default:

listen port:
    8427

required flag:
    -auth.config=/path/to/auth.yml

production default:
    expose vmauth, not raw :8428 / :8481 / :8480
    put TLS at vmauth or at an authenticated edge proxy

2. Minimal Basic Auth#

auth.yml:

users:
  - username: "grafana"
    password: "change-me"
    url_prefix: "http://victoria-metrics:8428/"

Run:

docker run -d \
  --name vmauth \
  -p 8427:8427 \
  -v "$PWD/auth.yml:/etc/vmauth/auth.yml:ro" \
  victoriametrics/vmauth:latest \
  -auth.config=/etc/vmauth/auth.yml

Verify:

curl -u grafana:change-me 'http://localhost:8427/api/v1/query?query=up'
curl -u grafana:change-me 'http://localhost:8427/vmui/'

3. Bearer Token#

适合 vmagent remote_write、automation job、Grafana datasource token。

users:
  - bearer_token: "replace-with-long-random-token"
    url_prefix: "http://victoria-metrics:8428/"

Verify:

curl -H 'Authorization: Bearer replace-with-long-random-token' \
  'http://localhost:8427/api/v1/query?query=up'

Security:

token:
    use long random token
    store in secret manager / Kubernetes Secret / Vault
    rotate periodically

transport:
    use HTTPS when crossing host or network boundary

4. Protect VMUI#

VMUI lives behind the same VictoriaMetrics HTTP endpoint, so it should go through vmauth.

users:
  - username: "platform"
    password: "change-me"
    url_prefix: "http://victoria-metrics:8428/"

Access:

http://vmauth.example.com/vmui/

Rule:

do:
    expose VMUI only through vmauth / VPN / private network / authenticated proxy

do not:
    expose victoria-metrics:8428/vmui/ directly to public internet

5. Cluster Routing#

VictoriaMetrics cluster uses different backends for query and write.

query:
    vmselect
    /select/<accountID>/prometheus/

write:
    vminsert
    /insert/<accountID>/prometheus/

Example for account 0:

users:
  - username: "cluster-query"
    password: "change-me"
    url_prefix:
      - "http://vmselect-1:8481/select/0/prometheus/"
      - "http://vmselect-2:8481/select/0/prometheus/"
    retry_status_codes: [500, 502, 503]

  - bearer_token: "replace-with-vmagent-token"
    url_prefix:
      - "http://vminsert-1:8480/insert/0/prometheus/"
      - "http://vminsert-2:8480/insert/0/prometheus/"
    retry_status_codes: [500, 502, 503]

Grafana datasource:

url:
    http://vmauth:8427/

auth:
    Basic Auth with cluster-query

vmagent remote_write:

remote_write:
  - url: http://vmauth:8427/api/v1/write
    authorization:
      type: Bearer
      credentials: replace-with-vmagent-token

6. Routing By Path#

Use url_map when one vmauth endpoint serves multiple components.

Same user for VictoriaLogs and VictoriaMetrics:

users:
  - username: "logs-reader"
    password: "change-me"
    url_map:
      - src_paths:
          - "/select/.*"
        url_prefix: "http://victoria-logs:9428/"

      - src_paths:
          - "/vmui/.*"
          - "/prometheus/.*"
          - "/api/v1/.*"
        url_prefix: "http://victoria-metrics:8428/"

Why:

VMUI path:
    /vmui/.*

VMUI backend API calls:
    /prometheus/api/v1/export
    /api/v1/query
    /api/v1/query_range

VictoriaLogs query path:
    /select/.*

If only /vmui/.* is configured, VMUI can load but query/export requests fail with:
    user <name> missing route for "/prometheus/api/v1/export..."

Generic multi-component route:

unauthorized_user:
  url_map:
    - src_paths:
        - "/vmui/.*"
        - "/prometheus/.*"
        - "/api/v1/query.*"
        - "/api/v1/query_range.*"
      url_prefix: "http://victoria-metrics:8428/"

    - src_paths:
        - "/vmalert/.*"
      url_prefix: "http://vmalert:8880/"

  url_prefix: "http://blackhole.local/404"

Use unauthorized_user only when auth is handled by an upstream proxy or the endpoint is private.

7. Protect vmalert And Alertmanager UI#

只想给 vmalert 和 Alertmanager 的 UI 加认证时，可以把 UI 入口放到 vmauth 后面，服务间通信继续走内网直连。

human access:
    https://monitor.example.com/vmalert/
        -> vmauth
        -> vmalert:8880

    https://monitor.example.com/alertmanager/
        -> vmauth
        -> alertmanager:9093

service traffic:
    vmalert
        -> http://alertmanager:9093/api/v2/alerts

不要让 vmalert 发送告警也绕 vmauth，除非专门给 vmalert 配好认证。UI 调试访问和告警投递链路分开，排障时更简单。

Component Flags#

vmalert UI 挂在 /vmalert：

vmalert \
  -datasource.url=http://victoriametrics:8428 \
  -notifier.url=http://alertmanager:9093 \
  -rule=/etc/vmalert/rules/*.yml \
  -http.pathPrefix=/vmalert

Alertmanager UI 挂在 /alertmanager：

alertmanager \
  --config.file=/etc/alertmanager/alertmanager.yml \
  --web.route-prefix=/alertmanager \
  --web.external-url=https://monitor.example.com/alertmanager

--web.external-url 要使用用户浏览器看到的外部地址，否则 Alertmanager UI 生成的链接、跳转地址可能不正确。

Where To Put Flags#

这些不是 vmauth 配置项，而是 vmalert 和 Alertmanager 进程自己的启动参数。放在哪里取决于你的部署方式。

Docker run:

docker run -d \
  --name vmalert \
  -p 8880:8880 \
  -v ./rules:/etc/vmalert/rules:ro \
  victoriametrics/vmalert:latest \
  -datasource.url=http://victoriametrics:8428 \
  -notifier.url=http://alertmanager:9093 \
  -rule=/etc/vmalert/rules/*.yml \
  -http.pathPrefix=/vmalert

docker run -d \
  --name alertmanager \
  -p 9093:9093 \
  -v ./alertmanager.yml:/etc/alertmanager/alertmanager.yml:ro \
  prom/alertmanager:latest \
  --config.file=/etc/alertmanager/alertmanager.yml \
  --web.route-prefix=/alertmanager \
  --web.external-url=https://monitor.example.com/alertmanager

Docker Compose:

services:
  vmalert:
    image: victoriametrics/vmalert:latest
    command:
      - -datasource.url=http://victoriametrics:8428
      - -notifier.url=http://alertmanager:9093
      - -rule=/etc/vmalert/rules/*.yml
      - -http.pathPrefix=/vmalert
    volumes:
      - ./rules:/etc/vmalert/rules:ro

  alertmanager:
    image: prom/alertmanager:latest
    command:
      - --config.file=/etc/alertmanager/alertmanager.yml
      - --web.route-prefix=/alertmanager
      - --web.external-url=https://monitor.example.com/alertmanager
    volumes:
      - ./alertmanager.yml:/etc/alertmanager/alertmanager.yml:ro

systemd:

[Service]
ExecStart=/usr/local/bin/vmalert \
  -datasource.url=http://victoriametrics:8428 \
  -notifier.url=http://alertmanager:9093 \
  -rule=/etc/vmalert/rules/*.yml \
  -http.pathPrefix=/vmalert
[Service]
ExecStart=/usr/local/bin/alertmanager \
  --config.file=/etc/alertmanager/alertmanager.yml \
  --web.route-prefix=/alertmanager \
  --web.external-url=https://monitor.example.com/alertmanager

Kubernetes container args:

containers:
  - name: vmalert
    image: victoriametrics/vmalert:latest
    args:
      - -datasource.url=http://victoriametrics:8428
      - -notifier.url=http://alertmanager:9093
      - -rule=/etc/vmalert/rules/*.yml
      - -http.pathPrefix=/vmalert

  - name: alertmanager
    image: prom/alertmanager:latest
    args:
      - --config.file=/etc/alertmanager/alertmanager.yml
      - --web.route-prefix=/alertmanager
      - --web.external-url=https://monitor.example.com/alertmanager

Verify:

curl -u admin:change-me -I https://monitor.example.com/vmalert/
curl -u admin:change-me -I https://monitor.example.com/alertmanager/
curl -s http://alertmanager:9093/-/ready
curl -s http://vmalert:8880/metrics

Expected:

browser:
    /vmalert/ 打开 vmalert UI
    /alertmanager/ 打开 Alertmanager UI

service traffic:
    vmalert -notifier.url 仍然指向 http://alertmanager:9093
    告警投递不依赖 https://monitor.example.com/alertmanager/

vmauth Config#

同时匹配带 / 和不带 / 的路径，避免访问 /vmalert 或 /alertmanager 时找不到 route。

users:
  - username: "admin"
    password: "change-me"
    url_map:
      - src_paths:
          - "/vmalert"
          - "/vmalert/.*"
        url_prefix: "http://vmalert:8880/"

      - src_paths:
          - "/alertmanager"
          - "/alertmanager/.*"
        url_prefix: "http://alertmanager:9093/"

Access:

https://monitor.example.com/vmalert/
https://monitor.example.com/alertmanager/

Debug:

vmalert UI:
    查看 rule 是否 loaded
    查看 alert 是否 firing
    查看 query / datasource 是否正常

Alertmanager UI:
    查看 alert 是否收到
    查看 alert 是否被 silence / inhibit
    查看 labels 是否匹配预期 route

Checklist:

do:
    raw vmalert:8880 和 alertmanager:9093 只暴露在内网
    人访问 UI 走 vmauth / VPN / authenticated edge proxy
    vmalert -notifier.url 使用内网 Alertmanager 地址
    Alertmanager external-url 使用浏览器访问的外部 URL

do not:
    不要把 vmalert -> Alertmanager 的告警投递链路强行绕到 vmauth
    不要只配置 /vmalert/.*，漏掉 /vmalert
    不要只配置 /alertmanager/.*，漏掉 /alertmanager

8. systemd#

Directory:

sudo useradd --system --home /var/lib/vmauth --shell /usr/sbin/nologin vmauth
sudo mkdir -p /etc/vmauth /var/lib/vmauth
sudo chown -R vmauth:vmauth /var/lib/vmauth
sudo chmod 0755 /etc/vmauth

Install binary:

sudo install -o root -g root -m 0755 vmauth /usr/local/bin/vmauth
/usr/local/bin/vmauth --version

Config:

sudo tee /etc/vmauth/auth.yml >/dev/null <<'EOF'
users:
  - username: "logs-reader"
    password: "change-me"
    url_map:
      - src_paths:
          - "/select/.*"
        url_prefix: "http://127.0.0.1:9428/"

      - src_paths:
          - "/vmui/.*"
          - "/prometheus/.*"
          - "/api/v1/.*"
        url_prefix: "http://127.0.0.1:8428/"
EOF

sudo chown root:vmauth /etc/vmauth/auth.yml
sudo chmod 0640 /etc/vmauth/auth.yml

Unit:

[Unit]
Description=VictoriaMetrics vmauth
Documentation=https://docs.victoriametrics.com/victoriametrics/vmauth/
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=vmauth
Group=vmauth
ExecStart=/usr/local/bin/vmauth \
  -auth.config=/etc/vmauth/auth.yml \
  -httpListenAddr=0.0.0.0:8427
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartSec=5s
LimitNOFILE=65536
NoNewPrivileges=true
ProtectSystem=full
ProtectHome=true
PrivateTmp=true

[Install]
WantedBy=multi-user.target

Install unit:

sudo tee /etc/systemd/system/vmauth.service >/dev/null <<'EOF'
[Unit]
Description=VictoriaMetrics vmauth
Documentation=https://docs.victoriametrics.com/victoriametrics/vmauth/
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=vmauth
Group=vmauth
ExecStart=/usr/local/bin/vmauth \
  -auth.config=/etc/vmauth/auth.yml \
  -httpListenAddr=0.0.0.0:8427
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartSec=5s
LimitNOFILE=65536
NoNewPrivileges=true
ProtectSystem=full
ProtectHome=true
PrivateTmp=true

[Install]
WantedBy=multi-user.target
EOF

Start:

sudo systemctl daemon-reload
sudo systemctl enable --now vmauth
sudo systemctl status vmauth --no-pager

Verify:

curl -u logs-reader:change-me 'http://127.0.0.1:8427/vmui/'
curl -u logs-reader:change-me 'http://127.0.0.1:8427/api/v1/query?query=up'
curl -u logs-reader:change-me 'http://127.0.0.1:8427/select/logsql/query?query=*'

Reload after config change:

sudo vmauth -auth.config=/etc/vmauth/auth.yml -dryRun
sudo systemctl reload vmauth
sudo journalctl -u vmauth -n 100 --no-pager

Operations:

sudo systemctl restart vmauth
sudo systemctl stop vmauth
sudo journalctl -u vmauth -f

9. TLS#

Terminate TLS at vmauth:

vmauth \
  -auth.config=/etc/vmauth/auth.yml \
  -httpListenAddr=0.0.0.0:443 \
  -tls \
  -tlsCertFile=/etc/vmauth/tls/server.crt \
  -tlsKeyFile=/etc/vmauth/tls/server.key \
  -tlsMinVersion=TLS12

Backend TLS:

users:
  - username: "grafana"
    password: "change-me"
    url_prefix: "https://victoria-metrics:8428/"

If backend uses private CA, install that CA into the vmauth host/container trust store.

Avoid in production:

tls_insecure_skip_verify: true

10. Reload#

Recommended reload:

kill -HUP "$(pidof vmauth)"

Alternative:

curl -X POST 'http://localhost:8427/-/reload?authKey=change-me'

When using /-/reload, protect it:

vmauth \
  -auth.config=/etc/vmauth/auth.yml \
  -reloadAuthKey=change-me

11. Monitoring#

endpoints:
    /metrics

watch:
    request rate by status code
    backend 5xx / timeout
    vmauth process restarts
    config reload errors
    p95 / p99 proxy latency

Scrape example:

scrape_configs:
  - job_name: vmauth
    static_configs:
      - targets:
          - vmauth:8427

Alert defaults:

P1:
    backend 5xx or timeout > 0 for 5m on production query/write path
    vmauth down for 2m

P2:
    p95 latency above dashboard SLO for 10m
    config reload failed

12. Production Checklist#

access:
    raw VictoriaMetrics endpoints are private
    vmauth is the only exposed VictoriaMetrics entrypoint
    TLS enabled when crossing network boundary

auth:
    Basic Auth / Bearer token / JWT configured
    tokens stored in secret manager
    no weak demo password in production

routing:
    query traffic goes to vmselect
    write traffic goes to vminsert
    accountID path is explicit for cluster

reliability:
    multiple vmselect / vminsert backends configured when available
    retry_status_codes configured for transient backend errors
    config reload procedure tested

observability:
    /metrics scraped
    5xx / timeout / latency / restarts alerted
Backward VMUI
TLS Forward