How to Filter Log Levels in Alertmanager's Webhook Notifications?

Note:
This topic has been translated from a Chinese forum by GPT and might contain errors.

Original topic: Alertmanager的webhook通知中如何过滤日志级别?

| username: TiDBer_ssvwtrcq

[TiDB Usage Environment] Test
[TiDB Version] v6.5.2

[Resource Configuration]
Configuration content of alertmanager.yml:

global:
  # The smarthost and SMTP sender used for mail notifications.
  smtp_smarthost: "localhost:25"
  smtp_from: "alertmanager@example.org"
  smtp_auth_username: "alertmanager"
  smtp_auth_password: "password"
  # smtp_require_tls: true
  # The Slack webhook URL.
  # slack_api_url: ''

route:
  # A default receiver
  receiver: "blackhole"

  # The labels by which incoming alerts are grouped together. For example,
  # multiple alerts coming in for cluster=A and alertname=LatencyHigh would
  # be batched into a single group.
  group_by: ["env", "instance", "alertname", "type", "group", "job"]

  # When a new group of alerts is created by an incoming alert, wait at
  # least 'group_wait' to send the initial notification.
  # This way ensures that you get multiple alerts for the same group that start
  # firing shortly after another are batched together on the first
  # notification.
  group_wait: 30s

  # When the first notification was sent, wait 'group_interval' to send a batch
  # of new alerts that started firing for that group.
  group_interval: 3m

  # If an alert has successfully been sent, wait 'repeat_interval' to
  # resend them.
  repeat_interval: 3m

  routes:
    - match:
        severity: "warning,critical,emergency"
      receiver: webhook-qt
      continue: true

receivers:
  - name: 'webhook-qt'
    webhook_configs:
    - send_resolved: true
      url: 'http://xxxx/xxx'

  # This doesn't alert anything, please configure your own receiver
  - name: "blackhole"

[Problem Description]
After configuring severity: “warning,critical,emergency”, the webhook does not receive any messages. When this item is not configured, many messages are received.
The version of alertmanager installed by tiup is 0.17. The minimum version on the Prometheus official site is 0.20, and the highest is 0.25. Any documentation for version 0.17 can no longer be found.

I would like to ask:
1. How to filter the message levels for the webhook?
2. What are the monitoring levels configured in TiDB?

| username: WalterWj | Original post link

| username: dba-kit | Original post link

Here you need to write it as a regular expression, and the label name should also be changed to level: ^warning|critical|emergency$

    - match_re:
        level: "^warning|critical|emergency$"
| username: dba-kit | Original post link

You can go to the conf directory of the Prometheus node to grep and count. Currently, there are three levels: warning|critical|emergency.

| username: TiDBer_ssvwtrcq | Original post link

Thank you all, handling it with regular expressions has already taken effect.

| username: system | Original post link

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.