When configuring new receivers or testing templates in Alertmanager, you might need to generate dummy alerts to see how everything works. Instead of manually creating and sending alerts, I use a simple bash script to streamline the process.

The Script

Below is the script I use to generate and resolve dummy alerts. It sends a POST request to Alertmanager’s API to simulate an alert.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

#!/bin/bash

name=$RANDOM
url='http://localhost:9093/api/v2/alerts'
default_severity='warning'
instance='127.0.0.1'
instance_name='test-instance'

send_alert() {
    local status=$1
    local custom_severity=$2
    local current_severity=${custom_severity:-$default_severity}

    curl -XPOST $url -H "Content-Type: application/json" -d "[
        {
            \"status\": \"$status\",
            \"labels\": {
                \"alertname\": \"$name\",
                \"service\": \"my-service\",
                \"severity\":\"$current_severity\",
                \"instance\": \"$instance\",
                \"instance_name\": \"$instance_name\"
            },
            \"annotations\": {
                \"summary\": \"This is a test alert\",
                \"title\": \"Test Alert\",
                \"description\": \"Test alert has been triggered.\"
            },
            \"generatorURL\": \"https://prometheus.local/<generating_expression>\"
        }
    ]"
    echo ""
}

# Main script
echo "Firing up alert $name"
send_alert "firing" "$1"

read -p "Press enter to resolve alert"

echo "Sending resolve"
send_alert "resolved" "$1"

How It Works

  1. The script starts by generating a random alert name ($RANDOM) to ensure each alert is unique.
  2. It sets a default severity (warning) and other labels such as instance and instance_name.
  3. The send_alert function sends a POST request to Alertmanager’s API endpoint (http://localhost:9093/api/v2/alerts) with a payload containing the alert’s status, labels, and annotations.
  4. The script fires an alert, waits for user input, and then resolves the alert.

Example Output

Running the script will output something like this:

Firing up alert 26314

Press enter to resolve alert
Sending resolve

This shows that the alert has been successfully sent to Alertmanager and resolved after pressing Enter.

Use Cases

This script is especially useful when:

  • Testing new receiver configurations in Alertmanager.
  • Validating templates for notifications (e.g., email, Slack, or custom integrations).
  • Generating test data for monitoring dashboards.

Customizing the Script

  • Severity: You can pass a custom severity as an argument to the script: 
    ./alertmanager_dummy_alert.sh critical
  • Instance Information: Update the instance and instance_name variables to match your test environment.
  • Alertmanager URL: Update url variables to your actual alertmanager url.

Improving the Script

The script is flexible and can be extended with additional features as needed. For instance, you might experiment with the "startsAt" and "endsAt" fields in the payload to define specific start and end times for an alert, rather than simply sending a resolved alert.

Conclusion

Generating dummy alerts can save a lot of time and ensure that your configurations are working as expected before deploying them in a production environment. Feel free to adapt the script to suit your specific testing needs.

Happy alerting!

P.S. Original idea can be found here