Prometheus Integration Guide
Prometheus is an open-source monitoring system with a dimensional data model, flexible query language, efficient time series database and modern alerting approach. Integrate AlertOps’ alert management platform with Prometheus to receive and respond to critical alerts through email, SMS, push notification, and phone alerts. AlertOps ensures that alerts received from Prometheus always reach the correct, available team member by utilizing escalation policies and on-call schedules.
- From the main menu, click on Integrations and then select Inbound Integrations from the sub menu.
- Select API tab
- Select the 'ADD API INTEGRATION' button.
- You should now be on the API Integration Detail page. Select Prometheus from the pre-built Integration Templates.
- Enter a name for the integration. Select an escalation rule to determine the integration's escalation policy. And, enter the names of recipient group(s) and recipient user(s). Click 'SAVE.' Then, click 'COPY URL' to copy the URL endpoint to be used in the Prometheus configuration.
Step 2: Create and configure an Alertmanager configuration file (default configuration file is labeled alertmanager.yml and located in the same directory as the Alertmanager program file. You can view an image below or find an example file on Github. Click here to view documentation.
Step 3: Create a
receiver for a webhook to post to AlertOps within your Alertmanager configuration file as shown above. Provide a nam
e for your receiver. In addition, paste the Integration URL you copied from AlertOps in the
url field, that you see, in the format below:
- name: Receiver_Name
- url: 'Integration_URL'
Step 4: You can configure the default
route within your Alertmanager configuration file to send to a single AlertOps endpoint as follows:
- group_by: [cluster]
You can configure custom
routes to send alerts to different
receivers (AlertOps Integrations) using various match conditions as follows. The example below depicts routing to the receiver when the
severity is critical:
View an example configuration that routes alerts to two different Integrations within AlertOps based on if an alert is critical or non-critical:
- name: Default_Receiver_Name
- url: 'Integration1_URL'
- name: Critical_Receiver_Name
- url: 'Integration2_URL'
Step 5: Restart the AlertManager to load your new configuration. Now you can setup targets or jobs to monitor to test your integration.
Alert Triggering Information
AlertOps will automatically create an incident when a new alert is received from Prometheus with an IncidentStatus status of “firing.”
If an alert with status “firing” matches an existing Open Alert, AlertOps will recognize the new alert as a duplicate and ignore the alert. The alert will be recorded in the Inbound Messages table as “Mapped Appended.”
AlertOps will automatically close the same incident when an alert with an IncidentStatus status “resolved” is received.
Testing and Troubleshooting
Click here to read about Web API Testing and Troubleshooting.