AlertOps and Logz.io
Logz.io observability platform allows customers to configure customizable alerts based on log query criteria.
AlertOps’ alert management system can be integrated with Logz.io to receive and respond to all (predefined status mappings) alerts through email, SMS, push notification or phone alerts. AlertOps would ensure that the alert/job status would reach the appropriate team by using proper workflows, escalation policies and schedules. Based on your ruleset, incidents can be automatically opened and closed, depending on whether Logz.io reports a problem or a recovery
The above scenario and scope for integration is due to the fact that AlertOps has a very flexible and simple API/Webhook configuration feature that can be leveraged with Logz.io's monitoring, notification and alerting capabilities.
AlertOps - Inbound Integration
We can define some rulesets in AlertOps so that Logz.io can send out monitoring alerts to the AlertOps platform. AlertOps would ensure based on these notifications received, that it would always reach out and assign to the correct person/team by utilizing its escalation policies, schedules, and workflow features.
AlertOps provides Inbound Integrations to integrate with numerous monitoring, chat and ITSM tools. You can configure an inbound integration for Logz.io alerts.
At a high level this is how the flow looks like, you define an API integration in the AlertOps platform by defining settings like Integration Name, Escalation rules, recipient users/groups. Once an integration is defined, a unique API URL is generated. This acts as webhook or the gateway through which notifications from Logz.io reach AlertOps and thus an incident/alert is created correspondingly. The API can be defined with various settings like URL mappings, filters, escalations etc. as required. Logz.io has to be defined with a notification endpoint.
To configure an Inbound Integration in AlertOps to receive alerts from Logz.io,
Under 'Integrations' select 'Inbound Integrations', select the category 'API' and then select add 'Add API Integration'
There are numerous integration options available in AlertOps, select Logz.io
Once you select the integration, you can then specify basic settings like the integration name, escalation policy, names of the recipients/groups for which the alerts must be assigned to.
Once you click save, the API Integration will be created, and you will be given a unique URL which acts as the access point and needs to be configured at the source (in this case Logz.io), to send alerts. You can find the integration you just created, and you can give advanced settings and define various configurations for the alerts to be received and processed. For example, you can define when to open and close alerts based on the payload obtained from the API call, filters etc.
Make a note of the API URL, which will be used in Logz.io, so it calls a HTTP POST request to this URL with the body in JSON format containing the alert specific information. AlertOps automatically creates an alert when the status variable (alert_severity) contains 'SEVERE' or 'HIGH'. The incident will also be closed automatically when the status 'LOW' or 'INFO' is received from Logz.io. (and updated when 'MEDIUM' is received)
You can similarly define URL mappings as you want, owing to the flexibility provided by AlertOps’ OpenAPI/Plug-and-Play integrations. You can provide other filters and match with regex expressions as well. You can also test the generated URL.
Configuration of Logz.io for the Integration,
Logz.io provides robust alerting and notification capabilities for every resource it monitors. To setup an outbound call to AlertOps for alert management you need to setup a Notification Endpoint,
In your Logz.io portal, in the left navigation pane, under 'Traces', select 'Notification Endpoints'
Select '+ Add Endpoint'
Select Endpoint Type as 'Custom', give a name, method type as 'POST' and for URL, paste the inbound integration URL which you obtained from AlertOps.
For payload body, paste the following JSON body in the text field provided,
{
"alert_title": "{{alert_title}}",
"alert_description": "{{alert_description}}",
"alert_severity": "{{alert_severity}}",
"alert_event_id": "{{alert_event_id}}",
"account_id": "{{account_id}}",
"alert_timeframe_start": "{{alert_timeframe_start}}",
"alert_timeframe_end": "{{alert_timeframe_end}}",
"alert_app_url": "{{alert_app_url}}",
"alert_event_html": "{{alert_event_html}}",
"alert_definition_id": "{{alert_definition_id}}"
}
You can run a test to make sure alerts are sent out to AlertOps. Save !
Make sure you configure the Alert itself to include the above created Notification Endpoint under the 'Notify' step (for Logs/Metrics you configure). You can also add the AlertOps endpoint from here.
Thats it! You have configured a notification endpoint and associated alerts with this endpoint. Any trigger event would now send an alert to AlertOps for incident management.
Message logs, alert specific information can be viewed in the “Inbound Log” section in AlertOps Dashboard. Alerts can be viewed in the ‘Alerts’ tab as well.
Alert Triggering Information:
AlertOps will automatically create an incident when a new alert is received from Logz.io when the alert_severity field contains "SEVERE" or "HIGH".
If an alert with status "SEVERE" or "HIGH" matches an existing Open Alert, AlertOps will recognize the new alert as a duplicate and ignore the alert.
The alert will be recorded in the Inbound Messages table as “Mapped Appended.”
AlertOps will automatically close the same incident when an alert with alert_severity contains 'LOW' or 'INFO'.