AlertOps and Instana
AlertOps’ alert management system can be integrated with Instana to receive and respond to critical (predefined status mappings) alarms/alerts through email, SMS, push notification or phone alerts. AlertOps would ensure that the alert would reach the appropriate team by using proper workflows, escalation policies and schedules. Based on your ruleset, incidents can be automatically opened and closed, depending on whether Instana reports a problem or a recovery.
AlertOps Inbound Integration
AlertOps would ensure based on notifications received from Instana, that it would always reach out and assign to the correct person/team by utilizing its escalation policies, schedules, and workflow features. AlertOps provides Inbound Integrations to integrate with numerous monitoring, chat and ITSM tools. You can configure an inbound integration for Instana events.
At a high level this is how the flow looks. You define an API integration in the AlertOps platform by defining settings like Integration Name, Escalation rules, recipient users/groups. Once an integration is defined, a unique API URL is generated. This acts as webhook or the gateway through which events from Instana reach AlertOps and thus an incident/alert is created correspondingly. The API can be defined with various settings like URL mappings, filters, escalations etc. as required. You will have to create a Webhook in Instana to send event information.
To configure an Inbound Integration in AlertOps to receive alerts from Instana:
In the menu on left pane, select Integrations > Inbound Integrations > Add API Integration.
There are numerous integration options available in AlertOps, select Instana
Once you select the integration, you can then specify basic settings like the integration name, escalation policy, names of the recipients/groups for which the alerts must be assigned to.
Once you click save, the API Integration will be created, and you will be given a unique URL which acts as the access point and needs to be configured at the source (in this case Instana), to send notifications. You can find the integration you just created, and you can give advanced settings and define various configurations for the alerts to be received and processed. For example, you can define custom conditions as when to open and close alerts based on the response obtained from the API call, filters for the incoming JSON payload etc.
Make a note of the API URL, which will be used in Instana, so it calls a HTTP POST request to this URL with the body in JSON format containing the alert specific information. Alerts will be recorded in the ‘Inbound Log’ section table. AlertOps automatically creates an alert when the state contains “OPEN” (as per the definition in the screenshot above). The incident will also be closed automatically when the status ‘CLOSED’ is received.
You can similarly define URL mappings as you want, owing to the flexibility provided by AlertOps’ API integrations. You can provide other filters and match with regex as well. You can also test the generated URL
Configuration of Instana for AlertOps Integration
Now that we have setup AlertOps with the Instana API Inbound Integration, along with a unique API URL; we can now define a Webhook in Instana to access this API and send out events to AlertOps.
Events need to be created in Instana (or rather an inbuilt event can be picked from Instana) so that they can be attached to an Alert, which is further attached to an Alert Channel Webhook to send out event payload to AlertOps. An example is to create an event when CPU load goes above a value. There are loads of pre-built events and other ways you can define events for your monitoring environment.
In the left navigation pane, go to Settings. In this section, there is another navigation pane in the left and an option “Events and Alerts”. Select “Events”. You can see a lot of pre-built events, descriptions and types. You can define an alert for any of these or create a new event.
Click “New Event”. In the first part, give a name, description for the event. You can define the type of issue severity (Critical or Warning), for which you want to create events and consequent alerts.
The second part defines the conditions for which the event will occur. Here you define the metric source, type, metric and then the condition itself. Refer to the screenshot below for a sample condition. For more information on metrics and different queries available refer to the links in the Reference section.
In the third part, you select the scope of the event. Click Save.
You have created an event. Now you will define an Alert Channel. This is where you will create the webhook and paste the API URL which you got from the creation of the inbound integration in AlertOps. In the Events and Alerts section select “Alert Channels”. Click “Add Alert Channel” and select “Generic Webhook”.
Give a name to the channel and paste the API URL in the Webhook URLs field. You can also provide custom headers if required. You can test the URL. Click Save.
All that is left now, is to create an alert for the event and channel you just created. In the left navigation pane, select “Alerts”. Click “New Alert”. The configuration is pretty self-explanatory. In the first part, you give a name.
In the second part, you attach the alert to the event you created before (or any event for that matter).
In the third part, you define the application scope and in the fourth part, you add the alert channel, which you created in the previous step. If everything went correctly, you should have an alert defined as shown below.
That’s it! You have created an event/alert and an integration with AlertOps. Any event of the specific type should send an alert to AlertOps for incident management. You can view the messages in the “Inbound Log”/"Alerts" in AlertOps.
Alert Triggering Information:
AlertOps will automatically create an incident when a new alert is received from Instana when the issue^state field contains “OPEN”.
If an alert with status “OPEN” matches an existing Open Alert, AlertOps will recognize the new alert as a duplicate and ignore the alert. The alert will be recorded in the Inbound Messages table as “Mapped Appended.”
AlertOps will automatically close the same incident when an alert with issue^state contains “CLOSED”.