• Products
  • Get started
  • Documentation
  • Resources

Integrate Opsgenie with Zenoss

Zenoss integration diagram

What does Opsgenie offer Zenoss users?

Opsgenie Zenoss integration plugin utilizes full capabilities of Opsgenie and provides a bi-directional integration with Zenoss. Integration leverages Opsgenie’s Zenoss-specific executable and OEC utility to automatically create alerts and synchronizes alert status between Zenoss and Opsgenie.

Installation

The steps below describe how to integrate Opsgenie and Zenoss using Opsgenie Zenoss integration plugin. Note that slight alteration to these instructions may be necessary depending on the exact Linux distribution and your Zenoss configuration.

Prerequisites

Packages provided support the following systems:

  • Red Hat based linux distributions

  • Debian based linux distributions

Installation of Opsgenie Zenoss Plugin

Download the latest version of OEC

To download the latest version of Zenoss package, please use readme file of oec-scripts repository.

For Red Hat Based Distributions

  • Run the following command: rpm -i opsgenie-zenoss-<your_version>.rpm

During upgrades, rpm package does not overwrite existing configurations. It saves the new default configuration file as opsgenie-integration.conf.rpmnew. Find more information about rpm upgrade config file handling from here.

To update from version 201X-XX-XX to 2.X.X, add --force parameter.

E.g.: rpm -U --force opsgenie-integration-<your_version>.rpm

We suggest backing up configuration files before updating!

For Debian Based Distributions

  • Run the following command: dpkg -i opsgenie-zenoss-<your_version>.deb

Add Zenoss integration in Opsgenie

  1. Go to Opsgenie Zenoss Integration page.

Under the Free and Essentials plans, the tabs under the Team dashboard are limited to Main, Integrations, Members, and Activity Stream.

2. Specify who is notified of Sentry alerts using the Responders field. Auto-complete suggestions are provided as you type.

Did you know that you can also add this integration from your team dashboard? 

If you want your team to be the owner of this integration, add it from your team dashboard. Opsgenie will assign the alerts received through this integration to your team only.

To do that;

  1. Go to Teams,

  2. Find your team and select it to go to your team dashboard,

  3. From your team navigation, select Integrations,

  4. Select Add integration.

3. Copy the API key.

4. Click Save Integration.

Zenoss integration settings

Opsgenie Plugin Configuration in Zenoss

The plugin uses a golang-executable file (included in the plugin as send2opsgenie) to create, acknowledge, and close alerts in Opsgenie. In order to make send2opsgenie work you need to make some configurations. The table below shows the parameters, where they are located and whether they are mandatory to fill or not.

Configuration Parameters

Description

Mandatory to fill

Location

apiKey

Copy the API key from the Zenoss integration you've created above. send2opsgenie uses this key to authenticate to Opsgenie. API key is also used to identify the right integration configuration that should be used to process alerts.

Yes

 

baseUrl

If you're using Opsgenie from another domain(eg. EU, sandbox), you should update this configuration. Opsgenie Zenoss integration, Advanced Zenoss page.

 

 

zenoss.command_url

URL to get detailed event data from Zenoss in send2opsgenie.

Optional

 

zenoss.user

Credentials to authenticate Zenoss web server

Optional

 

zenoss.password

Credentials to authenticate Zenoss web server

Optional

 

responders

Responders field is used to specify which responders should be notified for the Zenoss alerts. This field is used to set the default responder field value. It can be modified to route different alerts to different teams in Opsgenie Zenoss integration, Advanced Settings page.schedules who will be notified by Opsgenie. If you did not set recipients in the integration, this field is required.

Optional

 

tags

Tags field is used to specify the tags of the alert that created in Opsgenie.

Optional

 

logPath

Specifies the full path of the log file. (Default value is /var/log/opsgenie/send2opsgenie.log)

Optional

 

zenoss2opsgenie.http.proxy.enabled

zenoss2opsgenie.http.proxy.enabled field is to enable/disable external proxy configuration. The default value is false.

Optional

 

zenoss2opsgenie.http.proxy.host

It is the host of the proxy.

Optional

 

zenoss2opsgenie.http.proxy.port

It is the port of the proxy.

Optional

 

zenoss2opsgenie.http.proxy.scheme

It is the proxy connection protocol. It may be http or https depending on your proxy servers. Its default value is http.

Optional

 

zenoss2opsgenie.http.proxy.username

It is the Proxy authentication username.

Optional

 

zenoss2opsgenie.http.proxy.password

It is the Proxy authentication password.

Optional

 

There are three ways to configure golang-executable file:

  1. "Configuring from conf file": Configure from /home/opsgenie/oec/conf/opsgenie-integration.conf file. Configuring from conf file overwrites the configurations made in the script.

  2. "Configuring by using Golang Flags": Configure by entering flags to command of the notification created in Zenoss, which is described in "Configure Triggers in Zenoss" section. Use -apiKey flag for your apiKey.

To send additional custom arguments, add them after the flags as: customArgName1 customArgValue1 customArgName2 customArgValue2
Parse custom arguments by adding {{_payload.customArgName}} to wherever is needed on the input fields.
For more information about using raw parameters please visit Dynamic Fields document.

3. "Configuring from script": Configure apiKey send2opsgenie.go script. If choosing this option, build the script again and put the new executable to /home/opsgenie/oec/zenoss directory. Find information about the location of the zenoss2opsgenie.go and how to build a go script in the "Source" section.

Configure Triggers in Zenoss

Before creating a notification:

  1. Select Events > Triggers from the Navigation menu

  2. Create a trigger named Opsgenie.

After creating the trigger, follow the steps below:

  1. Select Events > Triggers from the Navigation menu.

  2. Select Notifications from the left panel.

  3. Create a notification.

Add notification

4. Choose the notification created previously and click Edit.

5. Under the "Notification" tab, enable the notification, set "Send Clear" as checked and add the trigger named "opsgenie" from the trigger list and click Add.

Edit notification

6. Under the "Content" tab, put the following into "Command" and "Clear Command" fields. Add optional -eventState=close to Clear Command, send2opsgenie executable does not try to get event details from Zenoss and directly closes the event's alert in Opsgenie.

/usr/bin/zenoss2opsgenie -evid=${evt/evid}

Edit notification content

7. Under the "Subscribers" tab, choose the subscribers and click SUBMIT.

Configure Opsgenie to Zenoss Integration (Optional)

Use Opsgenie’s OEC and the Zenoss script to update alerts on Zenoss. This enables deployment of your own scripts/ability to modify the ones provided and execute customized actions on Zenoss.

To be able to execute actions in Zenoss, OEC gets the configuration parameters from the configuration file. The configuration file can be found under /home/opsgenie/oec/conf/config.json

Configuration Parameters

 

Location

command_url

URL to update Zenoss events when alerts get acknowledged, closed, etc.

/home/opsgenie/oec/conf/config.json

user

Credentials to authenticate on Zenoss web server.

/home/opsgenie/oec/conf/config.json

password

Credentials to authenticate on Zenoss web server.

/home/opsgenie/oec/conf/config.json

The package that you downloaded also includes OEC utility which is located under /usr/local/bin and the script that is needed to be run by OEC which is under /home/opsgenie/oec/scripts. After the configuration of OEC is done, you need to run it. In order to learn more about how to run OEC, you can refer to the Running OEC documentation.

Zenoss integration package does not support SSL v1.0. If your Zenoss Server has SSL v1.0, we suggest upgrading the SSL server.

FAQ and Troubleshooting

If having trouble getting the integration to work, please check if the problem is mentioned below, and follow our advice:

1. Zenoss alerts are not getting created in Opsgenie:

Run the following test command from the shell. Check if the test alert is created in Opsgenie

/home/opsgenie/oec/opsgenie-zenoss/send2opsgenie -test

  • If you're getting a "Trace/breakpoint trap" error occurs It means the send2opsgenie plugin isn't compatible with the server distribution. Follow the "Source and Recompiling send2opsgenie" section below and rebuild the send2opsgenie.go according to the specific server environment.

  • If the alert is created in Opsgenie: It means the integration is installed correctly. The problem might be that Zenoss is not notifying the Opsgenie contact for alerts. Check your Zenoss alert notifications log.

  • If not: Check the logs at /var/log/opsgenie/send2opsgenie.log. Look for the following errors in the log file:

    • If "RestException[Could not authenticate.]" occurs in the logs, it means Opsgenie couldn't identify the API key. Check if the API key is set correctly, as explained in "Opsgenie Plugin Configuration in Zenoss" above.

    • If unsure of the problem, set the plugin's log level to debug, try again and send the logs to us with our contact us form.

  • If there is no /var/log/opsgenie/send2opsgenie.log file, or there are no logs in it, check the following:

    1. First, make sure the Zenoss user has permission to write to /var/log/opsgenie directory. The installation package should automatically do this for you. If you encounter problems, execute: chown -R zenoss:opsgenie /var/log/opsgenie

    2. Now check the Zenoss server logs at /opt/zenoss/log/zeneventd.log. See if there are error logs regarding zenoss2opsgenie, and contact us with them.

Setting zenoss2opsgenie plugin's log level to DEBUG:
Change the line zenoss2opsgenie.logger=warning to zenoss2opsgenie.logger=debug in /etc/opsgenie/conf/opsgenie-integration.conf file.

2. The Zenoss alert is not acknowledged when the alert is acked in Opsgenie:

  1. First, check the alert logs.

  • If "Posted [Acknowledge] action to Zenoss.." is not in the log, it means Opsgenie didn't send the Acknowledge action to Zenoss. Check the integration configuration, it might not have matched the alert action.

  • If only the "Posted [Acknowledge] action to Zenoss.." log occurs and no related log after that, it might mean OEC is having connection problems. Check the logs,

3. Could not open Zenoss RPM package

If while installing the rpm package you get an error saying that the package is obsolated, you need to use "rpm -i opsgenie-zenoss-1.0.4-rpm-x86-64.rpm --nodeps" instead.

If you get "is already installed" error, use "rpm -i opsgenie-zenoss-1.0.4-rpm-x86-64.rpm --force" instead.

Source and Recompiling send2opsgenie

The source for the executable send2opsgenie is located under /usr/bin/ and send2opsgenie.go is located under /etc/opsgenie/ and is also available at GitHub Opsgenie Integration repository. To change the behavior of the executable, edit send2opsgenie.go and build it using: 

go build send2opsgenie.go

For installing go, refer to http://golang.org/doc/install. Note that the executable in the plugin is built for linux/386 systems.

Last modified on Jun 18, 2021
Cached at 3:46 PM on Dec 5, 2021 |

Additional Help