Integrate Opsgenie with Nagios Plugin

Functionality of the integration

When a host or service state becomes down in Nagios, an alert is created in Opsgenie.
Upon creation of the new alert, related histogram and trends images from Nagios is attached to the alert automatically.
When the Opsgenie alert is acknowledged, the alert in Nagios is also acknowledged automatically, and vice versa.
When a note is added to the Opsgenie alert, the alert in Nagios is also updated automatically, and vice versa.

Access the previous version of Nagios Integration documents from Lamp based Nagios Integration page.

Installation

The steps below describe how to integrate Opsgenie and Nagios using Opsgenie Nagios integration plugin. Slightly alter these instructions depending on the exact Linux distribution and Nagios configuration.

If Lamp based Nagios plugin is being used, backup all existing configurations. Uninstall the old plugin, then install the new one.

Prerequisites

Packages provided support the following systems:

Red Hat based Linux distributions, REHL7 and above
Debian based Linux distributions, Debian 8(Jessie) and above

Installation of Opsgenie Nagios Plugin

Download the latest version of OEC

To download the latest version of Nagios packages, please use readme file of oec-scripts repository.

For Red Hat Based Distributions

Run the following command:

1rpm -i opsgenie-nagios-<your_version>.rpm

During upgrades, rpm package does not overwrite your existing configurations. It saves the new default configuration file as opsgenie-integration.conf.rpmnew. Find more information about rpm upgrade config file handling from here.

To update from version 201X-XX-XX to 2.X.X, you must add --force parameter. E.g.:

rpm -U --force opsgenie-integration-<your_version>.rpm

We suggest backing up your configuration files before update!

For Debian Based Distributions

Run the following command:
dpkg -i opsgenie-nagios-<your_version>.deb

Add Nagios integration in Opsgenie

You can add this integration from your team dashboard

If you're using Opsgenie's Free or Essentials plan, or if you’re using Opsgenie with Jira Service Management's Standard plan, you can only add this integration from your team dashboard as the Integrations page under Settings is not available in your plan.

Adding the integration from your team dashboard will make your team the owner of the integration. This means Opsgenie will assign the alerts received through this integration to your team only.

To do that,

Go to your team’s dashboard from Teams,
Select Integrations, and select Add integration.

Follow the rest of the steps to complete the integration.

Go to Settings > Integrations. Search for Nagios and select Add.
Specify who is notified of Nagios alerts using the Responders field. Autocomplete suggestions are provided as you type.
Copy the API key.
Select Save Integration.

Opsgenie Plugin Configuration in Nagios

The plugin uses a golang-executable file (included in the plugin as send2opsgenie) to create, acknowledge, and close alerts in Opsgenie. Nagios should be configured to execute this file on events to create, acknowledge, and close alerts in Opsgenie.

Setting the apiKey is required. Other configuration parameters are set to defaults that work with most Nagios implementations but may need to be modified as well.

Configuration Parameters	Description	Mandatory to fill	Location
apiKey	Copy the API key from the Nagios integration you've created above. send2opsgenie uses this key to authenticate to Opsgenie. API key is also used to identify the right integration configuration that should be used to process alerts.	Yes	/home/opsgenie/oec/conf/config.json
baseUrl	If you're using Opsgenie from another domain(eg. EU, sandbox), you should update this configuration.Opsgenie Nagios integration, Advanced Settings page.	Optional	/home/opsgenie/oec/conf/config.json
teams	Teams field is used to specify which teams should be notified for the Nagios alerts. This field is used to set the default teams field value. It can be modified to route different alerts to different teams in Opsgenie Nagios integration, Advanced Settings page.	Optional	/home/opsgenie/oec/conf/opsgenie-integration.conf
tags	Tags field is used to specify the tags of the alert that created in Opsgenie.	Optional	/home/opsgenie/oec/conf/opsgenie-integration.conf
nagios_server	nagios_server field is used to identify the Nagios server in Opsgenie, and only required when there are multiple Nagios servers. This field is used by Opsgenie when sending actions executed by users (acknowledge, close, etc.) back to your Nagios servers via OEC	Optional	/home/opsgenie/oec/conf/opsgenie-integration.conf
logPath	Specifies the full path of the log file. (Default value is /var/log/opsgenie/send2opsgenie.log)	Optional	/home/opsgenie/oec/conf/opsgenie-integration.conf
nagios2opsgenie.http.proxy.enabled	nagios2opsgenie.http.proxy.enabled field is to enable/disable external proxy configuration. The default value is false.	Optional	/home/opsgenie/oec/conf/opsgenie-integration.conf
nagios2opsgenie.http.proxy.host	It is the host of the proxy.	Optional	/home/opsgenie/oec/conf/opsgenie-integration.conf
nagios2opsgenie.http.proxy.port	It is the port of the proxy.	Optional	/home/opsgenie/oec/conf/opsgenie-integration.conf
nagios2opsgenie.http.proxy.scheme	It is the proxy connection protocol. It may be http or https depending on your proxy servers. Its default value is http.	Optional	/home/opsgenie/oec/conf/opsgenie-integration.conf
nagios2opsgenie.http.proxy.username	It is the Proxy authentication username.	Optional	/home/opsgenie/oec/conf/opsgenie-integration.conf
nagios2opsgenie.http.proxy.password	It is the Proxy authentication password.	Optional	/home/opsgenie/oec/conf/opsgenie-integration.conf

There are three ways to configure golang-executable file:

1.Configuring from conf file: Configure from /home/opsgenie/oec/conf/opsgenie-integration.conf file. Configuring from conf file overwrites the configurations made in the script.

2.Configuring by using Golang Flags: Configure by entering flags to command in the opsgenie.cfg file. Use -apiKey flag for apiKey and -ns flag for the nagios_server name. If you don't have multiple Nagios servers, there's no need to define the Nagios server. Using flags overwrites all the other configuration methods mentioned above.

Configure the apiKey from the cfg file as follows:

JSON

define command {
    command_name    notify-service-by-opsgenie
    command_line    /home/opsgenie/oec/opsgenie-nagios/send2opsgenie -apiKey="apiKey1" -entityType=service ...
}

When apiKey is added to the cfg file, it overrides the apiKey in the opsgenie-integration.conf file.

If you want to send additional custom arguments, you can add them after the flags as: customArgName1 customArgValue1 customArgName2 customArgValue2

You can parse custom arguments by adding {{_payload.customArgName}} to wherever you need on the input fields.

For more information about using raw parameters please visit Dynamic Fields document.

3. Configuring from script: You can configure apiKey and nagios_server from send2opsgenie.go script. If you use this option, you need to build the script again and put the new executable to/home/opsgenie/oec/nagios directory. You can find information about the location of the send2opsgenie.go and how to build a go script in the "Source" section.

Define Nagios contacts

Copy /home/opsgenie/oec/opsgenie-nagios/opsgenie.cfg config file, (configures a contact and its host and service notification commands) to /usr/local/nagios/etc/objects directory.
cp /home/opsgenie/oec/opsgenie-nagios/opsgenie.cfg /usr/local/nagios/etc/objects
Add following line to main Nagios configuration file (NAGIOS_HOME/etc/nagios.cfg)

Text

...
cfg_file=/usr/local/nagios/etc/objects/opsgenie.cfg
...

3. Add the contact "opsgenie" to your Nagios configuration’s main contact group in NAGIOS_HOME/etc/objects/contacts.cfg file. If you’re using the default configuration, contacts.cfg, add "opsgenie" user to the "admins" contact group.

4. Restart Nagios.

If everything goes well, alerts are visible in Opsgenie for every notification created in Nagios.

Configure Opsgenie to Nagios Integration (Optional)

In order to use this feature "Send Alert Actions To Nagios" checkbox should be enabled in Opsgenie Nagios Integration.

Use Opsgenie’s OEC and the Nagios script to update alerts on Nagios. This enables deployment of your own scripts/ability to modify the ones provided and execute customized actions on Nagios.

To be able to execute actions in Nagios, OEC gets the configuration parameters from the configuration file. The configuration file can be found under /home/opsgenie/oec/conf/config.json.

Configuration parameters

alert_histogram_image_url – OEC retrieves histogram images from Nagios using this URL. Localhost should be replaced with your Nagios server address.
trends_image_url – OEC retrieves trends images from Nagios using this URL. Localhost should be replaced with your Nagios server address.
command_url – URL to update Nagios alerts when alerts get acknowledged, commented, etc.
user password – Credentials to authenticate Nagios web server to get Nagios histogram and trends images.

The package that you downloaded also includes OEC utility which is located under /usr/local/bin and the script that is needed to be run by OEC which is under /home/opsgenie/oec/scripts. After the configuration of OEC is done, you need to run it. In order to learn more about how to run OEC, you can refer to the Running OEC documentation.

If using OEC, rich alerts are populated with host or service current status information in Opsgenie for every notification created in Nagios.

Nagios integration package does not support SSL v1.0. If your Nagios Server has SSL v1.0, we suggest you to upgrade your SSL server.

FAQ and Troubleshooting

If you're having trouble getting the integration to work, please check if your problem is mentioned below, and follow our advice:

1- Nagios alerts are not getting created in Opsgenie:

Run the following test command from the shell. Check if the test alert is created in Opsgenie:

/home/opsgenie/oec/opsgenie-nagios/send2opsgenie -entityType=host -t=PROBLEM -hs=DOWN -hn=test_host

"Trace/breakpoint trap" error: It means the send2opsgenie plugin isn't compatible with the server distribution. Follow the "Source and Recompiling send2opsgenie" section below and rebuild your send2opsgenie.go according to your specific server environment.

If the alert is created in Opsgenie: It means the integration is installed correctly. The problem might be that Nagios is not notifying the Opsgenie contact for alerts. Check your Nagios alert notifications log.

If not: Check the logs at /var/log/opsgenie/send2opsgenie.log. Look for the following errors in the log file:
"RestException[Could not authenticate.]": if visible in the logs, it means Opsgenie couldn't identify the API key. Check that the API key correctly, as explained in "Opsgenie Plugin Configuration in Nagios" above.
"Could not execute this action with apiKey of [Nagios] integration": if visible in the logs, the wrong integration package may have been downloaded. Make sure to download the Nagios integration package, not Nagios or any other.

If some other problem, set the plugin's log level to debug, try again and send the logs to us with out contact us form.

If there is no /var/log/opsgenie/send2opsgenie.log file, or there are no logs in it, check the following:

First, make sure the Nagios user has permission to write to /var/log/opsgenie directory. The installation package should automatically do this for you. If you encounter problems, execute:
chown -R nagios:opsgenie /var/log/opsgenie
Now check your Nagios server logs at /usr/local/nagios/var/nagios.log. See if there are error logs regarding send2opsgenie, and contact us with them.

Setting send2opsgenie plugin's log level to DEBUG:

Change the line nagios2opsgenie.logger=warning to nagios2opsgenie.logger=debug in */home/opsgenie/oec/conf/opsgenie-integration.conf** file.

2- The Nagios alert is not acknowledged when you ack the alert at Opsgenie:

First, check your alert logs.

If you don't see the "Posted [Acknowledge] action to Nagios.." log, it means Opsgenie didn't send the Acknowledge action to Nagios. Check your integration configuration, it might not have matched the alert action.
If you only see the "Posted [Acknowledge] action to Nagios.." log and no related log after that, it might mean OEC is having connection problems. Check the logs.

3. Could not open Nagios RPM package

If while installing the rpm package you get an error saying that the package is obsolated, you need to use "rpm -i opsgenie-nagios-1.0.4-rpm-x86-64.rpm --nodeps" instead.

If you get "is already installed" error, use "rpm -i opsgenie-nagios-1.0.4-rpm-x86-64.rpm --force" instead.

4. Permission error occurs while attaching zip file to alert in script

If you are getting "PermissionError: [Errno 13] Permission denied: details_*" error while running the script, prepend the path of the script to the "file_name" variable in the attach method of the script.

Source and Recompiling send2opsgenie

The source for the executable send2opsgenie is located under /home/opsgenie/oec/opsgenie-nagios and send2opsgenie.go is located under //home/opsgenie/oec/opsgenie-nagios and is also available at GitHub Opsgenie Integration repository. To change the behavior of the executable, edit send2opsgenie.go and build it using:

go build send2opsgenie.go

Refer to http://golang.org/doc/install/source. Note that send2opsgenie is built for linux/amd64 systems.

Was this helpful?

It wasn't accurate

It wasn't clear

It wasn't relevant

Opsgenie Support