• Products
  • Get started
  • Documentation
  • Resources

Integrate Opsgenie with Zabbix Plugin

What does Opsgenie offer Zabbix users?

Integration leverages Opsgenie’s Zabbix-specific executable and OEC utility to automatically create rich alerts and synchronize alert status between Zabbix and Opsgenie. Zabbix alarms are acknowledged automatically when the alert is acknowledged in Opsgenie. Opsgenie Zabbix plugin is the recommended integration method as it enables Zabbix to take advantage of advanced capabilities provided by Opsgenie.

Functionality of the integration

  • When users acknowledge an alert from their mobile devices using the Opsgenie app, the alert is acknowledged in Zabbix.

Installation

The steps below describe how to integrate Opsgenie and Zabbix using the Opsgenie Zabbix integration plugin. Note that these instructions may need slight alteration depending on the exact Linux distribution and the Zabbix configuration.

Prerequisites

Packages provided support the following systems:

  • Red Hat based linux distributions

  • Debian based linux distributions

Installation of Opsgenie Zabbix Plugin

Download the latest version of OEC

To download the latest version of Zabbix Plugin package, please use readme file of oec-scripts repository.

For Red Hat Based Distributions

  • Run following command : rpm -i opsgenie-zabbix-<your_version>.rpm

During upgrades, the rpm package does not overwrite existing configurations. It saves the new default configuration file as opsgenie-integration.conf.rpmnew. Find more information about rpm upgrade config file handling from here.

If you want to update from version 201X-XX-XX to 2.X.X, you must add --force parameter.

E.g.: 

rpm -U --force opsgenie-integration-<your_version>.rpm

We suggest that you backup your configuration files before update!

For Debian Based Distributions

  • Run following command : dpkg -i opsgenie-zabbix-<your_version>.deb

Add Zabbix integration in Opsgenie

If you're using Opsgenie's Free or Essentials plan or if you’re using Opsgenie with Jira Service Management's Standard plan, you can add this integration from your team dashboard only. The Integrations page under Settings is not available in your plan.

  1. Go to Teams and select your team.

  2. Select Integrations on the left navigation and then select Add integration.

Adding the integration from your team dashboard will make your team the owner of the integration. This means Opsgenie will assign the alerts received through this integration to your team only. Follow the rest of the steps in this section to set up the integration.

  1. Go to Settings > Integrations. Search for Zabbix and select Add

  2. Select Save Integration to save the integration. This will generate an API key for the integration. This key will be used by Zabbix to authenticate Opsgenie and specify the integration that should be used to process Zabbix alerts.

Opsgenie Plugin Configuration in Zabbix (for Zabbix version 3.0 and below)

The plugin uses a golang-executable file (included in the plugin as send2opsgenie) to create, acknowledge, and close alerts in Opsgenie. Configure Zabbix to execute this file on events to create, acknowledge, and close alerts in Opsgenie.

Configure an action via the Zabbix frontend, that executes /home/opsgenie/oec/opsgenie-zabbix/send2opsgenie when an event is triggered.

Under Actions enable "Recovery Message" checkbox for closing the alert on Opsgenie when an alert is closed on Zabbix.

Actions

Under Operations, define a Remote Command that executes on Zabbix server. Paste the command in */home/opsgenie/oec/opsgenie-zabbix/actionCommand.txt into Command.

Click Add then Save.

Add operation

In order to make send2opsgenie work you need to make some configurations. The table below shows the parameters, where they are located and whether they are mandatory to fill or not.

Configuration Parameters

Description

Mandatory to fill

Location

apiKey

Copy the API key from the Zabbix integration created above. send2opsgenie uses this key to authenticate to Opsgenie. API key is also used to identify the right integration configuration used to process alerts.

Yes

/home/opsgenie/oec/conf/config.json

baseUrl

If you're using Opsgenie from another domain(eg. EU, sandbox), you should update this configuration.Opsgenie Zabbix integration, Advanced Settings page.

Optional

/home/opsgenie/oec/conf/config.json

responders

Responders field is used to specify which responders should be notified for the Zabbix alerts. This field is used to set the default responder field value. It can be modified to route different alerts to different teams in Opsgenie Zabbix integration, Advanced Settings page.

Optional

/home/opsgenie/oec/conf/opsgenie-integration.conf

tags

Tags field is used to specify the tags of the alert that created in Opsgenie.

Optional

/home/opsgenie/oec/conf/opsgenie-integration.conf

logPath

Specifies the full path of the log file. (Default value is /var/log/opsgenie/send2opsgenie.log)

Optional

/home/opsgenie/oec/conf/opsgenie-integration.conf

zabbix2opsgenie.http.proxy.enabled

zabbix2opsgenie.http.proxy.enabled field is to enable/disable external proxy configuration. The default value is false.

Optional

/home/opsgenie/oec/conf/opsgenie-integration.conf

zabbix2opsgenie.http.proxy.host

It is the host of the proxy.

Optional

/home/opsgenie/oec/conf/opsgenie-integration.conf

zabbix2opsgenie.http.proxy.port

It is the port of the proxy.

Optional

/home/opsgenie/oec/conf/opsgenie-integration.conf

zabbix2opsgenie.http.proxy.scheme

It is the proxy connection protocol. It may be http or https depending on your proxy servers. Its default value is http.

Optional

/home/opsgenie/oec/conf/opsgenie-integration.conf

zabbix2opsgenie.http.proxy.username

It is the Proxy authentication username.

Optional

/home/opsgenie/oec/conf/opsgenie-integration.conf

zabbix2opsgenie.http.proxy.password

It is the Proxy authentication password.

Optional

/home/opsgenie/oec/conf/opsgenie-integration.conf

Please note: Special characters (except triggerUrl, ipAddress, date, and time fields) will be removed from Zabbix incoming payload before data is processed in Opsgenie.

Opsgenie Plugin Configuration in Zabbix (for Zabbix version 3.2 and above)

The plugin uses a golang-executable file (included in the plugin as send2opsgenie) to create, acknowledge, and close alerts in Opsgenie. Configure Zabbix to execute this file on events to create, acknowledge, and close alerts in Opsgenie.

Configure an action via the Zabbix frontend, that executes */home/opsgenie/oec/opsgenie-zabbix/send2opsgenie when an event is triggered.

Select Configurations -> Actions then select Create New Action.

Select Operations under Actions.

Select Operations under Actions

Under Operations, define a Remote Command that executes on Zabbix server. Paste the command in */home/opsgenie/oec/opsgenie-zabbix/actionCommand.txtinto Command.

Operations

Select Recovery Operations under Actions.

Under Recovery Operations, define a Remote Command that executes on Zabbix server. Paste the command in */home/opsgenie/oec/opsgenie-zabbix/actionCommand.txt into Command.

Recovery operations

Click Add and Save.

In order to make send2opsgenie work you need to make some configurations. The table below shows the parameters, where they are located and whether they are mandatory to fill or not.

Configuration Parameters

Description

Mandatory to fill

Location

apiKey

Copy the API key from the Zabbix integration you've created above. send2opsgenie uses this key to authenticate to Opsgenie. API key is also used to identify the right integration configuration that should be used to process alerts.

Yes

/home/opsgenie/oec/conf/config.json

baseUrl

If you're using Opsgenie from another domain(eg. EU, sandbox), you should update this configuration.Opsgenie Zabbix integration, Advanced Settings page.

Optional

/home/opsgenie/oec/conf/config.json

responders

Responders field is used to specify who should be notified for the Zabbix alerts. This field is used to set the default responders field value. The recipients can be modified to route different alerts to different people and teams in Opsgenie Zabbix integration, Advanced Settings page. Recipients can be set to users, groups, escalations or schedules who will be notified by Opsgenie. If you did not set responders in the integration, this field is required.schedules who will be notified by Opsgenie. If you did not set recipients in the integration, this field is required.

Optional

/home/opsgenie/oec/conf/opsgenie-integration.conf

tags

Tags field is used to specify the tags of the alert that created in Opsgenie.

Optional

/home/opsgenie/oec/conf/opsgenie-integration.conf

logPath

Specifies the full path of the log file. (Default value is /var/log/opsgenie/send2opsgenie.log)

Optional

/home/opsgenie/oec/conf/opsgenie-integration.conf

zabbix2opsgenie.http.proxy.enabled

zabbix2opsgenie.http.proxy.enabled field is to enable/disable external proxy configuration. The default value is false.

Optional

/home/opsgenie/oec/conf/opsgenie-integration.conf

zabbix2opsgenie.http.proxy.host

It is the host of the proxy.

Optional

/home/opsgenie/oec/conf/opsgenie-integration.conf

zabbix2opsgenie.http.proxy.port

It is the port of the proxy.

Optional

/home/opsgenie/oec/conf/opsgenie-integration.conf

zabbix2opsgenie.http.proxy.scheme

It is the proxy connection protocol. It may be http or https depending on your proxy servers. Its default value is http.

Optional

/home/opsgenie/oec/conf/opsgenie-integration.conf

zabbix2opsgenie.http.proxy.username

It is the Proxy authentication username.

Optional

/home/opsgenie/oec/conf/opsgenie-integration.conf

zabbix2opsgenie.http.proxy.password

It is the Proxy authentication password.

Optional

/home/opsgenie/oec/conf/opsgenie-integration.conf

Please note: Special characters (except triggerUrl, ipAddress, date, and time fields) will be removed from Zabbix incoming payload before data is processed in Opsgenie.

*If you want to send additional custom arguments, you can add them after the flags as: customArgName1 customArgValue1 customArgName2 customArgValue2

You can parse custom arguments by adding {{_payload.customArgName}} to wherever you need on the input fields.*

For more information about using raw parameters please visit the Dynamic Fields document.

Configure Opsgenie to Zabbix Integration (Optional)

If you are using Zabbix version 4.0 or above, you need to use actionExecutorForZabbix4.py script. You also need to update the conf file so that the actions will trigger this script.

Use Opsgenie’s OEC and the Zabbix script to update alerts on Zabbix. This enables deployment of your own scripts/ability to modify the ones provided and execute customized actions on Zabbix.

To be able to execute actions in Zabbix, OEC gets the configuration parameters from the configuration file. The configuration file can be found under /home/opsgenie/oec/conf/config.json

Configuration Parameters

 

Location

command_url

The Zabbix API endpoint to acknowledge Zabbix alerts.

/home/opsgenie/oec/conf/config.json

user

The username that OEC used to authenticate to Zabbix.

/home/opsgenie/oec/conf/config.json

password

The password that OEC uses to authenticate to Zabbix.

/home/opsgenie/oec/conf/config.json

The package that you downloaded also includes OEC utility which is located under /usr/local/bin and the script that is needed to be run by OEC which is under /home/opsgenie/oec/scripts. After the configuration of OEC is done, you need to run it. In order to learn more about how to run OEC, you can refer to the Running OEC documentation.

Zabbix integration package does not support SSL v1.0. If your Zabbix Server has SSL v1.0, we suggest you to upgrade your SSL server.

FAQ and Troubleshooting

If errors occur while getting the integration to work, please check if the problem is mentioned below, and follow our advice:

1. Zabbix alerts are not created in Opsgenie:

Run the following test command from the shell. Check if the test alert is created in Opsgenie:

Text

1 /home/opsgenie/oec/opsgenie-zabbix/send2opsgenie -triggerName='zabbix test alert' -triggerStatus='PROBLEM'
  • If a "Trace/breakpoint trap" error occurs: It means the send2opsgenie plugin isn't compatible with your server distribution. Follow the "Source and Recompiling send2opsgenie" section below and rebuild the send2opsgenie.go according to a specific server environment.

  • If the alert is created in Opsgenie: It means the integration is installed correctly. The problem might be that Zabbix is not executing the command.

  • If not: Check the logs at /var/log/opsgenie/send2opsgenie.log. Look for the following errors in the log file:

    • If "RestException[Could not authenticate.]" error occurs in the logs, it means Opsgenie couldn't identify the api key. Check if the API key is set correctly, as explained in "Opsgenie Plugin Configuration in Zabbix" above.

    • If unsure of the issue, set the plugin's log level to debug, try again and send the logs to us with out contact us form.

  • If there is no /var/log/opsgenie/send2opsgenie.log file, or there are no logs in it, check the following:

    • First, make sure the Zabbix user has permission to write to /var/log/opsgenie directory. The installation package should automatically do this. If problems occur, execute: chown -R zabbix:opsgenie /var/log/opsgenie.

    • Now check the Zabbix server logs at /var/log/zabbix/zabbix_server.log. See if there are error logs regarding send2opsgenie, and contact us with them.

Setting send2opsgenie plugin's log level to DEBUG:

Change the line zabbix2opsgenie.logger=warning to zabbix2opsgenie.logger=debug in /home/opsgenie/oec/conf/opsgenie-integration.conf file.

2. The Zabbix alarm is not acknowledged when you ack the alert at Opsgenie:

  1. Check the Alerts logs.

  • If the "Posted [Acknowledge] action to Zabbix.." log does not appear, it means Opsgenie didn't send the Acknowledge action to Zabbix. Check the integration configuration, it might not have matched the alert action.

  • If "Posted [Acknowledge] action to Zabbix.." log only occurs, and no related log after that, it might mean OEC is having connection problems. Check the logs.

3. Could not open Zabbix RPM package

If while installing Zabbix's rpm package you get the following error:

Text

1 2 3 zabbix is obsoleted by (installed) zabbix-server-pgsql-4.0.13-1.el7.x86_64 zabbix is obsoleted by (installed) zabbix-agent-4.0.13-1.el7.x86_64 zabbix is obsoleted by (installed) zabbix-java-gateway-4.0.13-1.el7.x86_64

You need to use "rpm -i opsgenie-zabbix-1.0.4-rpm-x86-64.rpm --nodeps" instead.

If you get "is already installed" error, use "rpm -i opsgenie-zabbix-1.0.4-rpm-x86-64.rpm --force" instead.

Source and Recompiling send2opsgenie

The source for the executable send2opsgenie, send2opsgenie.go, is located under /home/opsgenie/oec/opsgenie-zabbix and is also available at GitHub Opsgenie Integration repository. To change the behavior of the executable, edit send2opsgenie.go and build it using:

go build send2opsgenie.go

For installing go, refer to http://golang.org/doc/install. Note that the executable in the plugin is built for linux/386 systems.

Additional Help