Using Metadata in Notifications

The OpenNMS Metadata feature makes it possible to add key:value entries to your nodes, interfaces, or services. A few months ago, we described How to Monitor Websites Using Metadata with OpenNMS’ Metadata DSL (domain specific language).

Now (Horizon >=27), thanks to a recent DevJam project, you can use metadata in notifications (notifd) to improve your workflow, ability to troubleshoot problems, and deduplicate notification definitions dramatically. :tada:

The possibilities are endless. Here we discuss two:

  • linking to documentation related to an alarm
  • providing context about the nodes associated with an alarm

Use Case 1: It’s all about the docs

Having an alarm notification is good. At least you have noticed that something is going wrong. But notifications are not very useful if they don’t help you solve the problem.

Of course, we already have our documentation, troubleshooting guides, or emergency plans written down — all that’s left is to add them into the notifications.

In this simple example of a node definition with metadata, multiple keys include a URL to documentation about the node or references that might be helpful:

   <node foreign-id="web01-srv" node-label="web01-srv">
      <interface ip-addr="192.168.1.10" status="1" snmp-primary="N">
      <meta-data context="requisition" key="documentationurl" value="https://url_to_documentation"/>
      <meta-data context="requisition" key="troubleshootguideurl" value="https://url_to_troubleshootguide"/>
      <meta-data context="requisition" key="emergencyplanurl" value="https://url_to_emergencyplanurl"/>
   </node>

Since we are now able to use metadata in notifd, you basically only need this variable in your notification text:

${requisition:documentationurl}

Any time a notification for this node gets triggered, the URL to your document will also appear in the notification. This can be very helpful in the field, because the receiver always gets access directly out of the event notification tool (Outlook, Slack, Mattermost, for example) to the required documents.

Use Case 2: What’s that?

Operators often don’t know all the nodes that are being monitored. They don’t know if it’s an important node, and host names, IP addresses, and FQDNs do not give a hint about the nodes, where they belong, and what they are providing.

In the past, the only solution was the asset description field that might contain useful information. But having one field for multiple information was not optimal.

Often categories were used to group nodes somehow, but these were not usable in notifications. Also, adding customized fields into OpenNMS can be painful and not the best approach.

By attaching metadata to nodes in notifications, we are now able to label our nodes and provide a context to the node. Let’s have a look at the examples:

   <node foreign-id="web01-srv" node-label="web01-srv">
      <interface ip-addr="192.168.1.10" status="1" snmp-primary="N">
      <meta-data context="requisition" key="environment" value="pro"/>
      <meta-data context="requisition" key="department"  value="administration"/>
      <meta-data context="requisition" key="application" value="webserver"/>
   </node>
   <node foreign-id="web02-srv" node-label="web02-srv">
      <interface ip-addr="192.168.1.11" status="1" snmp-primary="N">
      <meta-data context="requisition" key="environment" value="test"/>
      <meta-data context="requisition" key="department"  value="it"/>
      <meta-data context="requisition" key="application" value="webserver"/>
   </node>

The nodes have three self-explanatory node metadata. Using them in notifications will provide an idea of what the node is doing or where it belongs.

In a notification configuration you could use the following:

${requisition:application} node %nodelabel% in ${requisition:department}'s ${requisition:environment} environment is down

OpenNMS will dynamically add the defined values into the notification:

webserver node web01-srv in administration pro environment is down

Doing this with Horizon <27, we needed two notification configurations and the amount of configuration can increase very fast if you have more than these two departments and environments.

Also it is possible to add metadata for the destination path name and let OpenNMS use them dynamically for each node, which can deduplicate even more.

2 Likes