How to exclude drive volumes from MIB2 datacollection

faq
sig-k8s
sig-docker
how-to

#1

Problem

When your monitored nodes are doing a lot Docker stuff for example, the SNMP table will catch a lot of disk volumes you probably not want to monitor.

Diagnosis

It can happen to have 50k /var/lib/docker/containers/xyz over the time, depending on the Docker usage.
Even when it’s not so much, it can take very long to load the node’s resource graph page or in worst case, it won’t load.

Just to get an overview you can search for those resources:

[19:50]root@opennms:/opt/opennms/share/rrd# find . -type d -name "var-lib-docker-container*" | wc -l
42355

Solution

Temporary

[19:50]root@opennms:/opt/opennms/share/rrd# find . -type d -name "var-lib-docker*" -exec rm -f {} \;

This will delete the RRD files and the resource page will load without issues.

Persistent

In the mib2 datacollection file in ${OPENNMS_HOME}/datacollection/mib2.xml you have to change the persistenceSelectorStrategy to org.opennms.netmgt.collectd.PersistRegexSelectorStrategy and add a match-expression parameter to match (not) against a specific volume name.

Example:

   <resourceType name="hrStorageIndex" label="Storage (SNMP MIB-2 Host Resources)" resourceLabel="${hrStorageDescr}">
      <persistenceSelectorStrategy class="org.opennms.netmgt.collectd.PersistRegexSelectorStrategy">
            <parameter key="match-expression" value="not(#hrStorageDescr matches '.*(containers).*')"/>
        </persistenceSelectorStrategy>
      <storageStrategy class="org.opennms.netmgt.dao.support.SiblingColumnStorageStrategy">
         <parameter key="sibling-column-name" value="hrStorageDescr"/>
         <parameter key="replace-first" value="s/^-$/_root_fs/"/>
         <parameter key="replace-all" value="s/^-//"/>
         <parameter key="replace-all" value="s/\s//"/>
         <parameter key="replace-all" value="s/:\\.*//"/>
      </storageStrategy>
   </resourceType>

OpenNMS needs a restart after this change.