Flow exporters missing from Helm template query

grafana
helm
netflow
#1

In Horizon 23, I have a node with successful SNMP discovery. Netflow is coming in through a Minion and the resulting ElasticSearch documents appear to be properly enriched.

Sample flow document:

{
    "_index": "netflow-2019-04-11",
    "_type": "netflow",
    "_score": null,
    "_source": {
        "@timestamp": 1554991931000,
        "@version": 1,
        "host": "10.xxx.xxx.xxx",
        "location": "remote",
        "netflow.application": "redacted",
        "netflow.bytes": 176,
        "netflow.convo_key": "redacted",
        "netflow.direction": "ingress",
        "netflow.dst_addr": "10.yyy.yyy.yyy",
        "netflow.dst_locality": "private",
        "netflow.dst_port": 9999,
        "netflow.first_switched": 1554991915122,
        "netflow.flow_locality": "private",
        "netflow.flow_records": 26,
        "netflow.flow_seq_num": 694279,
        "netflow.input_snmp": 11,
        "netflow.last_switched": 1554991915262,
        "netflow.next_hop": "zzz.zzz.zzz.zzz",
        "netflow.output_snmp": 7,
        "netflow.packets": 2,
        "netflow.protocol": 6,
        "netflow.sampling_algorithm": "Unassigned",
        "netflow.src_addr": "10.nnn.nnn.nnn",
        "netflow.src_locality": "private",
        "netflow.src_port": 8888,
        "netflow.tcp_flags": 24,
        "netflow.tos": 88,
        "netflow.version": "Netflow v9",
        "netflow.vlan": "redacted",
        "node_exporter": {
            "foreign_source": "remote locations",
            "foreign_id": "35",
            "node_id": 35,
            "categories": []
        }
    }
}

The host field matches up with node 35. The netflow.input_snmp and netflow.output_snmp fields are listed as SNMP interfaces on the node. The node_exporter field seems to indicate it was enriched properly.

Yet in Helm, interfacesOnExporterNodeWithFlows(35) does not give any usable interfaces with flows on them. It does list some interfaces, but not the ones that are actually exporting flows like the above document.

Any thoughts? Thanks!

#2

Can you please tell us which version of Grafana and which version the Helm app has?

#3

Grafana is at 6.0.2 and Helm is at 3.0.1

1 Like
#4

The REST API should pull these from the database. Does the expected node id and ifIndex appear if we query the database directly?

opennms=# select nodeid,snmpifindex,snmpifname from snmpinterface where hasflows = true order by nodeid,snmpifindex;
 nodeid | snmpifindex |    snmpifname    
--------+-------------+------------------
      1 |           2 | eth0
    138 |           1 | igb0
    138 |           2 | igb1
    138 |           3 | lo0
    138 |           4 | enc0
1 Like
#5
opennms=> select nodeid,snmpifindex,snmpifname from snmpinterface where hasflows = true and nodeid = 35 order by nodeid,snmpifindex;
 nodeid | snmpifindex |  snmpifname
--------+-------------+---------------
     35 |           3 | eth1-lan
     35 |          10 | eth1-lan.2000
     35 |          27 | eth1-lan.2410
     35 |          28 | eth1-lan.2420
     35 |          31 | eth1-lan.2501
     35 |          32 | eth1-lan.2590
     35 |          35 | eth1-lan.2904
     35 |          38 | eth1-lan.3010

I’m not sure why snmpifindex = 7 isn’t showing up in the table, when it has flows if I create a Grafana graph panel with includeOther(), topN(10), withExporterNode(35), withIfIndex(7), perSecond()

#6

A few days later, the interfaces show up now for the template query. Out of curiosity, what’s the logic for setting hasflows = true? Is it something like flow comes in => interface exists => set hasflows, or interface is discovered => flows exist for interface and node => set hasflows, or something else entirely?

The flows were coming in before the node and interface existed, but even after the node was provisioned and SNMP discovery found the interfaces and the documents were enriched, hasflows was false for iface 7 until some time later.

#7

Here’s the relevant code: https://github.com/OpenNMS/opennms/blob/007cb0085f1ea9a11a4db07b005119e50bd95f1f/features/flows/elastic/src/main/java/org/opennms/netmgt/flows/elastic/ElasticFlowRepository.java#L253

Based on what you reported it should have been updated immediately, looks like there may be a bug.