H29+ won't start with permission error to open ICMP socket

Problem

We have noticed some people ran into issues with ICMP and problems during startup when they have tried to upgrade to Horizon 29+. The biggest change in H29 is running now as a non-privileged user and it requires having a Linux Kernel > 3.10 which introduced a way to give users a way to use datagram socket for ICMP echo/reply datagrams. During the upgrade we install a sysctl file that controls this behavior:

cat /etc/sysctl.d/99-opennms-non-root-icmp.conf
net.ipv4.ping_group_range=1 995

If you run on very old Kernels this option is not available to you.

Symptoms

When you try to start OpenNMS you get error messages

2021-12-20 11:37:25,554 ERROR [Main] o.o.n.i.j.Jni6Pinger: Permission error received while attempting to open ICMP socket. See https://wiki.opennms.org/wiki/ICMP for information on configuring ICMP for non-root.

You might also see in journalctl -u opennms an error message like this:

Dec 20 11:16:52 localhost.localdomain systemd[1]: Started OpenNMS server.
Dec 20 11:18:05 localhost.localdomain systemd[1]: opennms.service: Supervising process 81854 which is not our child. We'll most likely not notice when it exits.

Solution

In case you can’t upgrade your OS to use a more modern Kernel, the only way to workaround is assigning CAP_NET_RAW capabilities. The easiest way to assign them is by modifying the systemd unit of OpenNMS to assign the capabilities accordingly.

Edit the systemd unit /etc/systemd/system/multi-user.target.wants/opennms.service by adding the lines in the [service] section as the following:

CapabilityBoundingSet=CAP_NET_RAW
AmbientCapabilities=CAP_NET_RAW

Reload the sytemd unit with systemctl daemon-reload and restart the service with systemctl restart opennms.

Kudos to jesk and @UberPinguin for help digging through these issues.


:woman_facepalming: You can fix me, I’m a wiki post.

We have added an enhancement to cover this topic also in our installation documentation: [NMS-13866] Add additional steps running as non-root on old Kernels, e.g. RHEL7 - The OpenNMS Issue Tracker