Problem
We have noticed some people ran into issues with ICMP and problems during startup when they have tried to upgrade to Horizon 29+. The biggest change in H29 is running now as a non-privileged user and it requires having a Linux Kernel > 3.10 which introduced a way to give users a way to use datagram socket for ICMP echo/reply datagrams. During the upgrade we install a sysctl file that controls this behavior:
cat /etc/sysctl.d/99-opennms-non-root-icmp.conf
net.ipv4.ping_group_range=1 995
If you run on very old Kernels this option is not available to you.
Symptoms
When you try to start OpenNMS you get error messages
2021-12-20 11:37:25,554 ERROR [Main] o.o.n.i.j.Jni6Pinger: Permission error received while attempting to open ICMP socket. See https://wiki.opennms.org/wiki/ICMP for information on configuring ICMP for non-root.
You might also see in journalctl -u opennms an error message like this:
Dec 20 11:16:52 localhost.localdomain systemd[1]: Started OpenNMS server.
Dec 20 11:18:05 localhost.localdomain systemd[1]: opennms.service: Supervising process 81854 which is not our child. We'll most likely not notice when it exits.
Solution
The opennms user requires permission to use a datagram socket to be able to send ICMP (ping) messages. By default, a user doesn’t have these permissions. During the installation set sysctl net.ipv4.ping_group_range and give the opennms group the permissions accordingly. If you don’t have a modern Linux Kernel you probably don’t have this option and you need to add the permissions on a process level. You need to add network capabilities to the use of raw datagram sockets. The easiest way to assign them is by modifying the systemd unit of OpenNMS to assign the capabilities accordingly.
Edit the systemd unit /etc/systemd/system/multi-user.target.wants/opennms.service by adding the lines in the [service] section as the following:
AmbientCapabilities=CAP_NET_RAW
Reload the sytemd unit with systemctl daemon-reload and restart the service with systemctl restart opennms.
Kudos to jesk and @UberPinguin for help digging through these issues.
Considerations with SELinux
If this doesn’t solve the problem verify if you have SELinux running and enabled. You can verify this with the sestatus command and gives you information if it’s enabled and the working mode like here:
SELinux status: enabled
SELinuxfs mount: /sys/fs/selinux
SELinux root directory: /etc/selinux
Loaded policy name: targeted
Current mode: enforcing
Mode from config file: enforcing
Policy MLS status: enabled
Policy deny_unknown status: allowed
Memory protection checking: actual (secure)
Max kernel policy version: 33
With audit2allow -a you can verify if opennms violates SELinux policies. In case the command is missing, run dnf install policycoreutils-devel package.
Look for entries preventing to bind ICMP sockets like this:
#============= unconfined_service_t ==============
allow unconfined_service_t node_t:icmp_socket node_bind;
============= unconfined_service_t ==============
allow unconfined_service_t port_t:icmp_socket name_bind;
Create a type enforcement file like this: JniPing.te
module JniPing 1.0;
require {
type unconfined_service_t;
type node_t;
type port_t;
class icmp_socket { name_bind node_bind };
}
#============= unconfined_service_t ==============
allow unconfined_service_t node_t:icmp_socket node_bind;
allow unconfined_service_t port_t:icmp_socket name_bind;
Create an SElinux module and compile it with:
checkmodule -M -m -o JniPing.mod JniPing.te
semodule_package -o JniPing.pp -m JniPing.mod
Install the SELinux package and enforce it with
semodule -i JniPing.pp
getenforce
You can verify the package with semodule --list-modules | grep JniPing. The permission denied message should be fixed now.
Kudos to @va13 for the help figuring this out. 
You can fix me, I’m a wiki post.