Horizon 29 with non-root breaks ICMPv6 pings on EL7

I have just updated to 29.0.0 on CentOS 7 (setting up the opennms user and group manually because the spec file uses a broken check).

Running as opennms user, however, all my IPv6 interfaces failed the ICMP pings, because opennms isn’t able to set up the ipv6 pinger.

2021-11-11 13:46:45,633 ERROR [Main] o.o.n.i.j.Jni6Pinger: Permission error received while attempting to open ICMP socket. See https://wiki.opennms.org/wiki/ICMP for information on configuring ICMP for non-root.

The installer (I guess) set up the kernel variable ping_group_range to include the opennms group which has gid number 500 on my system:

$ cat /etc/sysctl.d/99-opennms-non-root-icmp.conf 
net.ipv4.ping_group_range=1 500

So far so good. However: CentOS 7 and all RHEL7 derivates have kernel 3.10 as standard kernel. It’s my understanding from https://bugzilla.redhat.com/show_bug.cgi?id=1315335 that IPv6 support of net.ipv4.ping_group_range was only added in kernel 3.11.

In other words: if you are still running opennms on EL7 and you are using IPv6 on your devices upgrading to Horizon 29 and running as non-root will break your IPv6 ICMP pings. So I am back to running as root…

If you have an older Kernel without the net.ipv4.ping_group_range functionality you can give the Java binary the cap_net_raw capabilities. It gives the Java process a bit more permissions than necessary but is still better than running as root.

setcap cap_net_raw+ep ${path/to/java}/bin/java

You can verify the path to the Java binary OpenNMS is using, by looking up the java.conf configuration file.

1 Like

True. opennms is running on a dedicated server, thus giving java capabilities shouldn’t be a problem.

On the other hand, at the moment the migration to “not running as root” resulted in “all files are owned by root”, i.e. no separation of code and data. The opennms process can still modify all of it’s own code. From a security standpoint, all code, e.g. in the bin, contrib, lib, system, etc. should still be exclusively owned by root so that the opennms process isn’t able to modify the content. It should only be able to write into directories and files which are really necessary…

So I guess, I’ll wait a few updates until I try to run it at opennms again…

Ah ok, so what you are saying is you expect to have libs and executable binaries a different user than the data OpenNMS produces and its configuration.

Her an example, with a user name I just came up with opennms-data for:

  • /opt/opennms/etc
  • /opt/opennms/share/rrd
  • /opt/opennms/share/reports

and an opennms-service user for lib, Karaf, and executable JARs? Is this what you are looking for?

Ah, so, same as it’s always been, then?

No. I thought just of a standard setup like for any other server, e.g. httpd, postfix, MySQL, etc.

Binaries, libraries, message files, etc. and anything which doesn’t change and the server doesn’t have to modify can be installed with owner/group root.

Only data, caches etc would be owner “opennms”.

Take postfix installation: binaries, libraries, etc are all root owned. Queue and mail directories, caches etc. are set to allow postfix to write.

Basically, anything installed into /usr is root owned and technically might even be a read only partition. Data is written into /var and has postfix owner.

Config is in /etc and usually is root but readable by the service only, if it contains sensitive information. opennms is quite different in that aspect as it actively modifies it’s own configuration files, so that probably different.

But generally: if opennms is not supposed to modify a file that file can be root owned with no write permissions for user “opennms” running the service…

So most of these services require root permissions for different tasks depending a bit on the service, for example, some services like Postfix need to impersonate other process users which is the reason the master process of Postfix requires root. Another use case is for processes that need to bind privileged ports < 1024, they start as root bind the port but drop the permission afterward to a system user.

In the case of OpenNMS, the only reason we need a bit more than system user permissions is to send ICMP echo/reply datagram messages. The net.ipv4.ping_group_range kernel parameter would give the opennms user exact these permission. If you use setcap cap_net_raw you get a bit more permissions, you get access to RAW sockets which allow you do other things as well, like spoofing UDP address fields, but it is still not a full root user.

So if you have someone getting on your system through an OpenNMS service he will have exactly the same permissions as the OpenNMS system user but still, it is not a root user which can cause much more damage than the restricted system user.

I am not talking about the user running the service. I am talking about the owner of the files.

Let me put it this way: why does the running opennms service (while running as user opennms) need write access to /opt/opennms or the bin sub directory or the lib directory etc.? Because that’s how it’s currently set up if you simple chown /opt/opennms recursively to opennms.

Then all data and all code belongs to opennms and opennms can change all code binaries. That’s neither necessary nor secure.

Separate code and immutable files from data. The opennms code, libraries should be owned by root and only be written by the rpm installation. The opennms user running the opennms service should only have write permissions (and ownership) of files it needs to write to…

ok got it. Here a very naive first thought to achieve this:

  • /opt/opennms/{etc,data,deploy,logs,share,system} seem to me need read/write permissions, cause the OpenNMS services like the Web UI, Karaf OSGi and others read/writes files here.
  • The rest of it might be candidates where we would just need read permissions
  • There might be dragons with a) opennms.pid, karaf.pid files b) heap and thread dumps but we have soem control where we want to write them.

I think this might be something now to be tested and would we can create an enhancement in JIRA.