Ensure time synchronization for your OpenNMS components

When you run OpenNMS services and components distributed over several hosts. You have to ensure the time is synchronized to avoid strange and hard to debug problems. Unfortunately time synchronization is not installed by default. You as an administrator have to take care of it and this is a requirement a) operationally so your components work well together and b) time is critical in monitoring in general.
You can operationally run in strange issues when you distribute OpenNMS components over dedicated servers such as:

  • PostgreSQL database and OpenNMS core services
  • Sentinel to offload work close to the OpenNMS Server and the Minions in remote locations
  • A dedicated Cassandra Cluster for Time Series data with Newts
  • Network flows, alarms and events stored in a dedicated ElasticSearch Cluster

To ensure you have a synchronized time on all your servers here is a guide to set it up on common Linux systems.

On modern operation systems this can be achieved by installing chrony.

yum install chrony     [On CentOS/RHEL]
apt install chrony     [On Debian/Ubuntu]

Check if the service is running with

systemctl status chronyd

Check if the clock is synchronised and printing relevant information:

root@horizon:~$ chronyc tracking

Reference ID    : D5EF9A0C (arethusa.tweakers.net)
Stratum         : 3
Ref time (UTC)  : Wed Jan 30 13:19:30 2019
System time     : 0.000130851 seconds slow of NTP time
Last offset     : -0.000221488 seconds
RMS offset      : 0.000114738 seconds
Frequency       : 0.397 ppm fast
Residual freq   : -0.303 ppm
Skew            : 4.194 ppm
Root delay      : 0.018937271 seconds
Root dispersion : 0.001036274 seconds
Update interval : 64.6 seconds
Leap status     : Normal

If you run an older operating system you can use ntpd which can be installed with:

yum install ntp     [On CentOS/RHEL]
apt install ntp     [On Debian/Ubuntu]

To query time and set the clock you can execute the command.

ntpdate pool.ntp.org
31 Jan 09:18:48 ntpdate[2049]: adjust time server offset -0.001981 sec

:point_up: You should be aware this command will set the clock to a new time hard. Running chrony or ntpd slowly changes the time to avoid issues with drastically changing the system time.

It is useful to pick an NTP server closed to your zone, check the list of available NTP server pools.

To enable ntpd on system boot and start it immediately with systemd you can run the following commands:

systemctl enable ntpd
systemctl start ntpd

The systemctl status ntpd command should show you now an active (running) ntpd.service.

You can adjust the behaviour of ntpd in /etc/ntpd.conf such as the NTP server you want to synchronize with:

# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
server 0.centos.pool.ntp.org iburst
server 1.centos.pool.ntp.org iburst
server 2.centos.pool.ntp.org iburst
server 3.centos.pool.ntp.org iburst

You have to restart ntpd to apply changes you made in the configuration file.