Kafka message broker

Hi,

I am trying to setup Kafka message broker on Horizon 30.0.2, but I am running into the below error -

[INFO] Invoking start on object OpenNMS:Name=KarafStartupMonitor
org.apache.karaf.features.internal.util.MultiException: Error:
	Error downloading mvn:org.opennms.core.ipc.sink.kafka/org.opennms.core.ipc.sink.kafka.client/30.0.2
	at org.apache.karaf.features.internal.download.impl.MavenDownloadManager$MavenDownloader.<init>(MavenDownloadManager.java:91)
	at org.apache.karaf.features.internal.download.impl.MavenDownloadManager.createDownloader(MavenDownloadManager.java:72)
	at org.apache.karaf.features.internal.region.Subsystem.downloadBundles(Subsystem.java:457)
	at org.apache.karaf.features.internal.region.Subsystem.downloadBundles(Subsystem.java:452)
	at org.apache.karaf.features.internal.region.SubsystemResolver.resolve(SubsystemResolver.java:224)
	at org.apache.karaf.features.internal.service.Deployer.deploy(Deployer.java:399)
	at org.apache.karaf.features.internal.service.FeaturesServiceImpl.doProvision(FeaturesServiceImpl.java:1069)
	at org.apache.karaf.features.internal.service.FeaturesServiceImpl.lambda$doProvisionInThread$13(FeaturesServiceImpl.java:1004)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

Any ideas to resolve this?

What documentation were you following to enable Kafka? Never seen this before.

I am following the below documentation -

https://docs.opennms.com/horizon/30/deployment/core/setup-message-broker.html

Share your config? Anything helpful in the karaf.log ?

karaf.log for org.opennms.core.ipc.sink.kafka, I don’t see these error messages for org.opennms.core.ipc.rpc.kafka

logs/eventd.log:2022-09-07 12:48:28,498 INFO  [Main] o.s.b.f.x.XmlBeanDefinitionReader: Loading XML bean definitions from URL [jar:file:/usr/share/opennms/lib/org.opennms.core.ipc.sink.kafka.client-30.0.2.jar!/META-INF/opennms/component-dao.xml]
logs/eventd.log:2022-09-07 12:48:28,504 INFO  [Main] o.s.b.f.x.XmlBeanDefinitionReader: Loading XML bean definitions from URL [jar:file:/usr/share/opennms/lib/org.opennms.core.ipc.sink.kafka.server-30.0.2.jar!/META-INF/opennms/component-dao.xml]
logs/karaf.log:2022-09-07T12:49:43,635 | WARN  | downloader-6-thread-5 | AetherBasedResolver              | 4 - org.ops4j.pax.url.mvn - 2.6.10 | Error resolving artifact org.opennms.core.ipc.sink.kafka:org.opennms.core.ipc.sink.kafka.client:jar:30.0.2: [Could not find artifact org.opennms.core.ipc.sink.kafka:org.opennms.core.ipc.sink.kafka.client:jar:30.0.2]
logs/karaf.log:java.io.IOException: Error resolving artifact org.opennms.core.ipc.sink.kafka:org.opennms.core.ipc.sink.kafka.client:jar:30.0.2: [Could not find artifact org.opennms.core.ipc.sink.kafka:org.opennms.core.ipc.sink.kafka.client:jar:30.0.2]
logs/karaf.log:	Suppressed: shaded.org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact org.opennms.core.ipc.sink.kafka:org.opennms.core.ipc.sink.kafka.client:jar:30.0.2
logs/karaf.log:Caused by: shaded.org.eclipse.aether.resolution.ArtifactResolutionException: Error resolving artifact org.opennms.core.ipc.sink.kafka:org.opennms.core.ipc.sink.kafka.client:jar:30.0.2
logs/karaf.log:2022-09-07T12:49:43,650 | WARN  | downloader-6-thread-5 | AetherBasedResolver              | 4 - org.ops4j.pax.url.mvn - 2.6.10 | Error resolving artifact org.opennms.core.ipc.sink.kafka:org.opennms.core.ipc.sink.kafka.client:jar:30.0.2: [Could not find artifact org.opennms.core.ipc.sink.kafka:org.opennms.core.ipc.sink.kafka.client:jar:30.0.2]
logs/karaf.log:java.io.IOException: Error resolving artifact org.opennms.core.ipc.sink.kafka:org.opennms.core.ipc.sink.kafka.client:jar:30.0.2: [Could not find artifact org.opennms.core.ipc.sink.kafka:org.opennms.core.ipc.sink.kafka.client:jar:30.0.2]
logs/karaf.log:	Suppressed: shaded.org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact org.opennms.core.ipc.sink.kafka:org.opennms.core.ipc.sink.kafka.client:jar:30.0.2
logs/karaf.log:Caused by: shaded.org.eclipse.aether.resolution.ArtifactResolutionException: Error resolving artifact org.opennms.core.ipc.sink.kafka:org.opennms.core.ipc.sink.kafka.client:jar:30.0.2
logs/karaf.log:	Error downloading mvn:org.opennms.core.ipc.sink.kafka/org.opennms.core.ipc.sink.kafka.client/30.0.2
logs/karaf.log:	Suppressed: java.io.IOException: Error downloading mvn:org.opennms.core.ipc.sink.kafka/org.opennms.core.ipc.sink.kafka.client/30.0.2
logs/karaf.log:	Caused by: java.io.IOException: Error resolving artifact org.opennms.core.ipc.sink.kafka:org.opennms.core.ipc.sink.kafka.client:jar:30.0.2: [Could not find artifact org.opennms.core.ipc.sink.kafka:org.opennms.core.ipc.sink.kafka.client:jar:30.0.2]
logs/karaf.log:		Suppressed: shaded.org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact org.opennms.core.ipc.sink.kafka:org.opennms.core.ipc.sink.kafka.client:jar:30.0.2
logs/karaf.log:	Caused by: shaded.org.eclipse.aether.resolution.ArtifactResolutionException: Error resolving artifact org.opennms.core.ipc.sink.kafka:org.opennms.core.ipc.sink.kafka.client:jar:30.0.2

Yeah, that’s real strange. Maybe try $OPENNMS_HOME/bin/fix-karaf-setup.sh and restart OpenNMS afterwards.

I tried doing that, didn’t really help.

What’s in your $OPENNMS_HOME/etc/opennms.properties.d/kafka.properties ?

org.opennms.core.ipc.rpc.strategy=kafka
org.opennms.core.ipc.sink.strategy=kafka
org.opennms.core.ipc.sink.initialSleepTime=60000
org.opennms.core.ipc.rpc.kafka.bootstrap.servers=x.x.x.x:9092
org.opennms.core.ipc.sink.kafka.bootstrap.servers=x.x.x.x:9092

Not sure that it matters in this case, but there’s no need to separate out rpc and sink (and twin) anymore, you can use:

org.opennms.core.ipc.strategy=kafka
org.opennms.core.ipc.sink.initialSleepTime=60000
org.opennms.core.ipc.kafka.bootstrap.servers=x.x.x.x:9092,x.x.x.x:9092

And everything will inherit those parameters.

I made those changes, but still getting the same error.