Running Minion on Kubernetes with non-root

sig-k8s
minion

#1

I have created a deployment which has a mixture of environments, configMaps and secrets to run the Minion using Kafka. The only thing which is missing, I was not able to set the sysctls for net.ipv4.ping_group_range=0 429496729 so it can run as a minion user for ICMP. The only way to get it working correctly is setting a security context to run as root.

Any hints appreciated.


#2

Can you elaborate on how you tried setting the sysctls? Have you followed https://kubernetes.io/docs/tasks/administer-cluster/sysctl-cluster/?


#3

I tried it like this:

spec:
  securityContext:
    sysctls:
    - name: net.ipv4.ping_group_range
      value: "0 429496729"
  containers:
  - name: minion
    image: no42org/minion:23.0.1

but ran in the error message SysctlForbidden when starting the pod. The docs says net.* should be namespaced, but nevertheless I had to set the --allowed-unsafe-sysctls=net.ipv4.ping_group_range flag on my kubelet daemon.

Iā€™m using microk8s and did this change in /var/snap/microk8s/current/args/kubelet and restarted. No everything is awesome :slight_smile:

Thanks for the hint.

FYI: Ran on Ubuntu 18.0.4.1 LTS with Kernel 4.15.0-42-generic and Kubernetes stable 1.13.


#6

The following works for me:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: minion
spec:
  selector:
    matchLabels:
      app: minion
  template:
    metadata:
      labels:
        app: minion
    spec:
      initContainers:
      - name: init-sysctl
        image: busybox
        command:
        - sysctl
        - -w
        - net.ipv4.ping_group_range=0 429496729
        securityContext:
          privileged: true        
      containers:
      - name: minion
        image: opennms/minion:bleeding
        command: [ /docker-entrypoint.sh, -f ]
        env:
        - name: MINION_ID
          value: minion01
        - name: MINION_LOCATION
          value: K8s
        - name: OPENNMS_HTTP_URL
          value: https://onms.k8s.opennms.org/opennms
        - name: OPENNMS_HTTP_USER
          value: admin
        - name: OPENNMS_HTTP_PASS
          value: admin
        - name: KAFKA_RPC_ACKS
          value: '1'
        - name: KAFKA_RPC_BOOTSTRAP_SERVERS
          value: kafka.k8s.opennms.org:9094
        - name: KAFKA_SINK_BOOTSTRAP_SERVERS
          value: kafka.k8s.opennms.org:9094
        ports:
          - containerPort: 1162
            name: traps
          - containerPort: 1514
            name: syslog
          - containerPort: 8201
            name: shell
        resources:
          limits:
            cpu: 400m
            memory: 512Mi
          requests:
            cpu: 200m
            memory: 256Mi    

Check this out:

āžœ  kubectl get pod
NAME                      READY   STATUS    RESTARTS   AGE
minion-597546bc64-pr4zp   1/1     Running   0          5m27s

āžœ  kubectl exec minion-597546bc64-pr4zp -it -- ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
minion       1     0  0 14:24 ?        00:00:00 /bin/bash -e /docker-entrypoint.
minion     113     1  0 14:24 ?        00:00:00 /bin/sh ./karaf server
minion     174   113 15 14:24 ?        00:00:51 /usr/lib/jvm/java/bin/java -Xms1
minion     600     0  0 14:30 pts/0    00:00:00 ps -ef