For minimalist Linux Operating systems, you’ll need a little extra configuration to use Longhorn with RKE (Rancher Kubernetes Engine). This document outlines the requirements for using RKE and CoreOS.
CSI doesn’t work with CoreOS + RKE before Longhorn v0.4.1. The reason is that in the case of CoreOS, RKE sets the argument root-dir=/opt/rke/var/lib/kubelet for the kubelet , which is different from the default value /var/lib/kubelet.
For k8s v1.12+, the kubelet will detect the csi.sock according to argument <--kubelet-registration-path> passed in by Kubernetes CSI driver-registrar, and <drivername>-reg.sock (for Longhorn, it’s io.rancher.longhorn-reg.sock) on kubelet path <root-dir>/plugins.
For k8s v1.11, the kubelet will find both sockets on kubelet path /var/lib/kubelet/plugins.
By default, Longhorn CSI driver creates and expose these two sock files on the host path /var/lib/kubelet/plugins. Then the kubelet cannot find <drivername>-reg.sock, so CSI driver doesn’t work.
Furthermore, the kubelet will instruct the CSI plugin to mount the Longhorn volume on <root-dir>/pods/<pod-name>/volumes/kubernetes.io~csi/<volume-name>/mount. But this path inside the CSI plugin container won’t be bind mounted on the host path. And the mount operation for the Longhorn volume is meaningless.
Therefore, in this case, Kubernetes cannot connect to Longhorn using the CSI driver without additional configuration.
Kubernetes v1.11 or higher.
Longhorn v0.4.1 or higher.
1. Add extra binds for the kubelet
This step is only required for For CoreOS + and Kubernetes v1.11. It is not needed for Kubernetes v1.12+.
You will find root-dir in the cmdline of proc kubelet. If it’s not set, the default value /var/lib/kubelet would be used. In the case of CoreOS, the root-dir would be /opt/rke/var/lib/kubelet as shown above.
If the kubelet is using a configuration file, you need to check the configuration file to locate the root-dir parameter.