# External runtime dependencies k0s is packaged as a single binary, which includes all the needed components. All the binaries are statically linked which means that in typical use cases there's an absolute minimum of external runtime dependencies. However, depending on the node role and cluster configuration, some of the underlying components _may_ have specific dependencies, like OS level tools, packages and libraries. This page aims to provide a comprehensive overview. The following command checks for known requirements on a host (currently only available on Linux): ```shell k0s sysinfo ``` ## Linux specific ### Linux kernel configuration Needless to say, as k0s operates Kubernetes worker nodes, a certain number of Linux kernel modules and configurations are required on the host. This essentially stems from the requirement to run containers and set up networking for them. The needed kernel configuration items are listed below. All of them are available in Kernel versions 4.3 and above. If running on older kernels, check if the distro in use has backported some features; nevertheless, it might meet the requirements. k0s will check the Linux kernel release as part of its pre-flight checks and issue a warning if it's below 3.10. The list covers ONLY the k0s/Kubernetes components' needs on worker nodes. Your own workloads may require more. - [`CONFIG_CGROUPS`](https://github.com/torvalds/linux/blob/v4.3/init/Kconfig#L927): Control Group support - [`CONFIG_CGROUP_SCHED`](https://github.com/torvalds/linux/blob/v4.3/init/Kconfig#L1081): Group CPU scheduler - [`CONFIG_FAIR_GROUP_SCHED`](https://github.com/torvalds/linux/blob/v4.3/init/Kconfig#L1090): Group scheduling for SCHED_OTHER
[kubernetes/kubeadm#2335 (comment)](https://github.com/kubernetes/kubeadm/issues/2335#issuecomment-717996215) - _(optional)_ [`CONFIG_CFS_BANDWIDTH`](https://github.com/torvalds/linux/blob/v4.3/init/Kconfig#L1095): CPU bandwidth provisioning for FAIR_GROUP_SCHED
Required if CPU CFS quota enforcement is enabled for containers that specify CPU limits (`--cpu-cfs-quota`). - _(optional)_ [`CONFIG_BLK_CGROUP`](https://github.com/torvalds/linux/blob/v4.3/init/Kconfig#L1119): Block IO controller
[kubernetes/kubernetes#92287 (comment)](https://github.com/kubernetes/kubernetes/issues/92287#issuecomment-1010723587) - [`CONFIG_NAMESPACES`](https://github.com/torvalds/linux/blob/v4.3/init/Kconfig#L1168): Namespaces support - [`CONFIG_UTS_NS`](https://github.com/torvalds/linux/blob/v4.3/init/Kconfig#L1180): UTS namespace - [`CONFIG_IPC_NS`](https://github.com/torvalds/linux/blob/v4.3/init/Kconfig#L1187): IPC namespace - [`CONFIG_PID_NS`](https://github.com/torvalds/linux/blob/v4.3/init/Kconfig#L1210): PID namespace - [`CONFIG_NET_NS`](https://github.com/torvalds/linux/blob/v4.3/init/Kconfig#L1218): Network namespace - [`CONFIG_NET`](https://github.com/torvalds/linux/blob/v4.3/net/Kconfig#L5): Networking support - [`CONFIG_INET`](https://github.com/torvalds/linux/blob/v4.3/net/Kconfig#L58): TCP/IP networking - [`CONFIG_NETFILTER`](https://github.com/torvalds/linux/blob/v4.3/net/Kconfig#L109): Network packet filtering framework (Netfilter) - _(optional)_ [`CONFIG_NETFILTER_ADVANCED`](https://github.com/torvalds/linux/blob/v4.3/net/Kconfig#L171): Advanced netfilter configuration - [`CONFIG_NETFILTER_XTABLES`](https://github.com/torvalds/linux/blob/v4.3/net/netfilter/Kconfig#L567): Netfilter Xtables support - [`CONFIG_NETFILTER_XT_TARGET_REDIRECT`](https://github.com/torvalds/linux/blob/v4.3/net/netfilter/Kconfig#L853): REDIRECT target support - [`CONFIG_NETFILTER_XT_MATCH_COMMENT`](https://github.com/torvalds/linux/blob/v4.3/net/netfilter/Kconfig#L1002): "comment" match support - [`CONFIG_EXT4_FS`](https://github.com/torvalds/linux/blob/v4.3/fs/ext4/Kconfig#L33): The Extended 4 (ext4) filesystem - [`CONFIG_PROC_FS`](https://github.com/torvalds/linux/blob/v4.3/fs/proc/Kconfig#L1): /proc file system support **Note:** As part of its pre-flight checks, k0s will try to inspect and validate the kernel configuration. In order for that to succeed, the configuration needs to be accessible at runtime. There are some typical places that k0s will check. A bullet-proof way to ensure the accessibility is to enable [`CONFIG_IKCONFIG_PROC`](https://github.com/torvalds/linux/blob/v4.3/init/Kconfig#L807), and, if enabled as a module, to load the `configs` module: `modprobe configs`. ### Control Groups (cgroups) [cgroup v2] is required by default, k0s will fail its pre-flight checks if it's unavailable on the node. As of Kubernetes 1.31, [cgroup v1] is in [maintenance mode] ([KEP-4569]). As of Kubernetes 1.35, the kubelet will fail to start by default if cgroup v2 is not detected on the node. In order to run Kubernetes with cgroup v1, bypass the k0s pre-flight checks via `--ignore-pre-flight-checks` and set `failCgroupV1: false` in the kubelet configuration via the k0s [worker profiles]: ```yaml ... spec: workerProfiles: # By default, workers will pick up the default profile. - name: default values: failCgroupV1: false ``` !!! warning "Migrate to cgroup v2 now!" The above escape hatch will not be available indefinitely, so plan accordingly. [KEP-5573] states: > We will commit to removing cgroup v1 code but there is not yet a timeline for > removal. The removal will be done no earlier than 1.38 to maintain the k8s > deprecation policy. Required [cgroup] controllers: - `cpu` - `cpuacct` - `cpuset` - `memory` - `devices` - `freezer` - `pids` Optional cgroup controllers: - `hugetlb` ([kubernetes/kubeadm#2335 (comment)]) - `blkio` ([kubernetes/kubernetes#92287 (comment)])
containerd and cri-o will use blkio to track disk I/O and throttling in both cgroup v1 and v2. [cgroup]: https://man7.org/linux/man-pages/man7/cgroups.7.html [cgroup v1]: https://www.kernel.org/doc/html/v5.16/admin-guide/cgroup-v1/ [cgroup v2]: https://www.kernel.org/doc/html/v5.16/admin-guide/cgroup-v2.html [kubernetes/kubeadm#2335 (comment)]: https://github.com/kubernetes/kubeadm/issues/2335#issuecomment-722405527 [kubernetes/kubernetes#92287 (comment)]: https://github.com/kubernetes/kubernetes/issues/92287#issuecomment-1010723587 [maintenance mode]: https://kubernetes.io/blog/2024/08/14/kubernetes-1-31-moving-cgroup-v1-support-maintenance-mode/ [KEP-4569]: https://www.kubernetes.dev/resources/keps/4569/ [KEP-5573]: https://www.kubernetes.dev/resources/keps/5573/ [worker profiles]: worker-node-config.md#kubelet-configuration ### No integration with Name Service Switch (NSS) APIs The k0s Linux binaries, including the ones distributed on the GitHub releases pages, are by default statically linked and do not depend on the host's C library. This ensures that k0s can run seamlessly across a wide range of Linux environments. The `k0s` executable and most of the embedded binaries are built without [cgo] and use the [pure Go resolver][netdns-go], which only consults the `files` and `dns` sources of `/etc/nsswitch.conf`. The Go executables that do require cgo, such as `containerd` and `runc`, as well as all non-Go executables, are statically linked against [musl libc], which doesn't implement NSS at all. In all cases, k0s cannot use [glibc's NSS APIs], which require dynamic linking. This limitation is particularly relevant when a system uses NSS plugins, such as [nss-myhostname], for resolving network names like `localhost`. Systems lacking a dedicated stub resolver capable of handling `localhost` DNS queries specifically will encounter issues running k0s. To mitigate this, users are advised to either activate a stub DNS resolver, such as `systemd-resolved`, or to manually add `localhost` entries to the `/etc/hosts` file as shown below: ```text 127.0.0.1 localhost ::1 localhost ``` The same limitation applies to systems that resolve all names via NSS, such as through [nss-resolve] on hosts running `systemd-resolved`. Programs linked against glibc can resolve names just fine on such systems, even if `/etc/resolv.conf` doesn't list any nameservers. In contrast, k0s and its embedded components only ever consult `/etc/resolv.conf`. If that file has no nameserver entries, they fall back to querying a nameserver on localhost, which usually doesn't exist. This leaves k0s without working name resolution, although the host itself appears fine. Make sure that `/etc/resolv.conf` lists reachable nameservers. When using `systemd-resolved`, this typically means making `/etc/resolv.conf` a symbolic link to `/run/systemd/resolve/stub-resolv.conf`, as described in the [systemd-resolved documentation]. Pods can't reach that stub resolver on the host's loopback interface, which is why k0s configures the kubelet to use systemd-resolved's uplink file `/run/systemd/resolve/resolv.conf` for pods instead. The k0s pre-flight checks detect this situation by resolving a name under the reserved `invalid` top-level domain, which any working resolver answers with NXDOMAIN. If that lookup fails, k0s issues a warning. If `/etc/resolv.conf` doesn't list any nameservers on top of that, worker nodes refuse to start. Controllers only warn in that case. [cgo]: https://pkg.go.dev/cmd/cgo [netdns-go]: https://pkg.go.dev/net#hdr-Name_Resolution [musl libc]: https://musl.libc.org/ [glibc's NSS APIs]: https://www.gnu.org/software/libc/manual/html_node/Name-Service-Switch.html [nss-myhostname]: https://www.freedesktop.org/software/systemd/man/latest/nss-myhostname.html [nss-resolve]: https://www.freedesktop.org/software/systemd/man/latest/nss-resolve.html [systemd-resolved documentation]: https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html#/etc/resolv.conf ### Kernel Parameters #### Inotify Instance Limit [Inotify API](https://www.man7.org/linux/man-pages/man7/inotify.7.html) allows applications to monitor filesystem changes. Workloads that run many pods or watch large numbers of files can exhaust the node's default inotify limits, particularly `fs.inotify.max_user_instances`, which limits how many inotify instances one user (UID) can have. A value of at least 1024 is recommended; 8192 is commonly used in Kubernetes environments: ```text fs.inotify.max_user_instances = 8192 ``` 1. Check the current value: ```bash cat /proc/sys/fs/inotify/max_user_instances ``` 2. To apply the new value persistently, create a `sysctl` configuration file, for example, `/etc/sysctl.d/99-inotify.conf`, and place the parameter setting from above in it. 3. Load the new kernel values: ```bash sudo sysctl --system ``` ### External hard dependencies There are very few external tools that are needed or used. #### mount/umount When setting up pods, kubelet will call `mount` binary on the host. Similarly when destroying pods it will call `umount`. `mount` and `umount` are only needed on worker nodes where kubelet runs. ### External soft dependencies There are a few external tools that may be needed or used under specific circumstances: #### containerd and AppArmor In order to use containerd in conjunction with [AppArmor], it must be enabled in the kernel and the `/sbin/apparmor_parser` executable must be installed on the host, otherwise containerd will [disable][cd-aa] AppArmor support. [cd-aa]: https://github.com/containerd/containerd/blob/v2.3.5/pkg/apparmor/apparmor_linux.go#L34-L45 [AppArmor]: https://wiki.ubuntu.com/AppArmor #### iptables iptables may be executed to detect if there are any existing iptables rules and if those are in legacy or nft mode. If iptables is not found, k0s will assume that there are no pre-existing iptables rules. #### useradd / adduser During `k0s install` the external tool `useradd` will be used on the controllers to create system user accounts for k0s. If this doesn't exist it will fall-back to busybox's `adduser`. #### userdel / deluser `k0s reset` will execute either `userdel` or `deluser` to clean up system user accounts. #### modprobe On k0s worker `modprobe` will be executed to load missing kernel modules if they are not detected. #### id External `id` will be executed as a fallback if local user lookup fails, in case NSS is used. ## Windows specific TBD.