---
date: 2023-12-13
status: new
---
# Kubernetes 1.29 Release: Ambitious Mandala Theme
> Author: [mengjiao-liu](https://github.com/mengjiao-liu)
On December 13, 2023, Pacific Time, Kubernetes 1.29 with the theme "Mandala" is officially released.
This version is released 4 months after the previous version and is the third version of 2023, also the last version of this year.
The release team has tracked 49 enhancements in this version. Among them, 11 feature upgrades have been promoted to stable, 19 existing features have been optimized and upgraded to Beta, and there are as many as 19 brand new features at the Alpha level. Version 1.29 includes many important features and user experience optimizations, the next section of this article will provide detailed introduction to some important features.
## Important Features
### [Networking] KEP-3866: nftables as kube-proxy backend (Alpha)
In Kubernetes v1.29, Kubernetes uses nftables as the new backend for kube-proxy, and this feature is now in Alpha version. iptables has performance issues that cannot be fixed, and performance degradation increases as the size of the rule set increases. Due to its unfixable issues, the development speed of iptables in the kernel has slowed down significantly and most of it has stopped. New features will not be added to iptables, and new features and performance improvements are mainly being added to nftables. nftables can do everything that iptables can do, and do it better.
In particular, RedHat has announced that iptables has been deprecated in RHEL 9 and may be completely removed in RHEL 10 several years later. Debian has removed iptables from the __required__ software packages in Debian 11 (Bullseye). For the above reasons, kube-proxy introduces nftables as the new backend for kube-proxy.
The subsequent goal of this feature is to make nftables the default backend for kube-proxy (replacing iptables and ipvs).
Administrators must enable the feature gate __NFTablesProxyMode__ to make this feature available, and then run kube-proxy with the __--proxy-mode=nftables__ flag.
It should be noted that although the __nftables__ mode may be very similar to the __iptables__ mode, certain CNI plugins, NetworkPolicy implementations, etc. may need to be updated to use it. This may cause some compatibility issues.
### [Storage] KEP-2495: PV/PVC __ReadWriteOncePod__ Access Mode (GA)
In Kubernetes, the access mode defines how persistent storage is used. These access modes are part of the PersistentVolume (PV) and PersistentVolumeClaim (PVC) specifications. __ReadWriteOncePod__ was introduced as the fourth access mode (the previous three access modes are ReadWriteOnce, ReadOnlyMany, ReadWriteMany) in Kubernetes v1.22 as an Alpha feature and has reached GA in Kubernetes v1.29. The main purpose of this feature is to provide more flexible access control for storage resources, ensuring that only one Pod can access the storage in read-write mode. This restriction is very useful for certain specific use cases, such as ensuring that only one Pod can modify the data in the storage to prevent data conflicts or corruption.
This feature also updates the Kubernetes scheduler to support pod preemption related to __ReadWriteOncePod__ storage. This means that if two Pods request the __PersistentVolumeClaim__ with __ReadWriteOncePod__ , the Pod with the highest priority will be granted access to the __PersistentVolumeClaim__ , and any lower priority pods will be preempted from the node and will not be able to access the __PersistentVolumeClaim__ .
The following example creates a new PVC using the __ReadWriteOncePod__ access mode:
```yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: single-writer-only
spec:
accessModes:
- ReadWriteOncePod
resources:
requests:
storage: 1Gi
```
By introducing the ReadWriteOncePod feature, Kubernetes allows users to better control access permissions to storage resources and improves the flexibility of managing storage in containerized environments.
### [Auth] KEP-3299: KMS v2 Enhancement (GA)
One of the first things to consider when securing a Kubernetes cluster is encrypting static etcd data. KMS provides a way for providers to leverage keys stored in an external key service to perform this encryption.
Kubernetes KMS (Key Management Service) is crucial for the secure management and encryption of secrets. With the release of Kubernetes v1.29, the KMSv2 features with feature gates __KMSv2__ and __KMSv2KDF__ have been upgraded to GA, and the __KMSv1__ feature gate is now disabled by default.
This has become a stable feature that focuses on improving the KMS plugin framework, which is crucial for secure management. These improvements ensure that Kubernetes secrets remain a powerful and secure way to store sensitive information.
### [Node] KEP-753: Native Support for Sidecar Containers (Beta)
Native support for Sidecar containers was introduced as Alpha in Kubernetes v1.28 and upgraded to Beta in v1.29, with the __SidecarContainers__ feature gate enabled by default.
Starting from Kubernetes v1.29, if your Pod contains one or more sidecar containers (init containers with an always restart policy), Kubelet will delay sending TERM signals to these Sidecar containers until the last main container is fully terminated. Sidecar containers will be terminated in the opposite order defined in the Pod spec. This ensures that Sidecar containers continue to serve other containers in the Pod until they are no longer needed.
### [Auth/Apps] KEP-2799: Reduce Secret-based Service Account Tokens (Beta)
The __legacy-service-account-token-cleaner__ controller, which runs as part of the kube-controller-manager, now has Beta support for the __LegacyServiceAccountTokenCleanUp__ feature gate (enabled by default). The __legacy-service-account-token-cleaner__ controller periodically deletes unused service account token keys that have not been used within the time specified by __--legacy-service-account-token-clean-up-period__ (default is one year). The controller marks the Secret as invalid by adding a label named __kubernetes.io/legacy-token-invalid-since__ with the current date as the value. If the invalid Secret is not used within the specified time, the controller deletes it.
The following is an example of an old token that has been marked with __kubernetes.io/legacy-token-last-used__ and __kubernetes.io/legacy-token-invalid-since__ labels:
```yaml
apiVersion: v1
kind: Secret
metadata:
name: build-robot-secret
namespace: default
labels:
kubernetes.io/legacy-token-last-used: 2022-10-24
kubernetes.io/legacy-token-invalid-since: 2023-10-25
annotations:
kubernetes.io/service-account.name: build-robot
type: kubernetes.io/service-account-token
```
### [Windows] KEP-1287: Pod Resource In-place Update for Windows (Alpha)
In Kubernetes v1.29, Windows support for Pod resource in-place update is introduced as Alpha, allowing resources to be changed without recreating Pods or restarting containers.
### [Networking] KEP-1880: Dynamic Extension of IP Range for Service (Alpha)
Service is an abstraction that exposes applications running on a set of Pods. Services can have cluster-wide virtual IP addresses that are allocated from a pre-defined __CIDR__ set in the __kube-apiserver__ flag. However, users may want to add, remove, or adjust the existing IP ranges allocated for services without restarting the kube-apiserver.
This feature allows cluster administrators to dynamically adjust the size of the IP range allocated for services in the cluster using the __ServiceCIDR__ object.
During the bootstrap installation, this feature creates a default __ServiceCIDR__ object named __kubernetes__ based on the value of the __--service-cluster-ip-range__ command-line parameter of __kube-apiserver__ .
To use this feature, users need to enable the __MultiCIDRServiceAllocator__ feature gate and the __networking.k8s.io/v1alpha1__ API, and create or delete new __ServiceCIDR__ objects to manage the available IP ranges for services.
Here is an example:
```shell
cat <
2. Kubernetes 1.29 Release Team
3. Kubernetes 1.29 Changelog
4. Kubernetes 1.29 Theme Discussion