- [v1.27.16](#v12716) - [Downloads for v1.27.16](#downloads-for-v12716) - [Source Code](#source-code) - [Client Binaries](#client-binaries) - [Server Binaries](#server-binaries) - [Node Binaries](#node-binaries) - [Container Images](#container-images) - [Changelog since v1.27.15](#changelog-since-v12715) - [Important Security Information](#important-security-information) - [CVE-2024-5321: Incorrect permissions on Windows containers logs](#cve-2024-5321-incorrect-permissions-on-windows-containers-logs) - [Changes by Kind](#changes-by-kind) - [Feature](#feature) - [Bug or Regression](#bug-or-regression) - [Dependencies](#dependencies) - [Added](#added) - [Changed](#changed) - [Removed](#removed) - [v1.27.15](#v12715) - [Downloads for v1.27.15](#downloads-for-v12715) - [Source Code](#source-code-1) - [Client Binaries](#client-binaries-1) - [Server Binaries](#server-binaries-1) - [Node Binaries](#node-binaries-1) - [Container Images](#container-images-1) - [Changelog since v1.27.14](#changelog-since-v12714) - [Changes by Kind](#changes-by-kind-1) - [API Change](#api-change) - [Feature](#feature-1) - [Bug or Regression](#bug-or-regression-1) - [Dependencies](#dependencies-1) - [Added](#added-1) - [Changed](#changed-1) - [Removed](#removed-1) - [v1.27.14](#v12714) - [Downloads for v1.27.14](#downloads-for-v12714) - [Source Code](#source-code-2) - [Client Binaries](#client-binaries-2) - [Server Binaries](#server-binaries-2) - [Node Binaries](#node-binaries-2) - [Container Images](#container-images-2) - [Changelog since v1.27.13](#changelog-since-v12713) - [Changes by Kind](#changes-by-kind-2) - [Bug or Regression](#bug-or-regression-2) - [Dependencies](#dependencies-2) - [Added](#added-2) - [Changed](#changed-2) - [Removed](#removed-2) - [v1.27.13](#v12713) - [Downloads for v1.27.13](#downloads-for-v12713) - [Source Code](#source-code-3) - [Client Binaries](#client-binaries-3) - [Server Binaries](#server-binaries-3) - [Node Binaries](#node-binaries-3) - [Container Images](#container-images-3) - [Changelog since v1.27.12](#changelog-since-v12712) - [Important Security Information](#important-security-information-1) - [CVE-2024-3177: Bypassing mountable secrets policy imposed by the ServiceAccount admission plugin](#cve-2024-3177-bypassing-mountable-secrets-policy-imposed-by-the-serviceaccount-admission-plugin) - [Changes by Kind](#changes-by-kind-3) - [Feature](#feature-2) - [Bug or Regression](#bug-or-regression-3) - [Dependencies](#dependencies-3) - [Added](#added-3) - [Changed](#changed-3) - [Removed](#removed-3) - [v1.27.12](#v12712) - [Downloads for v1.27.12](#downloads-for-v12712) - [Source Code](#source-code-4) - [Client Binaries](#client-binaries-4) - [Server Binaries](#server-binaries-4) - [Node Binaries](#node-binaries-4) - [Container Images](#container-images-4) - [Changelog since v1.27.11](#changelog-since-v12711) - [Changes by Kind](#changes-by-kind-4) - [Feature](#feature-3) - [Bug or Regression](#bug-or-regression-4) - [Other (Cleanup or Flake)](#other-cleanup-or-flake) - [Dependencies](#dependencies-4) - [Added](#added-4) - [Changed](#changed-4) - [Removed](#removed-4) - [v1.27.11](#v12711) - [Downloads for v1.27.11](#downloads-for-v12711) - [Source Code](#source-code-5) - [Client Binaries](#client-binaries-5) - [Server Binaries](#server-binaries-5) - [Node Binaries](#node-binaries-5) - [Container Images](#container-images-5) - [Changelog since v1.27.10](#changelog-since-v12710) - [Changes by Kind](#changes-by-kind-5) - [Feature](#feature-4) - [Bug or Regression](#bug-or-regression-5) - [Dependencies](#dependencies-5) - [Added](#added-5) - [Changed](#changed-5) - [Removed](#removed-5) - [v1.27.10](#v12710) - [Downloads for v1.27.10](#downloads-for-v12710) - [Source Code](#source-code-6) - [Client Binaries](#client-binaries-6) - [Server Binaries](#server-binaries-6) - [Node Binaries](#node-binaries-6) - [Container Images](#container-images-6) - [Changelog since v1.27.9](#changelog-since-v1279) - [Changes by Kind](#changes-by-kind-6) - [Feature](#feature-5) - [Bug or Regression](#bug-or-regression-6) - [Other (Cleanup or Flake)](#other-cleanup-or-flake-1) - [Dependencies](#dependencies-6) - [Added](#added-6) - [Changed](#changed-6) - [Removed](#removed-6) - [v1.27.9](#v1279) - [Downloads for v1.27.9](#downloads-for-v1279) - [Source Code](#source-code-7) - [Client Binaries](#client-binaries-7) - [Server Binaries](#server-binaries-7) - [Node Binaries](#node-binaries-7) - [Container Images](#container-images-7) - [Changelog since v1.27.8](#changelog-since-v1278) - [Changes by Kind](#changes-by-kind-7) - [Feature](#feature-6) - [Bug or Regression](#bug-or-regression-7) - [Other (Cleanup or Flake)](#other-cleanup-or-flake-2) - [Dependencies](#dependencies-7) - [Added](#added-7) - [Changed](#changed-7) - [Removed](#removed-7) - [v1.27.8](#v1278) - [Downloads for v1.27.8](#downloads-for-v1278) - [Source Code](#source-code-8) - [Client Binaries](#client-binaries-8) - [Server Binaries](#server-binaries-8) - [Node Binaries](#node-binaries-8) - [Container Images](#container-images-8) - [Changelog since v1.27.7](#changelog-since-v1277) - [Important Security Information](#important-security-information-2) - [CVE-2023-5528: Insufficient input sanitization in in-tree storage plugin leads to privilege escalation on Windows nodes](#cve-2023-5528-insufficient-input-sanitization-in-in-tree-storage-plugin-leads-to-privilege-escalation-on-windows-nodes) - [Changes by Kind](#changes-by-kind-8) - [API Change](#api-change-1) - [Feature](#feature-7) - [Bug or Regression](#bug-or-regression-8) - [Dependencies](#dependencies-8) - [Added](#added-8) - [Changed](#changed-8) - [Removed](#removed-8) - [v1.27.7](#v1277) - [Downloads for v1.27.7](#downloads-for-v1277) - [Source Code](#source-code-9) - [Client Binaries](#client-binaries-9) - [Server Binaries](#server-binaries-9) - [Node Binaries](#node-binaries-9) - [Container Images](#container-images-9) - [Changelog since v1.27.6](#changelog-since-v1276) - [Changes by Kind](#changes-by-kind-9) - [Feature](#feature-8) - [Failing Test](#failing-test) - [Bug or Regression](#bug-or-regression-9) - [Other (Cleanup or Flake)](#other-cleanup-or-flake-3) - [Dependencies](#dependencies-9) - [Added](#added-9) - [Changed](#changed-9) - [Removed](#removed-9) - [v1.27.6](#v1276) - [Downloads for v1.27.6](#downloads-for-v1276) - [Source Code](#source-code-10) - [Client Binaries](#client-binaries-10) - [Server Binaries](#server-binaries-10) - [Node Binaries](#node-binaries-10) - [Container Images](#container-images-10) - [Changelog since v1.27.5](#changelog-since-v1275) - [Changes by Kind](#changes-by-kind-10) - [API Change](#api-change-2) - [Feature](#feature-9) - [Bug or Regression](#bug-or-regression-10) - [Other (Cleanup or Flake)](#other-cleanup-or-flake-4) - [Dependencies](#dependencies-10) - [Added](#added-10) - [Changed](#changed-10) - [Removed](#removed-10) - [v1.27.5](#v1275) - [Downloads for v1.27.5](#downloads-for-v1275) - [Source Code](#source-code-11) - [Client Binaries](#client-binaries-11) - [Server Binaries](#server-binaries-11) - [Node Binaries](#node-binaries-11) - [Container Images](#container-images-11) - [Changelog since v1.27.4](#changelog-since-v1274) - [Important Security Information](#important-security-information-3) - [CVE-2023-3955: Insufficient input sanitization on Windows nodes leads to privilege escalation](#cve-2023-3955-insufficient-input-sanitization-on-windows-nodes-leads-to-privilege-escalation) - [CVE-2023-3676: Insufficient input sanitization on Windows nodes leads to privilege escalation](#cve-2023-3676-insufficient-input-sanitization-on-windows-nodes-leads-to-privilege-escalation) - [Changes by Kind](#changes-by-kind-11) - [API Change](#api-change-3) - [Feature](#feature-10) - [Bug or Regression](#bug-or-regression-11) - [Dependencies](#dependencies-11) - [Added](#added-11) - [Changed](#changed-11) - [Removed](#removed-11) - [v1.27.4](#v1274) - [Downloads for v1.27.4](#downloads-for-v1274) - [Source Code](#source-code-12) - [Client Binaries](#client-binaries-12) - [Server Binaries](#server-binaries-12) - [Node Binaries](#node-binaries-12) - [Container Images](#container-images-12) - [Changelog since v1.27.3](#changelog-since-v1273) - [Changes by Kind](#changes-by-kind-12) - [Feature](#feature-11) - [Bug or Regression](#bug-or-regression-12) - [Dependencies](#dependencies-12) - [Added](#added-12) - [Changed](#changed-12) - [Removed](#removed-12) - [v1.27.3](#v1273) - [Downloads for v1.27.3](#downloads-for-v1273) - [Source Code](#source-code-13) - [Client Binaries](#client-binaries-13) - [Server Binaries](#server-binaries-13) - [Node Binaries](#node-binaries-13) - [Container Images](#container-images-13) - [Changelog since v1.27.2](#changelog-since-v1272) - [Important Security Information](#important-security-information-4) - [CVE-2023-2728: Bypassing enforce mountable secrets policy imposed by the ServiceAccount admission plugin](#cve-2023-2728-bypassing-enforce-mountable-secrets-policy-imposed-by-the-serviceaccount-admission-plugin) - [Changes by Kind](#changes-by-kind-13) - [Feature](#feature-12) - [Bug or Regression](#bug-or-regression-13) - [Dependencies](#dependencies-13) - [Added](#added-13) - [Changed](#changed-13) - [Removed](#removed-13) - [v1.27.2](#v1272) - [Downloads for v1.27.2](#downloads-for-v1272) - [Source Code](#source-code-14) - [Client Binaries](#client-binaries-14) - [Server Binaries](#server-binaries-14) - [Node Binaries](#node-binaries-14) - [Container Images](#container-images-14) - [Changelog since v1.27.1](#changelog-since-v1271) - [Changes by Kind](#changes-by-kind-14) - [API Change](#api-change-4) - [Feature](#feature-13) - [Failing Test](#failing-test-1) - [Bug or Regression](#bug-or-regression-14) - [Other (Cleanup or Flake)](#other-cleanup-or-flake-5) - [Dependencies](#dependencies-14) - [Added](#added-14) - [Changed](#changed-14) - [Removed](#removed-14) - [v1.27.1](#v1271) - [Downloads for v1.27.1](#downloads-for-v1271) - [Source Code](#source-code-15) - [Client Binaries](#client-binaries-15) - [Server Binaries](#server-binaries-15) - [Node Binaries](#node-binaries-15) - [Container Images](#container-images-15) - [Changelog since v1.27.0](#changelog-since-v1270) - [Changes by Kind](#changes-by-kind-15) - [Bug or Regression](#bug-or-regression-15) - [Dependencies](#dependencies-15) - [Added](#added-15) - [Changed](#changed-15) - [Removed](#removed-15) - [v1.27.0](#v1270) - [Downloads for v1.27.0](#downloads-for-v1270) - [Source Code](#source-code-16) - [Client Binaries](#client-binaries-16) - [Server Binaries](#server-binaries-16) - [Node Binaries](#node-binaries-16) - [Container Images](#container-images-16) - [Changelog since v1.26.0](#changelog-since-v1260) - [Known Issues](#known-issues) - [The PreEnqueue extension point doesn't work for Pods going to activeQ through backoffQ](#the-preenqueue-extension-point-doesnt-work-for-pods-going-to-activeq-through-backoffq) - [Urgent Upgrade Notes](#urgent-upgrade-notes) - [(No, really, you MUST read this before you upgrade)](#no-really-you-must-read-this-before-you-upgrade) - [Changes by Kind](#changes-by-kind-16) - [Deprecation](#deprecation) - [API Change](#api-change-5) - [Feature](#feature-14) - [Documentation](#documentation) - [Failing Test](#failing-test-2) - [Bug or Regression](#bug-or-regression-16) - [Other (Cleanup or Flake)](#other-cleanup-or-flake-6) - [Dependencies](#dependencies-16) - [Added](#added-16) - [Changed](#changed-16) - [Removed](#removed-16) - [v1.27.0-rc.1](#v1270-rc1) - [Downloads for v1.27.0-rc.1](#downloads-for-v1270-rc1) - [Source Code](#source-code-17) - [Client Binaries](#client-binaries-17) - [Server Binaries](#server-binaries-17) - [Node Binaries](#node-binaries-17) - [Container Images](#container-images-17) - [Changelog since v1.27.0-rc.0](#changelog-since-v1270-rc0) - [Changes by Kind](#changes-by-kind-17) - [Feature](#feature-15) - [Bug or Regression](#bug-or-regression-17) - [Dependencies](#dependencies-17) - [Added](#added-17) - [Changed](#changed-17) - [Removed](#removed-17) - [v1.27.0-rc.0](#v1270-rc0) - [Downloads for v1.27.0-rc.0](#downloads-for-v1270-rc0) - [Source Code](#source-code-18) - [Client Binaries](#client-binaries-18) - [Server Binaries](#server-binaries-18) - [Node Binaries](#node-binaries-18) - [Container Images](#container-images-18) - [Changelog since v1.27.0-beta.0](#changelog-since-v1270-beta0) - [Changes by Kind](#changes-by-kind-18) - [API Change](#api-change-6) - [Feature](#feature-16) - [Bug or Regression](#bug-or-regression-18) - [Dependencies](#dependencies-18) - [Added](#added-18) - [Changed](#changed-18) - [Removed](#removed-18) - [v1.27.0-beta.0](#v1270-beta0) - [Downloads for v1.27.0-beta.0](#downloads-for-v1270-beta0) - [Source Code](#source-code-19) - [Client Binaries](#client-binaries-19) - [Server Binaries](#server-binaries-19) - [Node Binaries](#node-binaries-19) - [Container Images](#container-images-19) - [Changelog since v1.27.0-alpha.3](#changelog-since-v1270-alpha3) - [Urgent Upgrade Notes](#urgent-upgrade-notes-1) - [(No, really, you MUST read this before you upgrade)](#no-really-you-must-read-this-before-you-upgrade-1) - [Changes by Kind](#changes-by-kind-19) - [Deprecation](#deprecation-1) - [API Change](#api-change-7) - [Feature](#feature-17) - [Documentation](#documentation-1) - [Failing Test](#failing-test-3) - [Bug or Regression](#bug-or-regression-19) - [Other (Cleanup or Flake)](#other-cleanup-or-flake-7) - [Dependencies](#dependencies-19) - [Added](#added-19) - [Changed](#changed-19) - [Removed](#removed-19) - [v1.27.0-alpha.3](#v1270-alpha3) - [Downloads for v1.27.0-alpha.3](#downloads-for-v1270-alpha3) - [Source Code](#source-code-20) - [Client Binaries](#client-binaries-20) - [Server Binaries](#server-binaries-20) - [Node Binaries](#node-binaries-20) - [Container Images](#container-images-20) - [Changelog since v1.27.0-alpha.2](#changelog-since-v1270-alpha2) - [Changes by Kind](#changes-by-kind-20) - [Deprecation](#deprecation-2) - [API Change](#api-change-8) - [Feature](#feature-18) - [Documentation](#documentation-2) - [Failing Test](#failing-test-4) - [Bug or Regression](#bug-or-regression-20) - [Other (Cleanup or Flake)](#other-cleanup-or-flake-8) - [Dependencies](#dependencies-20) - [Added](#added-20) - [Changed](#changed-20) - [Removed](#removed-20) - [v1.27.0-alpha.2](#v1270-alpha2) - [Downloads for v1.27.0-alpha.2](#downloads-for-v1270-alpha2) - [Source Code](#source-code-21) - [Client Binaries](#client-binaries-21) - [Server Binaries](#server-binaries-21) - [Node Binaries](#node-binaries-21) - [Container Images](#container-images-21) - [Changelog since v1.27.0-alpha.1](#changelog-since-v1270-alpha1) - [Changes by Kind](#changes-by-kind-21) - [API Change](#api-change-9) - [Feature](#feature-19) - [Bug or Regression](#bug-or-regression-21) - [Other (Cleanup or Flake)](#other-cleanup-or-flake-9) - [Dependencies](#dependencies-21) - [Added](#added-21) - [Changed](#changed-21) - [Removed](#removed-21) - [v1.27.0-alpha.1](#v1270-alpha1) - [Downloads for v1.27.0-alpha.1](#downloads-for-v1270-alpha1) - [Source Code](#source-code-22) - [Client Binaries](#client-binaries-22) - [Server Binaries](#server-binaries-22) - [Node Binaries](#node-binaries-22) - [Container Images](#container-images-22) - [Changelog since v1.26.0](#changelog-since-v1260-1) - [Changes by Kind](#changes-by-kind-22) - [Deprecation](#deprecation-3) - [API Change](#api-change-10) - [Feature](#feature-20) - [Documentation](#documentation-3) - [Failing Test](#failing-test-5) - [Bug or Regression](#bug-or-regression-22) - [Other (Cleanup or Flake)](#other-cleanup-or-flake-10) - [Dependencies](#dependencies-22) - [Added](#added-22) - [Changed](#changed-22) - [Removed](#removed-22) # v1.27.16 ## Downloads for v1.27.16 ### Source Code filename | sha512 hash -------- | ----------- [kubernetes.tar.gz](https://dl.k8s.io/v1.27.16/kubernetes.tar.gz) | 0939ac0ac9badaedb600b3f0900c51da29236485f03b7bbb073d62cafc6e672de9218c09ed0089566241dfe273188ef2e11206a93994ff824d213d83f43c63fc [kubernetes-src.tar.gz](https://dl.k8s.io/v1.27.16/kubernetes-src.tar.gz) | fba641f745b5eef6f17e16501025959e570ae88912d79e2e08b4c79d18475430b879f98947d516871407feb297d5dac9b2fbc4ca565d3770c600d9550f3cffac ### Client Binaries filename | sha512 hash -------- | ----------- [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.27.16/kubernetes-client-darwin-amd64.tar.gz) | 8c4a574dfa1140600c1b74b071aac5f893a609917652793e84cc344aa5b1fc8b9578109953203e6f968183f5b798144efa33cdc36c4a5df58489c8553f508e2f [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.27.16/kubernetes-client-darwin-arm64.tar.gz) | 6d175b2a884d09ce5a4b645c71200a92df4b2c8ef1a3890f36a672912d1c76e4ed9b159d538fb37892665d44d2314c537bd6ab98b2571330a3d47b516ddf7243 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.27.16/kubernetes-client-linux-386.tar.gz) | 8c00094792edccae78abeb697e206a2a6e6d3d335a9d91ef6bf9dfc3b122b36ef9a781420eb08f835eb78b7c3d3768a5eb1426a041375ff4570a34313126c87b [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.16/kubernetes-client-linux-amd64.tar.gz) | 19a757de6428ea418e55d8520b3cfaf4ce8a432dfb923714cfc1411d5f8d5d3b3b1d77819571ddc2c2c6c5e7a37e54bcfe8a062dd86e45c263a01f1cc2f39290 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.27.16/kubernetes-client-linux-arm.tar.gz) | de7ae9f784b5aeb68fd6bc85388c2f026a9e369c8ce4e880a1bfd31682e7988d87c9ead970a05100b708dec277de32c358a362b6615f0e17f004e65cd70f947c [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.16/kubernetes-client-linux-arm64.tar.gz) | 3bb8cceb0e47744c7ddb143f3def61c442ac6b3408f0e2d68768a28604eb76d8d03cced58d8c36d2d58d52e2be9008737015ddeeb823fa2ea492b003533b1b3b [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.16/kubernetes-client-linux-ppc64le.tar.gz) | 779db2eb3b9e740c5357c240b1a2b8ca156a6fc3a27e7dc4ebdf58c18a4b1e919622e4a7be708489343d10675fc2d64a49ec5cb282f7e09541dbf7f7dbb6218f [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.16/kubernetes-client-linux-s390x.tar.gz) | 0edcc792305e7f4fc63aead6018866c2f908c583080c4d4065476384bbb71ead01c08a9b3502e506bca3d201fa4ee2f1b8005c1e207acf798c58a698b4484cda [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.27.16/kubernetes-client-windows-386.tar.gz) | 2250c887fa139cb7411618c210a56f2e65ccf23c516e6d073779547890569577dce3f3bf16b42396dd00a7fa7ecb08c588bbbafc76750856d56da8a903395802 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.16/kubernetes-client-windows-amd64.tar.gz) | 9291fb5fb48e877d6d03937defb0c52cbe2af2fb762e30475a0849e89f0dc46318ed21a598b3d119f0cfb639ec38b8a7403b82263f7d0d28b634ac3921325c1a [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.27.16/kubernetes-client-windows-arm64.tar.gz) | 7f1e08bc17590bfece171a64aee3f34c7f519e7d11acb8b0afc1578bf2168f644ff8b9e57b0b0263f2387af238f3558585b1842f4e1880dc50ce2d7987a52345 ### Server Binaries filename | sha512 hash -------- | ----------- [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.16/kubernetes-server-linux-amd64.tar.gz) | cbcb0591a3a84f98cf2c7abe11c5531e7f79850714abdd957be52049afed14031d0ab9c584c2e54775bceedf8833a500148d5d01335474b33cbb8b30b7ce51b3 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.16/kubernetes-server-linux-arm64.tar.gz) | a608c476d32cbab392419095cc24dff36ffe8b75f6d3150b3a7e4351f3b38c67e659692941ce79a0428febab2e7dc0a255af33b4ac0f82465456ea2bf66c7240 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.16/kubernetes-server-linux-ppc64le.tar.gz) | 027ec52d46116616ee4a89584add4d696f0384ef705a9b79b690144fa47109f6dcc9d89aa48ebb64f9aaee9def5174eb7943d879b3dbc643ee64eda9ad0c5772 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.16/kubernetes-server-linux-s390x.tar.gz) | bae6ab6f82c35068f5835866d02c873a1efcb5824022b174d1c19b55cbf10d5b68c5028da0d7dd34aa2308ed6de6c9c78a84d4c31442431d17ce15f0e40ab8e0 ### Node Binaries filename | sha512 hash -------- | ----------- [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.16/kubernetes-node-linux-amd64.tar.gz) | 55532060b056be6c499b8d8cb5884827ae9b25d953d03611880264d96c164f3648c662e5b2b653d081242eb407811ecd873637c38ab0678654339a60774b1016 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.16/kubernetes-node-linux-arm64.tar.gz) | c8dea73d480b85ad25f34166b89a095f53cdbf41b5ef83570710421ed444ee9f6a6507d0dbea52a58ffb70b50840cbd9304b149434b0abdf4cbca85b04b3ddde [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.16/kubernetes-node-linux-ppc64le.tar.gz) | c7c4f7afa9aa039f9b3d3cf1c290e5f1765af919d0c87a69cf3416fd2051e3dc5d98a961cce3e15e56dc37094731f335d283dc48e3d371667dd7eaddaf5b2755 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.16/kubernetes-node-linux-s390x.tar.gz) | 7aeeddd496b0b5ef4437029a03eb20ee8f3f95fe9cac082ef8ae2d377cfa049d3c0dee702c9b2a646ed79e3ed1ac4f5481fa1f3ff8944b425fef6bf3f1ecdd2f [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.16/kubernetes-node-windows-amd64.tar.gz) | 8aa07111a478ee24d79001d93e4d1170b409883824077f9b6d95d49ab584595fdc285d05a71c2537668dcf6639c51ca9f35cc39e6d17a9937ca4862c0184e7ad ### Container Images All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures ---- | ------------- [registry.k8s.io/conformance:v1.27.16](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-s390x) [registry.k8s.io/kube-apiserver:v1.27.16](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-s390x) [registry.k8s.io/kube-controller-manager:v1.27.16](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-s390x) [registry.k8s.io/kube-proxy:v1.27.16](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-s390x) [registry.k8s.io/kube-scheduler:v1.27.16](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-s390x) ## Changelog since v1.27.15 ## Important Security Information This release contains changes that address the following vulnerabilities: ### CVE-2024-5321: Incorrect permissions on Windows containers logs A security issue was discovered in Kubernetes clusters with Windows nodes where BUILTIN\Users may be able to read container logs and NT AUTHORITY\Authenticated Users may be able to modify container logs. **Affected Versions**: - kubelet <= 1.27.15 - kubelet <= 1.28.11 - kubelet <= 1.29.6 - kubelet <= 1.30.2 **Fixed Versions**: - kubelet 1.27.16 - kubelet 1.28.12 - kubelet 1.29.7 - kubelet 1.30.3 This vulnerability was reported by Paulo Gomes @pjbgf from SUSE. **CVSS Rating:** Medium (6.1) [CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:L/A:N](https://www.first.org/cvss/calculator/3.1#CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:L/A:N) ## Changes by Kind ### Feature - Kubernetes is now built with go 1.22.4 ([#125701](https://github.com/kubernetes/kubernetes/pull/125701), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing] - Kubernetes is now built with go 1.22.5 ([#125898](https://github.com/kubernetes/kubernetes/pull/125898), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing] ### Bug or Regression - NONE ([#126126](https://github.com/kubernetes/kubernetes/pull/126126), [@cji](https://github.com/cji)) [SIG Node and Windows] ## Dependencies ### Added _Nothing has changed._ ### Changed _Nothing has changed._ ### Removed _Nothing has changed._ # v1.27.15 ## Downloads for v1.27.15 ### Source Code filename | sha512 hash -------- | ----------- [kubernetes.tar.gz](https://dl.k8s.io/v1.27.15/kubernetes.tar.gz) | e9a34287a0a8c6ccc5fb2e386dc850f7358f5ee5e2ec33b635e7ee92895a68e3693e22db22cf939a52b2bf85e17ede8c43a92eefd33823d80dc2f4a9b64e3eb6 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.27.15/kubernetes-src.tar.gz) | 6b5b5182e88c613a5e8452aa0d2446a3772e49533b471bfebf85edb5ed1f7f63e87a1e211d969706aedb1a1a0677ffbd8a37cd6df8a8df8fb3d1bc912fb37b64 ### Client Binaries filename | sha512 hash -------- | ----------- [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.27.15/kubernetes-client-darwin-amd64.tar.gz) | a518c1c092a69a69f722009856a80a83e4ac0229ff6128607a76919f91f7d7d0578190198e54c79cdb4d14ab94b1ce2e56de4c50c0ee4e4cd894380f26c6f423 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.27.15/kubernetes-client-darwin-arm64.tar.gz) | f64343b1ba63a44c3f8e5f9b1b18e818340fe9d34e14bbecabcb5a1d68df231ec457039edc3d58ce57eb571f33d146fbf06c215a24d963221ea2a21b8b4898d0 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.27.15/kubernetes-client-linux-386.tar.gz) | 673c80c2094f9ec7d9945c64ac3882f2017a1765a367afe0286a599b0f7249cc575cd9a7f3b5b5406cb453c55d341d8e64583cb5c82f96e7c5af74433c4ef896 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.15/kubernetes-client-linux-amd64.tar.gz) | 783cd10bf9bee1150b03aedf3295aef4b86ffe567acb1fb9c274b193dc52fa909a8da45aaca1247c24867772a0cac15599e72b4aecf9c4168fe32a980a6eb9cd [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.27.15/kubernetes-client-linux-arm.tar.gz) | 538b0127543336c422742d58b03d0aa0afb659da6c4f445c4ad1ef8c7ec5514fadb3c54aadaf208fd0ab437751adcae9adb4bc559e009548dd6b16e202863604 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.15/kubernetes-client-linux-arm64.tar.gz) | cf0815b1a3c6b4556c5b845d84fb1736f9a9c95e89fdd55446b71062011f5b35603061111708518272a651e4e4df5c5e8391369a68fe00946ee027b57f0c8066 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.15/kubernetes-client-linux-ppc64le.tar.gz) | c390240379c03a31a45261ba86078ef9c479991a6473277d4e89309a7b663ed88913e9f120f23d57deaf82836a2e1935182f50da9242670301bf60ea5ed15733 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.15/kubernetes-client-linux-s390x.tar.gz) | 148e14eb6a5a231ada7a5ae66fad38d9738adb71c6eeb7115b6c3ea5a3f267418ef82695b81e3b075d8f28c4b31e83597bea93bc4d7039a3894c609492ed04e4 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.27.15/kubernetes-client-windows-386.tar.gz) | d360ea469b1d265c9029a0037d566100ec4d143d60f6bb47b43035cdb65b0c9223b4a222682434fbd89dd5ebba107f2f87a59421f68aac49122e83a159798203 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.15/kubernetes-client-windows-amd64.tar.gz) | 2897920f4b7cabb6d45c36b62706084616cadad4900b19032179575b5ea013f457c6021c6e0569217c52567ea854ddf0149c5853e15e91c941073f224f20f551 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.27.15/kubernetes-client-windows-arm64.tar.gz) | 87ba5b55fef77dd3876aa4d68d26615177628645998597e8a96237778ebaa5a92aab500cfbc7be75fe3eddf21439c896625232078ab20faf4a407f84c753e19a ### Server Binaries filename | sha512 hash -------- | ----------- [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.15/kubernetes-server-linux-amd64.tar.gz) | 6349711101fc28795fa98160a2542bc3fd670d87cf360caf851e2fe266ad10579977e360a7e8525fd6e33afef1026b67ca58701801c11c5a9b0a0241913a256a [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.15/kubernetes-server-linux-arm64.tar.gz) | 467ae5eef630329b2c48b87892bf8bbfb6208c2891866d1e4b233869969deb985860c9befcef9427f5f8078947f8d27be8764d29979ce0ff29ace499927b14a7 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.15/kubernetes-server-linux-ppc64le.tar.gz) | ebf96d09281e5a73320340bca973098500b1c5b66f5cd57588b394ef22e0db85116dbf68e20db23aa354600b8a7472e9c54415b93ebb6d53ec4c13c6f6a7481d [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.15/kubernetes-server-linux-s390x.tar.gz) | 3373e5d15df34ac56c4564ebc44cd13c9f77c269db3ef3b2386c2a67cdff94bc2b344e3089b0f562f79c0a58180c43a7e1f03fec0d4790e4edbe37ba4056f4b6 ### Node Binaries filename | sha512 hash -------- | ----------- [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.15/kubernetes-node-linux-amd64.tar.gz) | 3561910f6e63cfa872865716867a1e6a6e3e568491e51bd20516c6210b752584a3316eac1941c980ead53e1566a6acb6f3078025da8c87e42a2e30f80504e525 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.15/kubernetes-node-linux-arm64.tar.gz) | bd07be66d1f21b516be8d8862caa81f5b5bdf6c12468a48fba875178697dd8f94188f34cf3c7791141e1fc64725deae11967549a1f71baa5ae0a54b03230cb3f [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.15/kubernetes-node-linux-ppc64le.tar.gz) | 6eedb5de2ff2559ffea5e19d9b433a967e336438a4f0e0ef619c8e8c0d494aeb27d8c677950d78dcc2dcce8704ff409061666e7e12809cadfb71bef01be2efbe [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.15/kubernetes-node-linux-s390x.tar.gz) | 48d77a23ae86131b8dd7c5fde35145fa20516d74939ae7332d69151eca7ff05710079e44780f3d61577a185ca69040da608b96d979524b86a004c564e5255459 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.15/kubernetes-node-windows-amd64.tar.gz) | c8ac579051abf0ffc6a61f71ee8a3ace8521f3068d1362c1134c0e3c8960608ae00b418e81bf875b2c92317a5faad6c9434d2b1a89269a03bad17eafa3eef7fd ### Container Images All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures ---- | ------------- [registry.k8s.io/conformance:v1.27.15](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-s390x) [registry.k8s.io/kube-apiserver:v1.27.15](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-s390x) [registry.k8s.io/kube-controller-manager:v1.27.15](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-s390x) [registry.k8s.io/kube-proxy:v1.27.15](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-s390x) [registry.k8s.io/kube-scheduler:v1.27.15](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-s390x) ## Changelog since v1.27.14 ## Changes by Kind ### API Change - Improved scheduling performance when many nodes, and prefilter returns 1-2 nodes (e.g. daemonset) For developers of out-of-tree PostFilter plugins, note that the semantics of NodeToStatusMap are changing: A node with an absent value in the NodeToStatusMap should be interpreted as having an UnschedulableAndUnresolvable status ([#125309](https://github.com/kubernetes/kubernetes/pull/125309), [@gabesaba](https://github.com/gabesaba)) [SIG Scheduling] ### Feature - Kubernetes is now built with go 1.21.10 ([#124832](https://github.com/kubernetes/kubernetes/pull/124832), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing] - Kubernetes is now built with go 1.21.11 ([#125369](https://github.com/kubernetes/kubernetes/pull/125369), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing] ### Bug or Regression - Improved scheduling latency when many gated pods ([#124866](https://github.com/kubernetes/kubernetes/pull/124866), [@gabesaba](https://github.com/gabesaba)) [SIG Scheduling and Testing] - Kube-scheduler: fixes a 1.27.14 regression that can lead to a scheduler crash when processing pods with affinity that doesn't match a real/valid node ([#125043](https://github.com/kubernetes/kubernetes/pull/125043), [@AxeZhan](https://github.com/AxeZhan)) [SIG Scheduling and Testing] ## Dependencies ### Added _Nothing has changed._ ### Changed _Nothing has changed._ ### Removed _Nothing has changed._ # v1.27.14 ## Downloads for v1.27.14 ### Source Code filename | sha512 hash -------- | ----------- [kubernetes.tar.gz](https://dl.k8s.io/v1.27.14/kubernetes.tar.gz) | d7f966e5b73963337323a6d95b28d198c2115bb60eb3b7a5eae749ae3875523694d91503e4031705a672682b06129c324f97ed9306888adb6a2a0ff67047c69f [kubernetes-src.tar.gz](https://dl.k8s.io/v1.27.14/kubernetes-src.tar.gz) | 28646ba71b4ddad212624fb7806bb3d667f367de9c908462fa581d6a48a5d041bb5ed34db5a4a529617150b7b1e5f242f1076ea0f3ce9aaf12aa11a40e828d1d ### Client Binaries filename | sha512 hash -------- | ----------- [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.27.14/kubernetes-client-darwin-amd64.tar.gz) | fc48ef433cad64c2ed6ee7c74d55577327da461cac8381e072030b424be1249efd72f91b1d5558b6c8b27949fdd3ea14f721af497e47c327515d88de75230253 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.27.14/kubernetes-client-darwin-arm64.tar.gz) | 5b59eca28d9f8fcc3039d549db91713e724c21eebbc81e8ea15a424d61e8638d918d02ab67bf7e279fb4e9980f46bdd0b8ddd1dc34c166d5c00b141e39664cab [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.27.14/kubernetes-client-linux-386.tar.gz) | 97b82d3671a07a754fd1ce82eeec0b1e81e8ff76cb73c4b79f24cdaa73d63d0039bb979773147bd38bf19bbe8802692a91e27d33c1a7544cc244bb5ff2e7493c [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.14/kubernetes-client-linux-amd64.tar.gz) | 9f4715aad258134b1ec3eee0e7245aa40dc8846e74cba5f68c9cb4496ecfebbb342c8f20e9552516a8f4903292cbd92f365c04886e374420f1153065ac8b93b4 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.27.14/kubernetes-client-linux-arm.tar.gz) | 4b67e6236108b5428a5158c699e5c1dc73e5a467c427e235965ee0ced13e22d22d96e7f242ff333cdc9aa3e137044f20cecf331fd49fd0801359d23bf7be5e88 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.14/kubernetes-client-linux-arm64.tar.gz) | cc80e23ec1d332da2264055078295ad9bd4c5a59ea43af57865d82c8e27ae5766837bdcab896e9fc9b6ff8ec82cf0840ed30d58440f99d5ec9de9eb3ef52d449 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.14/kubernetes-client-linux-ppc64le.tar.gz) | b14b9141fb1fc3ee0f7fd65628322353a092633cadefb46f31e8d4ae720422b47134d3476f5a2ae42f0ab08e4d7e3246316dea6c2372cb118d6fef016ca5af40 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.14/kubernetes-client-linux-s390x.tar.gz) | 80443a6837c25748b770aa52412e293c04cacf0ae58d7986ccce9d318c095828ef3d0f9c6b1267b85c8c0ea636159d2df711ef2b2a9d9dc0886ff8244269aef1 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.27.14/kubernetes-client-windows-386.tar.gz) | b4320130c044505ed6cf0544cf8d6b243a9b4a9387f05341bf1e8b8de889b810fcc5578f82d20e512c9cc1e048d2015e512ec3a90011a90cacb14910d7ad7cf4 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.14/kubernetes-client-windows-amd64.tar.gz) | 04c1ec7e4d98d253aad38ec7f2c6a15d40ef5077a1a7fe0f388cec3ba4ea13074cbe61ab0894b0a498cd223db2e8c71bc4e4e53df5f6e1cebf93f43cf2810663 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.27.14/kubernetes-client-windows-arm64.tar.gz) | 2b156588703b2aa5ef34186a1fb8ee866af28daf157b2450bfb94102ac89dc729dcc6f8f987229dcff0547aadaeff22073acb2ee4f93abb44433059b41a40ae6 ### Server Binaries filename | sha512 hash -------- | ----------- [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.14/kubernetes-server-linux-amd64.tar.gz) | 771e4d275897979d14798ca9c838fea0c0ba585be03e0b42b0fd7e3792366a8ea2af10a6fee5c8786d4cd7353b3199de0d64120988dc5da03d7e5f2ed8603173 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.14/kubernetes-server-linux-arm64.tar.gz) | 70cf9e3201e592f9aab6b47add3a113e95af2c20690ff46669dbcaf7ca0a08defb569b50435367e61b10f70dac1f3897607a99f65aedc39da4b15471f058277f [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.14/kubernetes-server-linux-ppc64le.tar.gz) | cef3e6859a394cfd6de1ebf19c3fd4c9ad3c71c28c1d21d1f591f79c07018d28e0194662dcf1d2a4b876838bb81bf4184706923a2a0b5d410442c2b2d97dc735 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.14/kubernetes-server-linux-s390x.tar.gz) | 35b663347df39669773cf1df25f3269d448b72fcc8242b4d89aca84f32005bd6bb0b98368679ba79295c43f3d1075c2c4ebeb3b6a02217ffc78f88225c478d62 ### Node Binaries filename | sha512 hash -------- | ----------- [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.14/kubernetes-node-linux-amd64.tar.gz) | 425fb236673983a72945997a209949151b5654a3fe21c2dcd20fe8bfa1fa31cf29f500ba094d308de2b0bc6ce8dd7a01369a4a0878959e743e2129ff6c7893c1 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.14/kubernetes-node-linux-arm64.tar.gz) | 4d1e3419dc375fd91a0d25d5f875a8013a33d517ffb3cb37f6dbfa57902fbf4314025ed3f1d687e6d6bc13359990e7cf6a606873bcc1897fb65adcbbc05c1c7a [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.14/kubernetes-node-linux-ppc64le.tar.gz) | 3b360f8da45eef36100945476a895598a3b9375f32443c9674569807f3b85d78528354d11643b8cad89ee51caed063d513ce19a34967058da12df0ca2958ce3a [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.14/kubernetes-node-linux-s390x.tar.gz) | a7f45b60c9dc3b5941e7134ae1ae56972f5c630682811bce3b2e8273d3b85cb1ac1e891045b84ad448af01e67ee7f9695f2cea017593c93786971b8929d5a2f6 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.14/kubernetes-node-windows-amd64.tar.gz) | 96833cebd6a50fba194d6e1fcdd1cc6ca77dc325ba5a80e135fd8332f44bcb9e61c97951765980f56f887e92ca529bd8fd53ce6098b23a777429807f49a0556d ### Container Images All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures ---- | ------------- [registry.k8s.io/conformance:v1.27.14](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-s390x) [registry.k8s.io/kube-apiserver:v1.27.14](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-s390x) [registry.k8s.io/kube-controller-manager:v1.27.14](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-s390x) [registry.k8s.io/kube-proxy:v1.27.14](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-s390x) [registry.k8s.io/kube-scheduler:v1.27.14](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-s390x) ## Changelog since v1.27.13 ## Changes by Kind ### Bug or Regression - Fixed PersistentolumeLabel providing wrong topology labels to Azure Disk PersistentVolumes when the external Azure cloud provider is used. ([#124528](https://github.com/kubernetes/kubernetes/pull/124528), [@jsafrane](https://github.com/jsafrane)) [SIG Cloud Provider] - Fixed a bug that a pod may remain unscheduled if any PreFilter plugin returns nodes that do not exist. ([#124765](https://github.com/kubernetes/kubernetes/pull/124765), [@chengjoey](https://github.com/chengjoey)) [SIG Scheduling and Testing] ## Dependencies ### Added _Nothing has changed._ ### Changed _Nothing has changed._ ### Removed _Nothing has changed._ # v1.27.13 ## Downloads for v1.27.13 ### Source Code filename | sha512 hash -------- | ----------- [kubernetes.tar.gz](https://dl.k8s.io/v1.27.13/kubernetes.tar.gz) | d3fe54bd77a722b0d46b89aba321a5ab16c93f4f1b32646643a9d780f466149fd35d205cd18838a5a46abfb0f6ff29185e0c93d627a9b4dfe3f2ca3710c67f75 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.27.13/kubernetes-src.tar.gz) | 65dfd50d6a1cdd5cbcbe1cf346949d1e0e99fbe747a7f95e66db52bf3a318afcfb514232e856f4fccaef4935d85fc71cd27dc77544711189c059e45587aa6067 ### Client Binaries filename | sha512 hash -------- | ----------- [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.27.13/kubernetes-client-darwin-amd64.tar.gz) | 7ddf02f44ee8d119b2876f1864da1cde5e1c3b2b10dc14a535b9b30799e08064bfb7285ae9df55a1a4dec73f83fe2b0631cc17eb3b607152cd6ab6879dd5f987 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.27.13/kubernetes-client-darwin-arm64.tar.gz) | 3a0fd8050e3cfe36bcbd905b199139589a7dcacb5be4f218b5e0f2bfef692c082ee007b8415e91cbb6566648f4dd5ff7566d24834764a75e879b7e6a64bcd658 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.27.13/kubernetes-client-linux-386.tar.gz) | e95d44a3859d86fadf35a0727c628e0c12aea3cc4d98a0ca06d9aaa68800155dc355eb1fdc54711be7a630c98263e155f82f1557196895f307ee85b45db6f5fb [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.13/kubernetes-client-linux-amd64.tar.gz) | 6480d87386b766f379f6cd2356e87be089c8e1656e71ee8e4cb6878b71b724c0b8461ba40b2b416f6e43b9525ccffaa1df7d528b891fe57da97bfbde7c51352c [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.27.13/kubernetes-client-linux-arm.tar.gz) | b13502a952b8bbc481ea08657ec86c2520e2e9b563017fc706e902f437e14532bd71f5805d611dc5c013754566d555230a561692e86c0f4b8171db0bcc80422c [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.13/kubernetes-client-linux-arm64.tar.gz) | 38b59a701b21d2ce4473579556354d1beeec71d69e31a0138a4c036af163281aed3677e641827226b9ee6aedf56c0f96bb7c7f0dcce23b7242be138282ca42eb [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.13/kubernetes-client-linux-ppc64le.tar.gz) | 37b34738df126e7812b4609fed507bc01914d40acba0db5e9b23082a000df37eb9d3e867a6dc79c48057ecfd0c5d3c25259fcd3acc7e1d9ea98be9dfecaf8ccb [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.13/kubernetes-client-linux-s390x.tar.gz) | 20e62246595d9605e8611cd6972da3be57fffa14da62c502f3cb3267266eb7c891cb4af66677a872bf638a27a7c822fa4628a428e39f08a9e3c9ae00277f4e73 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.27.13/kubernetes-client-windows-386.tar.gz) | 9ba18d9a959f17c2ff5caf99f31d3e0e0e84927d7c1be408518d99b7bc669e1393ff3b5f3c1368f27e4dae303ad70c2dc79bdcea42f7853170398f885d77bbe9 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.13/kubernetes-client-windows-amd64.tar.gz) | f009f13428119ce9239ab1444bcd7c20796382c108c2e44adfc7cdb8874cf7acaf65bf3ce723dd2b395154196b1bcc1fec79625409226b31b563aae86741903d [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.27.13/kubernetes-client-windows-arm64.tar.gz) | 12a1716be5aa0bbd863ac330ae0beb59a0aea5f74be704bfd2e2849b9ecdb5ce10e2a86e71558ee3a40d839993ae21692f9cf6e8350c2969f528b6406e8f243c ### Server Binaries filename | sha512 hash -------- | ----------- [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.13/kubernetes-server-linux-amd64.tar.gz) | 6d5a3313e5e336c3877ca4202a94c4d8320f53d3c1fe29d81aff2ca8b2d3ae3cce035c84b5322821fa78f4e2cdc51321dc33b95f9baf00942d6d4d56cfd19e69 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.13/kubernetes-server-linux-arm64.tar.gz) | a14658d0147077d2c0a63d84ebb321666537e6be55e9dccf86eb391e10a65d917b02ea5b3ccf7b8ee14af32e4f7a9b71dd631b691cc18f53ceda19e61e6b2e7b [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.13/kubernetes-server-linux-ppc64le.tar.gz) | e51c928c658457b32ff1c7cbf716fdf9519342f0da54dd32f198c4c0edbf5c62b1379c06e872ec07780672b2e1b203d84e1cd103f40e3de2a30ae4dc25c060ae [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.13/kubernetes-server-linux-s390x.tar.gz) | fa89cb300e4b9ac75c0439b32fc1a4f77904b3f2b47184530345d00d45b2cfe8555244f62be562c47aef50c6c9132e1c7f109de198ca07d6d4ca75801a87659e ### Node Binaries filename | sha512 hash -------- | ----------- [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.13/kubernetes-node-linux-amd64.tar.gz) | b6fc891bda8f29f214723cf9ccd08ac282e4aed0ba76981008be40088a7d8132ed74d8ddd4aa23cd09e438a9801b51aebc582d52f06526114b19aa164c9725f1 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.13/kubernetes-node-linux-arm64.tar.gz) | 751ac5bf9d06b3ca521a3c17b5f4265c76dade6759af543ca57e5395c63baf6db4a91d67df09b604aaec128b312e92a7faff43d8a7ae3218df5481201445a7ae [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.13/kubernetes-node-linux-ppc64le.tar.gz) | 5ef40c6c887c8a8a91612bbd751d4e7c63582c74e3475c9444ad36415577653ee01f4a72a987f20e0faee5dc35bc2c91b4de737caf0e10336b3828cecaa18378 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.13/kubernetes-node-linux-s390x.tar.gz) | d418dee6b26d781d13b6e3d6b7baaf6cda994474042651385e56891c4823421cb4a868f7f2d4824afcdbfaa9e314ef40791db6afa5354163180083475406dc2c [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.13/kubernetes-node-windows-amd64.tar.gz) | a87a4f1e64026ba8dab92f64b6fdd02bba095be6098ab79e647082b2312d9a4442e9926f09e8a07f12d8003b712a15fbe109626ae2e796f961cf1c42b439925c ### Container Images All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures ---- | ------------- [registry.k8s.io/conformance:v1.27.13](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-s390x) [registry.k8s.io/kube-apiserver:v1.27.13](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-s390x) [registry.k8s.io/kube-controller-manager:v1.27.13](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-s390x) [registry.k8s.io/kube-proxy:v1.27.13](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-s390x) [registry.k8s.io/kube-scheduler:v1.27.13](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-s390x) ## Changelog since v1.27.12 ## Important Security Information This release contains changes that address the following vulnerabilities: ### CVE-2024-3177: Bypassing mountable secrets policy imposed by the ServiceAccount admission plugin A security issue was discovered in Kubernetes where users may be able to launch containers that bypass the mountable secrets policy enforced by the ServiceAccount admission plugin when using containers, init containers, and ephemeral containers with the envFrom field populated. **Affected Versions**: - kube-apiserver v1.29.0 - v1.29.3 - kube-apiserver v1.28.0 - v1.28.8 - kube-apiserver <= v1.27.12 **Fixed Versions**: - kube-apiserver v1.29.4 - kube-apiserver v1.28.9 - kube-apiserver v1.27.13 This vulnerability was reported by tha3e1vl. **CVSS Rating:** Low (2.7) [CVSS:3.1/AV:N/AC:L/PR:H/UI:N/S:U/C:L/I:N/A:N](https://www.first.org/cvss/calculator/3.1#CVSS:3.1/AV:N/AC:L/PR:H/UI:N/S:U/C:L/I:N/A:N) ## Changes by Kind ### Feature - Kubernetes is now built with go 1.21.9 ([#124199](https://github.com/kubernetes/kubernetes/pull/124199), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing] ### Bug or Regression - Fix pod restart after node reboot when NewVolumeManagerReconstruction feature gate is enabled and SELinuxMountReadWriteOncePod disabled ([#124142](https://github.com/kubernetes/kubernetes/pull/124142), [@bertinatto](https://github.com/bertinatto)) [SIG Node] - Golang.org/x/net is bumped to v0.23.0 to address CVE-2023-45288 ([#124178](https://github.com/kubernetes/kubernetes/pull/124178), [@MadhavJivrajani](https://github.com/MadhavJivrajani)) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node and Storage] - Kube-apiserver: fixes a 1.27+ regression in watch stability by serving watch requests without a resourceVersion from the watch cache by default, as in <1.27 (disabling the change in #115096 by default). This mitigates the impact of an etcd watch bug (https://github.com/etcd-io/etcd/pull/17555). If the 1.27 change in #115096 to serve these requests from underlying storage is still desired despite the impact on watch stability, it can be re-enabled with a `WatchFromStorageWithoutResourceVersion` feature gate. ([#124007](https://github.com/kubernetes/kubernetes/pull/124007), [@serathius](https://github.com/serathius)) [SIG API Machinery] - Kubeadm: fix panic in the command "kubeadm certs check-expiration" when "/etc/kubernetes/pki" exists but cannot be read. ([#124124](https://github.com/kubernetes/kubernetes/pull/124124), [@carlory](https://github.com/carlory)) [SIG Cluster Lifecycle] - NONE ([#124325](https://github.com/kubernetes/kubernetes/pull/124325), [@ritazh](https://github.com/ritazh)) [SIG Auth] ## Dependencies ### Added _Nothing has changed._ ### Changed - golang.org/x/crypto: v0.16.0 → v0.21.0 - golang.org/x/net: v0.19.0 → v0.23.0 - golang.org/x/sys: v0.15.0 → v0.18.0 - golang.org/x/term: v0.15.0 → v0.18.0 ### Removed _Nothing has changed._ # v1.27.12 ## Downloads for v1.27.12 ### Source Code filename | sha512 hash -------- | ----------- [kubernetes.tar.gz](https://dl.k8s.io/v1.27.12/kubernetes.tar.gz) | 2d36d018d6626b843e68204792d41d62fd839c0356179ef786c7f5b8cc00564172f9749bdd73b52778e587ad1448f534e9f418fd13013f1dda5974b0787fe5e6 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.27.12/kubernetes-src.tar.gz) | f177e1a45a9d273955d9dffeb801f2e964bb1f88bd1e494bf0cf5c9bbf5112480338ec27d8b5eab407cd87ac461f6df499b1a1ba5f94fae3a4121318cd99b4dc ### Client Binaries filename | sha512 hash -------- | ----------- [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.27.12/kubernetes-client-darwin-amd64.tar.gz) | 22786d81da77a4cae3e213c6f73173d0109fb5406a63f6a9949fbffe32bd9b93c21aa48eb7146c184d60cf61fa95d4eb6f7837cf59d7cf69e5dba37ffe2e6bb9 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.27.12/kubernetes-client-darwin-arm64.tar.gz) | f1498ae12c619e94455f72efafc1d08766737f438d9f52489637159940bf71b7e241a07069be1b7aade6423eb53e4e3e95eb62e46325777c7d3ade326263249c [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.27.12/kubernetes-client-linux-386.tar.gz) | acdb9a1254f09950411831d364c7f60482651d96122df35af400f27808b9c5f41c791ce56b4954529f5b15f48f95a3b4393c5c2f0803fa26f7e9079ac5572033 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.12/kubernetes-client-linux-amd64.tar.gz) | fb0feb0ff69808741a12ed0d498c7e7be292e96e90ba7da954a9c24c9c02a7fef82b432ab8c0afd098eced66779b36426ea86d46be320cf598147dcef5322bd2 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.27.12/kubernetes-client-linux-arm.tar.gz) | 06f928e8e83b93c0fbee7e023b7e6bd8ce976022b8bc2681e78393e597b73744a06ac0e23b9ea1cb227f01385d15a1fb6d791fbeb0a26c8fc38dab284ac94534 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.12/kubernetes-client-linux-arm64.tar.gz) | 0e453c4e68b254b6dc1d9e42388962e6c9952e8a7080a5ffefae7de56b52bf46027d0cc12393471017bca34d5ef36196f1d00fc4b8cbca29ae714253376ffa8d [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.12/kubernetes-client-linux-ppc64le.tar.gz) | 1438d4793fef13348508117bb5de9711fcc32886ea680cf64edf6898f73da5202465a7ed130303a670af3128a11464ca7a54d8d6f1af08bca574429dd5b56978 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.12/kubernetes-client-linux-s390x.tar.gz) | 513aa091b4cbfd3e7f5f6baf23909d2edae166255a09d876a14120dec80d87749c6b9e1b5e66f6b54b8018f7a6a015dad53bdb38028c697c4543942e87b8888e [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.27.12/kubernetes-client-windows-386.tar.gz) | 60412665e2e9764c9bae2f38162ed137b7f4147077b4a18a7312ed942c8145b3ea2ad4ea63181d5c344c5e7964c6d101e483291edf7d2e8e7c1ef2e9e1e86b59 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.12/kubernetes-client-windows-amd64.tar.gz) | 4175ebb2454eb30865082a44d5e58273710ca252abdd31c3b60348a742782a4c3b9c2f8923a9e96b638566bde893bc3a7467c644444473fcb56d45f74e726e5c [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.27.12/kubernetes-client-windows-arm64.tar.gz) | 6d91beee31d1a512b256ca5743b2db6b5ebe574d68bfbe18d0a65aa9e1002eae2974bcb9ebb793de841f366f5aa583549b588f91692ee6d73f1062e9a8ac2de7 ### Server Binaries filename | sha512 hash -------- | ----------- [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.12/kubernetes-server-linux-amd64.tar.gz) | 78c816adec98739bcadd1a5c66a2a95e1deaa486b2b5edda5b84ece158eeaf63d54e91cac4dc2277efd5ab1c9049d6d4ee6566219766011991c608db77cda067 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.12/kubernetes-server-linux-arm64.tar.gz) | 5c6130c76fc627887b003e4fa2fb36760bc2a88bd9cc390bc2157e584e5c85c5d48b00dd6b2ec9b7ec901b1207bb5ed6ee7374f8298c77d06dec36669e311005 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.12/kubernetes-server-linux-ppc64le.tar.gz) | 2e6c36e89533798c0cb80aaef5cefc6dcd9c5bcc58ec0bee30ba592524672d9ee33f61880c9ac2093971eed69695e8b0e642bfb5918168cb3fb52538c2e6147b [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.12/kubernetes-server-linux-s390x.tar.gz) | 42f3814f16eadb0b7173d71186f0cac8e874528ac91ecd2a4b042ce906ecf23b31409bef3f0230691c2ec751056c9ea288642d9510b8feb75ba37f1f5c2787ee ### Node Binaries filename | sha512 hash -------- | ----------- [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.12/kubernetes-node-linux-amd64.tar.gz) | e8183e5666fb3950f7e7c4ffcf1fffc21901f14ce369a427cf6b8c90a1c3d354c62b8312eb14af2f9ac0debbf67b60fad5c52f6fe34efe4575f29d6b3dad227a [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.12/kubernetes-node-linux-arm64.tar.gz) | c1737bf7bff67ec514a6a3e085419afc740833e896529e119e3e5e7b19b3f72806c6d164d0aa35dfd88964515673071af155fa2993487ce5a48d8d9e4f50ed4d [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.12/kubernetes-node-linux-ppc64le.tar.gz) | 91f423959b5384682cece3ba89d94b0d028a76339e2dbd7f0696248498dbf5f8cbeea54d3c9838d598cb5ed4acb20d91fbed83a3ed45ed2ba862c7176ab40b27 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.12/kubernetes-node-linux-s390x.tar.gz) | 6d723542905889d2600eb4b557228d15299b53b7941628c4486149b9e30ec72ff64cd2a781839e8b7bcb4966d849d2a11d2dabe3a76c4077feb9136e86464dc1 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.12/kubernetes-node-windows-amd64.tar.gz) | c19cad0359c86a55296b222b1f0cac7895fb7c55337ab6b7984e4c32338f9401f66194be53c19b4ab3ff1cada291a3d7a2a3f2865bd7c6184cdbb9a529cab9c2 ### Container Images All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures ---- | ------------- [registry.k8s.io/conformance:v1.27.12](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/conformance-s390x) [registry.k8s.io/kube-apiserver:v1.27.12](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-apiserver-s390x) [registry.k8s.io/kube-controller-manager:v1.27.12](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-controller-manager-s390x) [registry.k8s.io/kube-proxy:v1.27.12](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-proxy-s390x) [registry.k8s.io/kube-scheduler:v1.27.12](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler) | [amd64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/artifacts/docker/k8s-artifacts-prod/southamerica-east1/images/kube-scheduler-s390x) ## Changelog since v1.27.11 ## Changes by Kind ### Feature - Kubernetes is now built with go 1.21.8 - update distroless-iptables to v0.4.6 ([#123771](https://github.com/kubernetes/kubernetes/pull/123771), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing] ### Bug or Regression - Fixed cleanup of Pod volume mounts when a file was used as a subpath. ([#123052](https://github.com/kubernetes/kubernetes/pull/123052), [@jsafrane](https://github.com/jsafrane)) [SIG Node] - Fixed the disruption controller's PDB status synchronization to maintain all PDB conditions during an update. ([#122056](https://github.com/kubernetes/kubernetes/pull/122056), [@dhenkel92](https://github.com/dhenkel92)) [SIG Apps] - Fixes an issue calculating total CPU usage reported for Windows nodes ([#122999](https://github.com/kubernetes/kubernetes/pull/122999), [@marosset](https://github.com/marosset)) [SIG Node and Windows] - Updates google.golang.org/protobuf to v1.33.0 to resolve CVE-2024-24786 ([#123765](https://github.com/kubernetes/kubernetes/pull/123765), [@liggitt](https://github.com/liggitt)) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node and Storage] ### Other (Cleanup or Flake) - Build etcd image v3.5.12 ([#123069](https://github.com/kubernetes/kubernetes/pull/123069), [@bzsuni](https://github.com/bzsuni)) [SIG API Machinery and Etcd] ## Dependencies ### Added _Nothing has changed._ ### Changed - github.com/golang/protobuf: [v1.5.3 → v1.5.4](https://github.com/golang/protobuf/compare/v1.5.3...v1.5.4) - google.golang.org/protobuf: v1.31.0 → v1.33.0 ### Removed _Nothing has changed._ # v1.27.11 ## Downloads for v1.27.11 ### Source Code filename | sha512 hash -------- | ----------- [kubernetes.tar.gz](https://dl.k8s.io/v1.27.11/kubernetes.tar.gz) | 9a31c35d35d7b28a25b0bafdb95a960c8855db4c0736e5a6e434a52683deb0cf366ffcdb7e673c4d1e77891667c509368fba29bf3f3254802b7d0557d76b591a [kubernetes-src.tar.gz](https://dl.k8s.io/v1.27.11/kubernetes-src.tar.gz) | de20cf5725b5b67418c55636c92682f186f4881ab0d6239570ac2c080bf661e759f22d2b194f9f152580906a06303df5b7a631fecd2d101e4b51c790ecfacb08 ### Client Binaries filename | sha512 hash -------- | ----------- [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.27.11/kubernetes-client-darwin-amd64.tar.gz) | a37750a02efaf5809353bb65cccf3dded569dfac601ddfa2f9212f1714f1e419a5c9dcd81e64d80077f52f723a27744146f9f6d0aaa0c2b8216eb95317a65e20 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.27.11/kubernetes-client-darwin-arm64.tar.gz) | 95b734b6ce492a4936d4cce1807cc6246da774f54af73de59ca24dd59304d899422b904e236b55410ffac188fc331ab5591b9d2a51a636ab4419a626871eee16 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.27.11/kubernetes-client-linux-386.tar.gz) | c6e689e642d994147632f8ce2ba2833d873517f5b0d3dc22fa39437e1e8fbca7a5bd9aaa057699370d293c7b65e11a032fda274926bfd07f9485e4f6b5085f5c [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.11/kubernetes-client-linux-amd64.tar.gz) | 00af83d099771ae5cdca9e81cb372f0581675b229ab453493ec5452962cae590ca39931dbfb33350acee19b567a40b2cda3f4c881b554a9c1480b66dddcaea18 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.27.11/kubernetes-client-linux-arm.tar.gz) | 752e329575e6264b3063053605059ca08c55e5fd54bf0981b74e24dacb4081876b4af6487e1eb9b55eb611355756017625e354e0d7b5d8677b4c7cf2b6d17811 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.11/kubernetes-client-linux-arm64.tar.gz) | e0eeb4bdd249af17127b94bad5c2e6d72d088a0c017df2368a06c3997a910946f7109f1d1a04787effb2e6f14450b66f0a469aadbd46e0c5ab316b92584756da [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.11/kubernetes-client-linux-ppc64le.tar.gz) | e499df039a3efa659b8e49a4e268f5803bdb5514b5c5c37f4e08a05a0319176fcfdaf1f7fcf21aba13c27c09148052ce6fe4ada763ce0f3ea64bc74f922461cf [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.11/kubernetes-client-linux-s390x.tar.gz) | f59fe77e0f9a867d25595f9ef357b9bf499b481f198b739796db6152dd5e1389e7e240088e70b52e94f9122efe40c82cc6ef3b518a75d111ac2abfdf306dcb7f [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.27.11/kubernetes-client-windows-386.tar.gz) | 69a6e4c7e99deb397910eb72996a373f6cb67e075ea0f0002024b5057cd75d7f4645ffebeef1d07f578e95737cc6e302cccb7b7964ab42cd9bf56466423d2562 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.11/kubernetes-client-windows-amd64.tar.gz) | c1ed87946c8fe6870c57216043c7014133d5b1418a4815d89a338a3e6ac0bda6dc458ef3230cde92019251b918d875175c351bea490aabadaa876dd4e6acf61e [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.27.11/kubernetes-client-windows-arm64.tar.gz) | 95e834054b7ea6b461326dce7a8dda5a36dc6abdd362e9bf340607b6ce4072c13e9f9c628a86589807b055f5dd3c78f4bc6b4e227f893f945e44431ce43e869d ### Server Binaries filename | sha512 hash -------- | ----------- [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.11/kubernetes-server-linux-amd64.tar.gz) | f1e029b409b284a7ecb9e7dc09a66e2d0cf7807e1f69612078126c8c531d0195768a2ce03d43f632f7e72aa919755ed42c27a17b9d50ef0d2c0aeac075b59463 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.11/kubernetes-server-linux-arm64.tar.gz) | 6326edc5e2c7155665f4bb38fa402f56379b0d5d11294a07bac9e02e7490ff8e2f11d6fa6894e2356a283b567c339a6ec36ffc0d5094dfe5f1b0551e4f6c7d0a [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.11/kubernetes-server-linux-ppc64le.tar.gz) | b949861fd497fe305e27d02bcd7e7a8220d14d55cd8b0d0448c3da7dbd14191f4cc85cb05dd3c7202ee553ecd2b43fb220b193d3273c94c2a7215e137606c784 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.11/kubernetes-server-linux-s390x.tar.gz) | e081129da8bb8a7b279b7e0630c9e826ac5191b67c2afcd148b403b63604689ba7c48b223e37749596e9de7005edfad9683d49baae50a1bf6c3b6c52f80b7b88 ### Node Binaries filename | sha512 hash -------- | ----------- [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.11/kubernetes-node-linux-amd64.tar.gz) | cd92496fc7449cefdc2fb27dc1f734695a3d9e7b637089859a29e317ddf3991ed3dd79f6c583a7207192f2ad6568ed1cef8c4a499edc031183e23103160c76a0 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.11/kubernetes-node-linux-arm64.tar.gz) | fac3175d89cbf5f912e00e20905ea14ff14649263a6a090ee679152883900f2db1bf73bb7d39c6e63d40b3ff108d64d2ea78963c79471f4a9e511509887c1863 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.11/kubernetes-node-linux-ppc64le.tar.gz) | 76d3f9cfb44ff1d825fdb1ab47eb747c98a55f9b56413d798605e3b953b424409d3b722d28605964fa76d86751e52adc933efe5981964cdf0e8041308863a55c [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.11/kubernetes-node-linux-s390x.tar.gz) | 93b250bdea1993ef480aa4b30664602415d85166b0ebced21edc52b49cc8a4643158d39292a1b0e0375d89ea2da277b41ea7a0b3924f8b09a229b23d8b9839cc [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.11/kubernetes-node-windows-amd64.tar.gz) | 443578690e8f363de3cd53a5f0a944b9a0d9b3eec13748cb922d9ff45b352ac237ba9b58285ff6b35472fa18bdc3e3ed48d558b4d918b6b315cafa649a42ae74 ### Container Images All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures ---- | ------------- [registry.k8s.io/conformance:v1.27.11](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) [registry.k8s.io/kube-apiserver:v1.27.11](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) [registry.k8s.io/kube-controller-manager:v1.27.11](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) [registry.k8s.io/kube-proxy:v1.27.11](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) [registry.k8s.io/kube-scheduler:v1.27.11](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) ## Changelog since v1.27.10 ## Changes by Kind ### Feature - Add process_start_time_seconds to /metrics/slis endpoint of all components ([#122750](https://github.com/kubernetes/kubernetes/pull/122750), [@richabanker](https://github.com/richabanker)) [SIG Architecture, Instrumentation and Testing] - Kubernetes is now built with go 1.21.7 - update distroless-iptables to v0.4.5 ([#123229](https://github.com/kubernetes/kubernetes/pull/123229), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing] ### Bug or Regression - Fixes a race condition in the iptables mode of kube-proxy in 1.27 and later that could result in some updates getting lost (e.g., when a service gets a new endpoint, the rules for the new endpoint might not be added until much later). ([#122815](https://github.com/kubernetes/kubernetes/pull/122815), [@wedaly](https://github.com/wedaly)) [SIG Network] - If a pvc has an empty storageClassName, persistentvolume controller won't try to assign a default StorageClass ([#122704](https://github.com/kubernetes/kubernetes/pull/122704), [@carlory](https://github.com/carlory)) [SIG Apps and Storage] - Kubeadm: do not upload kubelet patch configuration into `kube-system/kubelet-config` ConfigMap ([#123106](https://github.com/kubernetes/kubernetes/pull/123106), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle] ## Dependencies ### Added _Nothing has changed._ ### Changed _Nothing has changed._ ### Removed _Nothing has changed._ # v1.27.10 ## Downloads for v1.27.10 ### Source Code filename | sha512 hash -------- | ----------- [kubernetes.tar.gz](https://dl.k8s.io/v1.27.10/kubernetes.tar.gz) | e2c2bd6339607fbe2da31b62e6ad3987c3789830a89c76db2edb0744d863a65ace41a2da33ad71489de51a2d950fc34c3700746a12439cbb3fc1d290d9df6b94 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.27.10/kubernetes-src.tar.gz) | 7a939d7ed2dec0b3b8b5ea2935a85cebdf56e6a9a91ff5404245ddb14ef267701284ed4cb04498ef73e58d50dd0f3d6ab5840c7271e6e0178f78492eb41d82d7 ### Client Binaries filename | sha512 hash -------- | ----------- [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.27.10/kubernetes-client-darwin-amd64.tar.gz) | 9e1364ea78d65d8ec318ec7db963714f2796bd2837e85447bc23095102cdfb0ca3093bb91c414fc957237bfeec0ce4cc304cb9710c248bb26493947b38869b45 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.27.10/kubernetes-client-darwin-arm64.tar.gz) | c63e41d5b34968cf0eff8e5315d04623d0aa311555b5d2b78bba58d9de938919ccaa4b7b698c123c79bbdf2bc0a35ce4c92bfdad7cc120b98d4bf6429cab1460 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.27.10/kubernetes-client-linux-386.tar.gz) | 93e545e1111bb8fd99843d96eb3ab0a5eb5323448b07e4b31d97f158756f95a75bad432ffc6df8f6be1ffa64c0b6529cb95f48a5710debb8ebf2d0a8f2ef11a4 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.10/kubernetes-client-linux-amd64.tar.gz) | da988a720d1eccdfc3be41a18b4cbb5c95bd81b465720afb2e8250d4b336196efe390b0944ef8ff456bb557fe4ee570e793f612959cae190c0dc32ffbc4c84ec [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.27.10/kubernetes-client-linux-arm.tar.gz) | c94631388be496dcd529da8b6b703e8980ecb00681e17bb67192ad8a8893aaaba1972a1f45d84281e68f75c04b4559385101826b84fbbd3502eef66231fc7b4f [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.10/kubernetes-client-linux-arm64.tar.gz) | f32cd2498b5b527b63505530ac55aba3e1a6626d132e730c9699d59fc909d8e24fa0898991a07007e57811ce2c049b8d0c645d6a0ec7e3d264f1b9d2aab7cb53 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.10/kubernetes-client-linux-ppc64le.tar.gz) | b9f914adf2aa5432cc9379cf7e4e8a1e5c5d5a4e099fded65bde39e35dfcf7540b33e003e89e3f291c13c2a97cb9c14b598f62574da1afa7af9834f87faeea24 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.10/kubernetes-client-linux-s390x.tar.gz) | 845e951aeceef72af7b057a77627b9ed42bbbf616b3a315dc82d9064d5d89abb306119543411ba1f68ec07f274530ed798cce4b35748de80e8fd4cccdcc88599 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.27.10/kubernetes-client-windows-386.tar.gz) | aabd7f420a0e75a9aa5f2918a2ed9abab6322c6ed96128625886212e9271f1c3efb79df349a358f83c5f60e51d18700c0dad8f1b635f17cf6e679324b6b4bd63 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.10/kubernetes-client-windows-amd64.tar.gz) | 372143250084888a54e535f10298efb49b3d9a74580100676a25d055150c93eafb58fa263de3253563b011613911726f3eb57349dcb1f9a0d462a962c17ce1e4 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.27.10/kubernetes-client-windows-arm64.tar.gz) | ef18c6aafe213f648403a65173bd6b540563d9469b2056816307a891d8a47c7eacb219efd9efb73bef9192a275f3387e8c34da022797f0c2f55cd8f4c3fb0981 ### Server Binaries filename | sha512 hash -------- | ----------- [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.10/kubernetes-server-linux-amd64.tar.gz) | 82ba4e852298f8ecb1b0b1d80d686d49b1fddbaad7352ad74c1530f3bccd1d088e11ea354224dd1c2caad9f0575a8cda9ae089429a0748b30c16fb4ca16f455a [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.10/kubernetes-server-linux-arm64.tar.gz) | 3ce89c7b4c6ef3ae262213d86d335336e3181c184306c48ed591aec9b8955e4d2c54f8f953395394d947761162d15818f9cc7a30a8a132c9e6a5fbd1753e10a5 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.10/kubernetes-server-linux-ppc64le.tar.gz) | 43cb867c6ddccaaaedc6254e7432301039521af8459531e1d6d398cd5672da153bc57a89832600b7a6d6e59be424ba37f4a316af869116b5ad14ac8b6104d27e [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.10/kubernetes-server-linux-s390x.tar.gz) | bfdd574d39acc9a1c465296a25b692cbc038c959b131688ac8a7355c4e9b3c687a654e6bfaa7e34330640e7cbba5dd7131e4592a20f9f62db368356bda49b234 ### Node Binaries filename | sha512 hash -------- | ----------- [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.10/kubernetes-node-linux-amd64.tar.gz) | f53d3d2f708fe6aa5fcc042469db85fa54655b324831c75b7edb6f692ec3c04691b95bd45338a8f481694d883e3aa3e9d22a70aabba4b5551fd13f1023fc0a47 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.10/kubernetes-node-linux-arm64.tar.gz) | b1734a6bdf7520d7c47230c75d29b29443c8140ce06b9d1f4c9ee06b84b8573df5617aa7e6eba893f5be64fc4bc64d15cb70a57d1a9c1657a9d87456a1ef8ef9 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.10/kubernetes-node-linux-ppc64le.tar.gz) | 7d458c310e3caed614762dabb495c90a65d9efd6924141becd361f820cccb4e1b33f314b4f3cfa0222c2e6a57e968ca63adf79d9aa6c9180b4a5d40b7ae4114e [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.10/kubernetes-node-linux-s390x.tar.gz) | 24524b5ed886fd818c8e389b3bdb6e4a3273ebb762e29961a5d3457aff8c22c702e1b58547e1864d4bd6e8339af68155a31f9da793c552f34c373ad3050a81ec [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.10/kubernetes-node-windows-amd64.tar.gz) | e82471d53325f6171003bfe2c641c4a8bfff8b4a5737de00b4ab6fb65099b157b97cd1408d46c99a5f151b548472af95bc3ec3e6955d4b78886aa2799b82e60a ### Container Images All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures ---- | ------------- [registry.k8s.io/conformance:v1.27.10](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) [registry.k8s.io/kube-apiserver:v1.27.10](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) [registry.k8s.io/kube-controller-manager:v1.27.10](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) [registry.k8s.io/kube-proxy:v1.27.10](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) [registry.k8s.io/kube-scheduler:v1.27.10](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) ## Changelog since v1.27.9 ## Changes by Kind ### Feature - Kubernetes is now built with Go 1.20.13 ([#122713](https://github.com/kubernetes/kubernetes/pull/122713), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing] ### Bug or Regression - Allow deletion of pods that use raw block volumes on node reboot ([#122211](https://github.com/kubernetes/kubernetes/pull/122211), [@gnufied](https://github.com/gnufied)) [SIG Node and Storage] - Etcd: Update to version 3.5.10 ([#121804](https://github.com/kubernetes/kubernetes/pull/121804), [@mzaian](https://github.com/mzaian)) [SIG API Machinery, Cloud Provider, Cluster Lifecycle, Etcd and Testing] - Fix: Mount point may become local without calling NodePublishVolume after node rebooting. ([#119923](https://github.com/kubernetes/kubernetes/pull/119923), [@cvvz](https://github.com/cvvz)) [SIG Node and Storage] - Fix: statle smb mount issue when smb file share is deleted and then unmount ([#121851](https://github.com/kubernetes/kubernetes/pull/121851), [@andyzhangx](https://github.com/andyzhangx)) [SIG Storage] - Fixed a regression since 1.24 in the scheduling framework when overriding MultiPoint plugins (e.g. default plugins). The incorrect loop logic might lead to a plugin being loaded multiple times, consequently preventing any Pod from being scheduled, which is unexpected. ([#122370](https://github.com/kubernetes/kubernetes/pull/122370), [@caohe](https://github.com/caohe)) [SIG Scheduling] ### Other (Cleanup or Flake) - Reverts the EventedPLEG feature (beta, but disabled by default) back to alpha for a known issue ([#122720](https://github.com/kubernetes/kubernetes/pull/122720), [@pacoxu](https://github.com/pacoxu)) [SIG Node] ## Dependencies ### Added _Nothing has changed._ ### Changed - golang.org/x/crypto: v0.14.0 → v0.16.0 - golang.org/x/mod: v0.12.0 → v0.14.0 - golang.org/x/net: v0.17.0 → v0.19.0 - golang.org/x/sync: v0.3.0 → v0.5.0 - golang.org/x/sys: v0.13.0 → v0.15.0 - golang.org/x/term: v0.13.0 → v0.15.0 - golang.org/x/text: v0.13.0 → v0.14.0 - golang.org/x/tools: v0.12.0 → v0.16.1 ### Removed _Nothing has changed._ # v1.27.9 ## Downloads for v1.27.9 ### Source Code filename | sha512 hash -------- | ----------- [kubernetes.tar.gz](https://dl.k8s.io/v1.27.9/kubernetes.tar.gz) | c45eb288ebcb3e44abd6dea94b82ca691051950a38e7d419d0628af8afe9d03b06005d22ef2045d535788165ad75b1798fdf79920bc84a26ec1b232764ce3097 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.27.9/kubernetes-src.tar.gz) | 17b25d4e773d56356109ea964b9aa19c50a506f166133049b95f7c77a55479d76099ddfc66f5de9612b4a316bd02c544cfcbbb6ebec89e4f8727e8f60005a78d ### Client Binaries filename | sha512 hash -------- | ----------- [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.27.9/kubernetes-client-darwin-amd64.tar.gz) | 9f0c69acc969fc169c344eb14b3909fc29609d4e7386bd77f9e660d0f0523b77feef12e575972c57b51eb0ea5bf29d4e91cb0191d05c6cdc9fb75dcc3b4fb9b2 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.27.9/kubernetes-client-darwin-arm64.tar.gz) | 84b9d7af34861e3dc09b7937ee40dc4c75c1e417314605706ff039d44afad973ecbf802354d7543bd6c926eafbdf09bf4ae9b9f69298374ff7d51c14e24e830e [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.27.9/kubernetes-client-linux-386.tar.gz) | 5920b386634463f9c66dd0c2e1c7a08b11f32cb40e6a937b9412db345fe106c9428d121e7400f16627a66e4301e16dd0147d5ff87a69ce5509c68d833530ad97 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.9/kubernetes-client-linux-amd64.tar.gz) | 3297ca12174a071158ebd4bcb8c2a3624e08d5a3bef5f5cd74acdb30c03183005592cf4190ed4f4ff95430e21b2520b95a4481db60982eb4e30934cc568023f0 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.27.9/kubernetes-client-linux-arm.tar.gz) | e864f921b04e27a7e8547c517a4a72b71ac32f5af60904a66f6dbec5145ec5baf05b934f0ed1e3622fdacb2436472bb65d0f1f6dc9de217bd7f5dae694510814 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.9/kubernetes-client-linux-arm64.tar.gz) | 9c0323c6f8400d2dd6a1512ac0d0f03f24d50aa1bf5d6da1ff8b012d5efc8ffbb23d7249edb451976d48610414bcabfdaa9494d73505e21dd9d5bd054dd9a31d [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.9/kubernetes-client-linux-ppc64le.tar.gz) | 7227dbc25ef30593781b99696b32db60dd9d961e6d0d5649db539fcdc573f700affe49411300572628e27d1dbb15089dc8f96a6f4a52239d75ccbcbbcd3dd016 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.9/kubernetes-client-linux-s390x.tar.gz) | 4d78042bba3156ab609e0abbe2eb64798955d35fe825683019448e6ea1822860349653431b8d28b731ffbd07ef315e2badd58e2362681c1b69b4db3071cf9d22 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.27.9/kubernetes-client-windows-386.tar.gz) | ac1484f912ef46897356aa23db889dd6e4da97fc7c3022e3587ddaa5fee96bd7cfe9f839f8762e94992eec71869161818457cef4614d860dfbdcb4a12571fc97 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.9/kubernetes-client-windows-amd64.tar.gz) | 973b0588f481dc4bcf8f6bd10834ed1eb668898729718611a6f0e2302f9aabd8f22712883a103031f85a9a6ce9f574b08c0c4457c53b38f836ccb83e707f5897 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.27.9/kubernetes-client-windows-arm64.tar.gz) | 013c2586baecd615b3108802ada0670ffdfcb62766afdefde65ccb5d84b7d2d10797335e2c78f0561c752aa0e88ab0adcf874c340b960039548b49f97120b973 ### Server Binaries filename | sha512 hash -------- | ----------- [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.9/kubernetes-server-linux-amd64.tar.gz) | cf4e15a89a6166162de4fb3ca6ed2d82a7f5756135efea8a57bfee8b49753424280d48d39cb49d5195d15efe08b0b0f0a5dad8ea60bf2b03b2ce044dcece98c6 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.9/kubernetes-server-linux-arm64.tar.gz) | 069b4c33c74766a6842aa3086c459cbc5958feaf238422778ad89d26bf6f82d0ffc02f0a40df1bc38f46aefa101190e8d6c2e2e341d1ffc60002aa51d9c1c19e [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.9/kubernetes-server-linux-ppc64le.tar.gz) | 067653d632d6204e605ed2c908d9ede87bd715dca45e4ee7cbef54563d23c9abc140481fca44ea337b58f60ca380b159970724e158bf1cd5a031a85c4610ace5 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.9/kubernetes-server-linux-s390x.tar.gz) | ad0aa5398f4832e06ead0e7e3c56821bf99a9e23c4390db98ff250791da961ff568c74a5b7358d3a14bd011fcd176995f5e403ab477c0d34184f38252aa8bce9 ### Node Binaries filename | sha512 hash -------- | ----------- [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.9/kubernetes-node-linux-amd64.tar.gz) | 4e3a347367a0e3acc9ba53dd2bb6460ca615a626ccd39f573155cf7256942731a66df74eccce571a55e19603b4626057fc5ca619a7b5c990ec422a0e16bc0823 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.9/kubernetes-node-linux-arm64.tar.gz) | dc61d17cd12ddf93d2a7db731ddbcd4c8d1fc809f6228b5c409723a0c4c343e0654b40cf3a4ff464f2f802822b806e4848c5203bda28832a4df2ec5e6e896ae9 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.9/kubernetes-node-linux-ppc64le.tar.gz) | b3f81b10e0ab7b7f779e5374611a2f8f0483a595a1d7daa2037c261da5c04ba6e19ccafab77c624a20f41654f3d2c047a0224c462dee2f35d90b67a132c72f63 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.9/kubernetes-node-linux-s390x.tar.gz) | 2bd20281feef3c2d0305302c2d18f334744fd2e9b90e479bbcd9b05ee55bebcac485ec755cea84dd433f3b93caf05c47d17dc0084680de7998c67d63cc745622 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.9/kubernetes-node-windows-amd64.tar.gz) | eda9bf9e5543be317d932ae859a509cb7ba34f047c59e7b3fa7017be3233b8f426b11bf33e4e97dbdf6e43d6939f196850476078f0096d395d9b6a9f659ac9c7 ### Container Images All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures ---- | ------------- [registry.k8s.io/conformance:v1.27.9](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) [registry.k8s.io/kube-apiserver:v1.27.9](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) [registry.k8s.io/kube-controller-manager:v1.27.9](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) [registry.k8s.io/kube-proxy:v1.27.9](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) [registry.k8s.io/kube-scheduler:v1.27.9](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) ## Changelog since v1.27.8 ## Changes by Kind ### Feature - Kubernetes is now built with Go 1.20.12 ([#122217](https://github.com/kubernetes/kubernetes/pull/122217), [@xmudrii](https://github.com/xmudrii)) [SIG Release and Testing] ### Bug or Regression - Fixed a regression since 1.27.0 in scheduler framework when running score plugins. The `skippedScorePlugins` number might be greater than `enabledScorePlugins`, so when initializing a slice the cap(len(skippedScorePlugins) - len(enabledScorePlugins)) is negative, which is not allowed. ([#121666](https://github.com/kubernetes/kubernetes/pull/121666), [@kerthcet](https://github.com/kerthcet)) [SIG Scheduling] - Fixes a kube-apiserver log volume regression bug in default 1.27 configurations (introduced in 1.26, activated by the AggregatedDiscoveryEndpoint feature enablement in 1.27) ([#122074](https://github.com/kubernetes/kubernetes/pull/122074), [@ritazh](https://github.com/ritazh)) [SIG API Machinery] - Fixes an issue where StatefulSet might not restart a pod after eviction or node failure. ([#121389](https://github.com/kubernetes/kubernetes/pull/121389), [@aleksandra-malinowska](https://github.com/aleksandra-malinowska)) [SIG Apps and Testing] - The scheduling queue didn't notice any extenders' failures, it could miss some cluster events, and it could end up Pods rejected by Extenders stuck in unschedulable pod pool in 5min in the worst-case scenario. Now, the scheduling queue notices extenders' failures and requeue Pods rejected by Extenders appropriately. ([#122044](https://github.com/kubernetes/kubernetes/pull/122044), [@sanposhiho](https://github.com/sanposhiho)) [SIG Scheduling] ### Other (Cleanup or Flake) - Bump distroless-iptables to 0.2.8 based on Go 1.20.11 ([#121975](https://github.com/kubernetes/kubernetes/pull/121975), [@cpanato](https://github.com/cpanato)) [SIG Testing] - Makefile and scripts now respect GOTOOLCHAIN and otherwise ensure ./.go-version is used ([#122076](https://github.com/kubernetes/kubernetes/pull/122076), [@BenTheElder](https://github.com/BenTheElder)) [SIG Release and Testing] ## Dependencies ### Added _Nothing has changed._ ### Changed - github.com/cyphar/filepath-securejoin: [v0.2.3 → v0.2.4](https://github.com/cyphar/filepath-securejoin/compare/v0.2.3...v0.2.4) - github.com/rogpeppe/go-internal: [v1.11.0 → v1.12.0](https://github.com/rogpeppe/go-internal/compare/v1.11.0...v1.12.0) ### Removed _Nothing has changed._ # v1.27.8 ## Downloads for v1.27.8 ### Source Code filename | sha512 hash -------- | ----------- [kubernetes.tar.gz](https://dl.k8s.io/v1.27.8/kubernetes.tar.gz) | f3d0a8d8b2c84d3d03ac49bc819cb4cc7e175feb9445c159e8271e045af64bb7fbf7023e1b11ca3bcbe502ec6e7d15d9ee84c165ad8d13716f58e5db1d345b21 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.27.8/kubernetes-src.tar.gz) | 119afe357398239dd00ac065a57327be75bd4f562a94ad800e472bd2068c58dadbd50970b6876e7cccaa66b4f34ccb2023c14173dbab0f44cfa41dfa6adafa37 ### Client Binaries filename | sha512 hash -------- | ----------- [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.27.8/kubernetes-client-darwin-amd64.tar.gz) | 54d48ff1afc5b33c0ec026f882f111098f096ffa9afb35e3b757bf9c106b82e4ecc35ad57448276d4419d069ad1d3cdaacc1d62a98c3c16de590840d9911557b [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.27.8/kubernetes-client-darwin-arm64.tar.gz) | 13d19a302ca82a80477c1a20ded6835221cff8302ec5c7203ea687f8cf683acbb347dbe7aeae5095461f5ed1f6431887825895aa8f91bb71bda0a4b0d1b55bc8 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.27.8/kubernetes-client-linux-386.tar.gz) | 9f79954501acc553cff3cffca7f5a77e51ab7171b013cb76a801b56343459df3adb202b7e21421dafd09fb87aae43a480514103da9a8000bf9f8b7e1f79c4571 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.8/kubernetes-client-linux-amd64.tar.gz) | d52faf06b8b5499564ddb06836f76e3a330f8f21e0fe7ffee8e6f36a95c40bbed7ef8db8aefd48f867b4d63fe02f1f562a146b71f669e960f1c6ed18820f36dd [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.27.8/kubernetes-client-linux-arm.tar.gz) | be14007563bf5d09556bb81608de1d63f1ec1ece3f8774e81bda1e05ecf6ef387a733483f96822c924611220b1edea926ade163cbfb5d610b6404cfb62e4890e [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.8/kubernetes-client-linux-arm64.tar.gz) | e0d3f3b9d6f003f5e51d9c995b6b73a4f1e0b8247b387923aca2a8bce2ffa79120e956d5041a9a7431c7b8f8fdc455d55b2d3270f7ca71228ef52bf06e887b6f [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.8/kubernetes-client-linux-ppc64le.tar.gz) | a53e0fb2395834dc40cf5647ec754d109e17e2dfc90c83b73867b770e01d7fa78c505602354a707df2212255e73a038af4452f233b50532cff6fbb089fa69176 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.8/kubernetes-client-linux-s390x.tar.gz) | ce097bc724f5ab8558174a8acbac92cbdb41a92a4297cddc69c875055f7c8dbbf12a8e7918adf1365524b57950f886ea6d81278f242c862870b51a802925085d [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.27.8/kubernetes-client-windows-386.tar.gz) | febefb0c8dbe92bae9280c027387942d131b2ec89853fa675645f9f881dee4a773a2fba42fd39e845fc05496539a02faf47eb2d90db93e4d9d25866fdeb1cc7c [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.8/kubernetes-client-windows-amd64.tar.gz) | 1ff515f3d7e68c7c87636c282ee61c02951e5f633840cddfcd01de4af34d75baa10d417f7f24bac141601aff893773e5e995d0bea2914431f38ff9daa7ea576f [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.27.8/kubernetes-client-windows-arm64.tar.gz) | 5a93472bdbb42d51c7d044890b2f97427feb44427cbfb05fcb5dc689be37723b19fbb85f1262ac69f396357c0db06bd9d8a65919c518f9daa4d2ed5d8033d598 ### Server Binaries filename | sha512 hash -------- | ----------- [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.8/kubernetes-server-linux-amd64.tar.gz) | 663358df01ee06bd9a261aa22237d0b91dcccc9cacdf39fa65409d54e16efe91dbe3d246cc30011c123f2d4803583ea88eed8bd537e82ca894f11cb31559538f [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.8/kubernetes-server-linux-arm64.tar.gz) | 080acd75eff58f72dee89f49c34cf772f11214f4eb9857287f604d9b20e2d4d7e954039010cb6a510ea5f3c360ab71cad853bf4cdbdbd821c0bfe82e5bcb5df8 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.8/kubernetes-server-linux-ppc64le.tar.gz) | fb04b0f5d06156aed6c0d76ad4d9844d8274f295daa85920fb3befa9a7d57e2b4d9ff78b1cfdccedfe48e11ef5b4d24b703656b71bd497be1b078551b13921ed [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.8/kubernetes-server-linux-s390x.tar.gz) | 3929f0f31861ce6db6a2b00a6e798c8015a7da33a7f4b4e0e8caa732766201baa8d8e4e60bf12d51302d8896a19ed82266b0199ef1305d3d0861ca7cb1838476 ### Node Binaries filename | sha512 hash -------- | ----------- [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.8/kubernetes-node-linux-amd64.tar.gz) | b07e49275501535f21c69794e938741408a891fb5f1305f430c9d2fa679fe63263edf238afccc9e41b76dae7340244a1798583de6a0504a513b6986e376f24f7 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.8/kubernetes-node-linux-arm64.tar.gz) | f7b6e63945edeaa582c47f8407900a10fbe0739a16d974f413a57e0723639989af092f7a92571acc69d2c97666b79ad00175cef581bbc01966647353c676c5e9 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.8/kubernetes-node-linux-ppc64le.tar.gz) | 3b5ad668e8fa85fdde6d0225f2ededd7ddd1f59aa1e14cb50a06171585d373e077e050ebfe879ea7603223e26c69555f40ea15c4f89637bd00bbc124681f2ea3 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.8/kubernetes-node-linux-s390x.tar.gz) | 64145434328efd21189bf21c9a36bc709cb6e0166a0351a157392129ce22ac040adbaf51c265b3fb8c8ff8329c7cc7ce8d1af08a0b5500a595fe816435ebe22b [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.8/kubernetes-node-windows-amd64.tar.gz) | 914e840bd1375e42258121c6ea5fb1394b5cc8321e5fe4520c2ebb5b979fea881a2af262599742f751673bb1d557e6fab08281e5c9da4687614bf4dc0f3d6064 ### Container Images All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures ---- | ------------- [registry.k8s.io/conformance:v1.27.8](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) [registry.k8s.io/kube-apiserver:v1.27.8](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) [registry.k8s.io/kube-controller-manager:v1.27.8](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) [registry.k8s.io/kube-proxy:v1.27.8](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) [registry.k8s.io/kube-scheduler:v1.27.8](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) ## Changelog since v1.27.7 ## Important Security Information This release contains changes that address the following vulnerabilities: ### CVE-2023-5528: Insufficient input sanitization in in-tree storage plugin leads to privilege escalation on Windows nodes A security issue was discovered in Kubernetes where a user that can create pods and persistent volumes on Windows nodes may be able to escalate to admin privileges on those nodes. Kubernetes clusters are only affected if they are using an in-tree storage plugin for Windows nodes. **Affected Versions**: - kubelet >= v1.8.0 **Fixed Versions**: - kubelet v1.28.4 - kubelet v1.27.8 - kubelet v1.26.11 - kubelet v1.25.16 This vulnerability was reported by Tomer Peled @tomerpeled92" **CVSS Rating:** High (7.2) [CVSS:3.1/AV:N/AC:L/PR:H/UI:N/S:U/C:H/I:H/A:H](https://www.first.org/cvss/calculator/3.1#CVSS:3.1/AV:N/AC:L/PR:H/UI:N/S:U/C:H/I:H/A:H) ## Changes by Kind ### API Change - Retry NodeStageVolume calls if CSI node driver is not running ([#120330](https://github.com/kubernetes/kubernetes/pull/120330), [@rohitssingh](https://github.com/rohitssingh)) [SIG Apps, Storage and Testing] ### Feature - Kubernetes is now built with Go 1.20.11 ([#121811](https://github.com/kubernetes/kubernetes/pull/121811), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing] ### Bug or Regression - Fix 121094 by re-introducing the readiness predicate for externalTrafficPolicy: Local services. ([#121116](https://github.com/kubernetes/kubernetes/pull/121116), [@alexanderConstantinescu](https://github.com/alexanderConstantinescu)) [SIG Cloud Provider and Network] - Fixed a regression in default configurations, which enabled PodDisruptionConditions by default, that prevented the control plane's pod garbage collector from deleting pods that contained duplicated field keys (env. variables with repeated keys or container ports). ([#121380](https://github.com/kubernetes/kubernetes/pull/121380), [@mimowo](https://github.com/mimowo)) [SIG Apps, Auth, Scheduling and Testing] - Fixed the issue where pod with ordinal number lower than the rolling partitioning number was being deleted it was coming up with updated image. ([#120731](https://github.com/kubernetes/kubernetes/pull/120731), [@adilGhaffarDev](https://github.com/adilGhaffarDev)) [SIG Apps and Testing] - Fixes calculating the requeue time in the cronjob controller, which results in properly handling failed/stuck jobs ([#121327](https://github.com/kubernetes/kubernetes/pull/121327), [@soltysh](https://github.com/soltysh)) [SIG Apps] - Service Controller: update load balancer hosts after node's ProviderID is updated ([#121138](https://github.com/kubernetes/kubernetes/pull/121138), [@code-elinka](https://github.com/code-elinka)) [SIG Cloud Provider, Network, Release and Testing] ## Dependencies ### Added - cloud.google.com/go/accessapproval: v1.6.0 - cloud.google.com/go/accesscontextmanager: v1.7.0 - cloud.google.com/go/aiplatform: v1.37.0 - cloud.google.com/go/analytics: v0.19.0 - cloud.google.com/go/apigateway: v1.5.0 - cloud.google.com/go/apigeeconnect: v1.5.0 - cloud.google.com/go/apigeeregistry: v0.6.0 - cloud.google.com/go/appengine: v1.7.1 - cloud.google.com/go/area120: v0.7.1 - cloud.google.com/go/artifactregistry: v1.13.0 - cloud.google.com/go/asset: v1.13.0 - cloud.google.com/go/assuredworkloads: v1.10.0 - cloud.google.com/go/automl: v1.12.0 - cloud.google.com/go/baremetalsolution: v0.5.0 - cloud.google.com/go/batch: v0.7.0 - cloud.google.com/go/beyondcorp: v0.5.0 - cloud.google.com/go/billing: v1.13.0 - cloud.google.com/go/binaryauthorization: v1.5.0 - cloud.google.com/go/certificatemanager: v1.6.0 - cloud.google.com/go/channel: v1.12.0 - cloud.google.com/go/cloudbuild: v1.9.0 - cloud.google.com/go/clouddms: v1.5.0 - cloud.google.com/go/cloudtasks: v1.10.0 - cloud.google.com/go/compute/metadata: v0.2.3 - cloud.google.com/go/compute: v1.19.1 - cloud.google.com/go/contactcenterinsights: v1.6.0 - cloud.google.com/go/container: v1.15.0 - cloud.google.com/go/containeranalysis: v0.9.0 - cloud.google.com/go/datacatalog: v1.13.0 - cloud.google.com/go/dataflow: v0.8.0 - cloud.google.com/go/dataform: v0.7.0 - cloud.google.com/go/datafusion: v1.6.0 - cloud.google.com/go/datalabeling: v0.7.0 - cloud.google.com/go/dataplex: v1.6.0 - cloud.google.com/go/dataproc: v1.12.0 - cloud.google.com/go/dataqna: v0.7.0 - cloud.google.com/go/datastream: v1.7.0 - cloud.google.com/go/deploy: v1.8.0 - cloud.google.com/go/dialogflow: v1.32.0 - cloud.google.com/go/dlp: v1.9.0 - cloud.google.com/go/documentai: v1.18.0 - cloud.google.com/go/domains: v0.8.0 - cloud.google.com/go/edgecontainer: v1.0.0 - cloud.google.com/go/errorreporting: v0.3.0 - cloud.google.com/go/essentialcontacts: v1.5.0 - cloud.google.com/go/eventarc: v1.11.0 - cloud.google.com/go/filestore: v1.6.0 - cloud.google.com/go/functions: v1.13.0 - cloud.google.com/go/gaming: v1.9.0 - cloud.google.com/go/gkebackup: v0.4.0 - cloud.google.com/go/gkeconnect: v0.7.0 - cloud.google.com/go/gkehub: v0.12.0 - cloud.google.com/go/gkemulticloud: v0.5.0 - cloud.google.com/go/gsuiteaddons: v1.5.0 - cloud.google.com/go/iam: v0.13.0 - cloud.google.com/go/iap: v1.7.1 - cloud.google.com/go/ids: v1.3.0 - cloud.google.com/go/iot: v1.6.0 - cloud.google.com/go/kms: v1.10.1 - cloud.google.com/go/language: v1.9.0 - cloud.google.com/go/lifesciences: v0.8.0 - cloud.google.com/go/logging: v1.7.0 - cloud.google.com/go/longrunning: v0.4.1 - cloud.google.com/go/managedidentities: v1.5.0 - cloud.google.com/go/maps: v0.7.0 - cloud.google.com/go/mediatranslation: v0.7.0 - cloud.google.com/go/memcache: v1.9.0 - cloud.google.com/go/metastore: v1.10.0 - cloud.google.com/go/monitoring: v1.13.0 - cloud.google.com/go/networkconnectivity: v1.11.0 - cloud.google.com/go/networkmanagement: v1.6.0 - cloud.google.com/go/networksecurity: v0.8.0 - cloud.google.com/go/notebooks: v1.8.0 - cloud.google.com/go/optimization: v1.3.1 - cloud.google.com/go/orchestration: v1.6.0 - cloud.google.com/go/orgpolicy: v1.10.0 - cloud.google.com/go/osconfig: v1.11.0 - cloud.google.com/go/oslogin: v1.9.0 - cloud.google.com/go/phishingprotection: v0.7.0 - cloud.google.com/go/policytroubleshooter: v1.6.0 - cloud.google.com/go/privatecatalog: v0.8.0 - cloud.google.com/go/pubsublite: v1.7.0 - cloud.google.com/go/recaptchaenterprise/v2: v2.7.0 - cloud.google.com/go/recommendationengine: v0.7.0 - cloud.google.com/go/recommender: v1.9.0 - cloud.google.com/go/redis: v1.11.0 - cloud.google.com/go/resourcemanager: v1.7.0 - cloud.google.com/go/resourcesettings: v1.5.0 - cloud.google.com/go/retail: v1.12.0 - cloud.google.com/go/run: v0.9.0 - cloud.google.com/go/scheduler: v1.9.0 - cloud.google.com/go/secretmanager: v1.10.0 - cloud.google.com/go/security: v1.13.0 - cloud.google.com/go/securitycenter: v1.19.0 - cloud.google.com/go/servicedirectory: v1.9.0 - cloud.google.com/go/shell: v1.6.0 - cloud.google.com/go/spanner: v1.45.0 - cloud.google.com/go/speech: v1.15.0 - cloud.google.com/go/storagetransfer: v1.8.0 - cloud.google.com/go/talent: v1.5.0 - cloud.google.com/go/texttospeech: v1.6.0 - cloud.google.com/go/tpu: v1.5.0 - cloud.google.com/go/trace: v1.9.0 - cloud.google.com/go/translate: v1.7.0 - cloud.google.com/go/video: v1.15.0 - cloud.google.com/go/videointelligence: v1.10.0 - cloud.google.com/go/vision/v2: v2.7.0 - cloud.google.com/go/vmmigration: v1.6.0 - cloud.google.com/go/vmwareengine: v0.3.0 - cloud.google.com/go/vpcaccess: v1.6.0 - cloud.google.com/go/webrisk: v1.8.0 - cloud.google.com/go/websecurityscanner: v1.5.0 - cloud.google.com/go/workflows: v1.10.0 - github.com/googleapis/enterprise-certificate-proxy: [v0.2.3](https://github.com/googleapis/enterprise-certificate-proxy/tree/v0.2.3) - google.golang.org/genproto/googleapis/api: 1aefcd6 - google.golang.org/genproto/googleapis/rpc: 28d5490 ### Changed - cloud.google.com/go/bigquery: v1.8.0 → v1.50.0 - cloud.google.com/go/datastore: v1.1.0 → v1.11.0 - cloud.google.com/go/firestore: v1.1.0 → v1.9.0 - cloud.google.com/go/pubsub: v1.3.1 → v1.30.0 - cloud.google.com/go: v0.97.0 → v0.110.0 - github.com/census-instrumentation/opencensus-proto: [v0.2.1 → v0.4.1](https://github.com/census-instrumentation/opencensus-proto/compare/v0.2.1...v0.4.1) - github.com/cespare/xxhash/v2: [v2.1.2 → v2.2.0](https://github.com/cespare/xxhash/v2/compare/v2.1.2...v2.2.0) - github.com/cncf/udpa/go: [04548b0 → c52dc94](https://github.com/cncf/udpa/go/compare/04548b0...c52dc94) - github.com/cncf/xds/go: [cb28da3 → e9ce688](https://github.com/cncf/xds/go/compare/cb28da3...e9ce688) - github.com/envoyproxy/go-control-plane: [49ff273 → 9239064](https://github.com/envoyproxy/go-control-plane/compare/49ff273...9239064) - github.com/envoyproxy/protoc-gen-validate: [v0.1.0 → v0.10.1](https://github.com/envoyproxy/protoc-gen-validate/compare/v0.1.0...v0.10.1) - github.com/golang/glog: [v1.0.0 → v1.1.0](https://github.com/golang/glog/compare/v1.0.0...v1.1.0) - github.com/googleapis/gax-go/v2: [v2.1.1 → v2.7.1](https://github.com/googleapis/gax-go/v2/compare/v2.1.1...v2.7.1) - go.opencensus.io: v0.23.0 → v0.24.0 - golang.org/x/mod: v0.9.0 → v0.12.0 - golang.org/x/oauth2: ee48083 → v0.7.0 - golang.org/x/sync: v0.1.0 → v0.3.0 - golang.org/x/time: 90d013b → v0.3.0 - golang.org/x/tools: v0.7.0 → v0.12.0 - google.golang.org/api: v0.60.0 → v0.114.0 - google.golang.org/genproto: c8bf987 → 438c736 - google.golang.org/grpc: v1.51.0 → v1.56.3 - google.golang.org/protobuf: v1.28.1 → v1.31.0 - gopkg.in/gcfg.v1: v1.2.0 → v1.2.3 ### Removed _Nothing has changed._ # v1.27.7 ## Downloads for v1.27.7 ### Source Code filename | sha512 hash -------- | ----------- [kubernetes.tar.gz](https://dl.k8s.io/v1.27.7/kubernetes.tar.gz) | b12f023254d40f791355aeec2eb84d521035648cf3e19994eacdc6c7516373f11dad942ae97d4bc8a7f255654aa7c742c1c10f18b4f4830b64e78a0b7bb35083 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.27.7/kubernetes-src.tar.gz) | 137db7a6ddb85c7eb0676cc3cb2bfadd726073a34b1edae4e2c3cc15165a43c0f16d163930015de8a5e357e8ff099c0f8d03f036aa245704b10348c7c91483b1 ### Client Binaries filename | sha512 hash -------- | ----------- [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.27.7/kubernetes-client-darwin-amd64.tar.gz) | a2ce6205bb613454167b1e6f5c6be34516e9624f1cc0eec2b6b2aa0e0b3bfc7d266379f035a7eea08625bf97413ff4cf23c9dc65669529026ad8589a0e4f9a70 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.27.7/kubernetes-client-darwin-arm64.tar.gz) | 28b6df012e1af6a062f5815a0e8c8bd440c824e520c6954a55ea9fba917c328f23069c124bab7f5bfc4b37e3a20542b33cf41d07d715f7a54bd78bcdcabca70a [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.27.7/kubernetes-client-linux-386.tar.gz) | 7fcaa119db37f5a5212fd3a5fd08db37b1ce701d67922d1f65cc757edc02f282566ae4d001e11f7b7dab3e24b27f3745189dd7fc63c90e97e9ce6a070ba8b094 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.7/kubernetes-client-linux-amd64.tar.gz) | 87b7ac839cac8d96efa1c6170cf32ed2bbe14e7194971df4b4736699152e294a0aa0018f3d8ae1dcf9905c3c784a7a15c297382450c0431a0daf98f300d3ef16 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.27.7/kubernetes-client-linux-arm.tar.gz) | 1bc0420005fa0e564568aa4bcf5a61e96d4c2c42afee4d34df940c4b89f0639e90771deda40a1b30f536ce8f1bd4d04cf228af98edf48ba0fa6685babe11311a [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.7/kubernetes-client-linux-arm64.tar.gz) | d4e96a6be6e15530e866399a5760f9410fe319217f7d91026d93a27e1a2ce9398380adc62f463a347f383ced253e359fba2fb291bd8a644f067ffc4ce8457d6c [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.7/kubernetes-client-linux-ppc64le.tar.gz) | abcc2c651514f0b2a4cf28934078ea701d3591d318e5eac080e7958f70fa94cb4b83ca9ee0f0130749c29a20c3bf8bea545c7641cfff0b78dc78571cb8e14f22 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.7/kubernetes-client-linux-s390x.tar.gz) | 0fb9a5bd534a29b84be6f1a5aae59e2a05531b1ff40019896e4bbbd3bd948a96313f65140764a656e0305e6f48cdb113e6fcf1c8195d4fadfa8bf62dea18db5b [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.27.7/kubernetes-client-windows-386.tar.gz) | 36a78882ec5960a561f928e2bb4ffb1c5dc7e884ee6471441d5de6d8fe0fec6cfd5a1bcc48dd933e490b07d7f837de93eecfb9ae353dcefc5dc4f699f02b5757 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.7/kubernetes-client-windows-amd64.tar.gz) | a662732fa75fb2fc9ca733239eb2e5b82e3cba0311e7ed0d89b045a8a099697889f5febff50384d845600f4142936160106f60ff233961a5e143c363a839ee45 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.27.7/kubernetes-client-windows-arm64.tar.gz) | 993f365214cd7a66284f4e6612681c2d6bf4844717c4d990ab1baf21a8d03eb94d7ab591c1eb584389fd6985867e14ed61400b74a02935da6b8b0ec34284e8e4 ### Server Binaries filename | sha512 hash -------- | ----------- [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.7/kubernetes-server-linux-amd64.tar.gz) | 8dc92641fe27bf7dcb2688a5c48cc0ff1b91b913d1f2eeac35b5bcbabc8413f768c23955141bdf707040b5ad0de55bbb7e407b4eed3d9d26c1e1c9e3acdb409b [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.7/kubernetes-server-linux-arm64.tar.gz) | a554b9319897c4cc65d8e4ca32b83103a71cf4512fb187a7f2b85898e4d10618c17b26ff0aadd8e265be009b215f100de4ab0b14a77b3b309b0786f03d479eb5 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.7/kubernetes-server-linux-ppc64le.tar.gz) | 001fc616801c239bddc02789ec9b9cd765a635d3f2d6cd04086b42eb81200bc9d0904e0a9e5b72756c420e36ce244169532c01cefd60f423eaaa85e236f06a49 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.7/kubernetes-server-linux-s390x.tar.gz) | a44f54c8fedea4e52e205831594f72f63d5c61cde0b9fa0ad6939eddf83664385edf767591187e166178a9af6b40da40607f26e098cb24bc2e5a88f7105318d3 ### Node Binaries filename | sha512 hash -------- | ----------- [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.7/kubernetes-node-linux-amd64.tar.gz) | ee31a3ad00412e122aacb82070a3257d558cd52e270312af538d9e7d22ad1638b71d9e02dddebf0b853c911284e172d7a16c0927c0e2012f761219850c0950aa [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.7/kubernetes-node-linux-arm64.tar.gz) | 95567ed1c5b892ee47f147d6ca8faef5e7915dfe34dc17141fa01326b4d0db0a8ff2e6589f681f9df5145b91878054c2b1e5030012d43500e4d525d28d3cb97b [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.7/kubernetes-node-linux-ppc64le.tar.gz) | 37cef5fb54c7a2ba8f542356ada66183bf281df41c989616694c87d06156aff241c03b7005c288dafad1889a5f989bd583bdf18039df9e31a874a71d4e5d9316 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.7/kubernetes-node-linux-s390x.tar.gz) | 414a71046b78be9937225396bccd747d17aba14890b77da672272129dd6b7675ae3522237388436ae79026c1837741af45e3c239c50149a4f5c16bd01066e1a9 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.7/kubernetes-node-windows-amd64.tar.gz) | df34997bfe6e5f2526adc73680988cf6e8670efd15efb5c60cca0c3dc8e384eb0bbf85c0a5ffa8edb93f6d129dd58531ab00830e511603e4ded87c18fb8a60d3 ### Container Images All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures ---- | ------------- [registry.k8s.io/conformance:v1.27.7](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) [registry.k8s.io/kube-apiserver:v1.27.7](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) [registry.k8s.io/kube-controller-manager:v1.27.7](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) [registry.k8s.io/kube-proxy:v1.27.7](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) [registry.k8s.io/kube-scheduler:v1.27.7](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) ## Changelog since v1.27.6 ## Changes by Kind ### Feature - Kubernetes is now built with Go 1.20.10 ([#121152](https://github.com/kubernetes/kubernetes/pull/121152), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing] ### Failing Test - E2e framework: retrying after intermittent apiserver failures was fixed in WaitForPodsResponding ([#120559](https://github.com/kubernetes/kubernetes/pull/120559), [@pohly](https://github.com/pohly)) [SIG Testing] ### Bug or Regression - Adds an opt-in mitigation for http/2 DOS vulnerabilities for CVE-2023-44487 and CVE-2023-39325 for the API server when the client is unauthenticated. The mitigation may be enabled by setting the `UnauthenticatedHTTP2DOSMitigation` feature gate to `true` (it is disabled by default). An API server fronted by an L7 load balancer that already mitigates these http/2 attacks may choose not to enable the kube-apiserver mitigation to avoid disrupting load balancer → kube-apiserver connections if http/2 requests from multiple clients share the same backend connection. An API server on a private network may choose not to enable the kube-apiserver mitigation to prevent performance regressions for unauthenticated clients. Authenticated requests rely on the fix in golang.org/x/net v0.17.0 alone. https://issue.k8s.io/121197 tracks further mitigation of http/2 attacks by authenticated clients. ([#121199](https://github.com/kubernetes/kubernetes/pull/121199), [@enj](https://github.com/enj)) [SIG API Machinery] - Fix a bug in cronjob controller where already created jobs may be missing from the status. ([#120649](https://github.com/kubernetes/kubernetes/pull/120649), [@andrewsykim](https://github.com/andrewsykim)) [SIG Apps] - Fixed a 1.27.4 regression where kube-controller-manager can crash when StatefulSet with Parallel policy and PVC labels is scaled up. ([#121185](https://github.com/kubernetes/kubernetes/pull/121185), [@aleksandra-malinowska](https://github.com/aleksandra-malinowska)) [SIG Apps] - Fixed an issue to not drain all the pods in a namespace when an empty-selector i.e. "{}" is specified in a Pod Disruption Budget (PDB) ([#121132](https://github.com/kubernetes/kubernetes/pull/121132), [@sairameshv](https://github.com/sairameshv)) [SIG Apps] - Fixed attaching volumes after detach errors. Now volumes that failed to detach are not treated as attached, Kubernetes will make sure they are fully attached before they can be used by pods. ([#120595](https://github.com/kubernetes/kubernetes/pull/120595), [@jsafrane](https://github.com/jsafrane)) [SIG Apps and Storage] - Fixes a bug where Services using finalizers may hold onto ClusterIP and/or NodePort allocated resources for longer than expected if the finalizer is removed using the status subresource ([#120655](https://github.com/kubernetes/kubernetes/pull/120655), [@aojea](https://github.com/aojea)) [SIG Network and Testing] - Fixes bug where OpenAPIV2 config was used instead of V3, and gives clear error message about OpenAPIV3 requirement ([#120612](https://github.com/kubernetes/kubernetes/pull/120612), [@alexzielenski](https://github.com/alexzielenski)) [SIG API Machinery] - Fixes creationTimestamp: null causing unnecessary writes to etcd ([#116865](https://github.com/kubernetes/kubernetes/pull/116865), [@alexzielenski](https://github.com/alexzielenski)) [SIG API Machinery and Testing] - Revised the logic for DaemonSet rolling update to exclude nodes if scheduling constraints are not met. This eliminates the problem of rolling updates to a DaemonSet getting stuck around tolerations. ([#120786](https://github.com/kubernetes/kubernetes/pull/120786), [@mochizuki875](https://github.com/mochizuki875)) [SIG Apps and Testing] - Sometimes, the scheduler incorrectly placed a pod in the "unschedulable" queue instead of the "backoff" queue. This happened when some plugin previously declared the pod as "unschedulable" and then in a later attempt encounters some other error. Scheduling of that pod then got delayed by up to five minutes, after which periodic flushing moved the pod back into the "active" queue. ([#120334](https://github.com/kubernetes/kubernetes/pull/120334), [@pohly](https://github.com/pohly)) [SIG Scheduling] ### Other (Cleanup or Flake) - Etcd: update to v3.5.9 ([#118079](https://github.com/kubernetes/kubernetes/pull/118079), [@nikhita](https://github.com/nikhita)) [SIG API Machinery, Cloud Provider, Cluster Lifecycle and Testing] - Fixes an issue where the vsphere cloud provider will not trust a certificate if: - The issuer of the certificate is unknown (x509.UnknownAuthorityError) - The requested name does not match the set of authorized names (x509.HostnameError) - The error surfaced after attempting a connection contains one of the substrings: "certificate is not trusted" or "certificate signed by unknown authority" ([#120767](https://github.com/kubernetes/kubernetes/pull/120767), [@MadhavJivrajani](https://github.com/MadhavJivrajani)) [SIG Architecture and Cloud Provider] - Kubernetes is now built with Go 1.20.9 ([#121024](https://github.com/kubernetes/kubernetes/pull/121024), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing] - Set the resolution for the job_controller_job_sync_duration_seconds metric from 4ms to 1min ([#120668](https://github.com/kubernetes/kubernetes/pull/120668), [@mimowo](https://github.com/mimowo)) [SIG Apps and Instrumentation] ## Dependencies ### Added _Nothing has changed._ ### Changed - github.com/vmware/govmomi: [v0.30.0 → v0.30.6](https://github.com/vmware/govmomi/compare/v0.30.0...v0.30.6) - golang.org/x/crypto: v0.1.0 → v0.14.0 - golang.org/x/net: v0.8.0 → v0.17.0 - golang.org/x/sys: v0.6.0 → v0.13.0 - golang.org/x/term: v0.6.0 → v0.13.0 - golang.org/x/text: v0.8.0 → v0.13.0 ### Removed _Nothing has changed._ # v1.27.6 ## Downloads for v1.27.6 ### Source Code filename | sha512 hash -------- | ----------- [kubernetes.tar.gz](https://dl.k8s.io/v1.27.6/kubernetes.tar.gz) | cbd2427412bbd229439afcf7d7b1712536515f93657c9971b66a430d9858a54383db33046eade4ce2891f26f264cccee5ad34ca6e03e874af8557c840b7c627d [kubernetes-src.tar.gz](https://dl.k8s.io/v1.27.6/kubernetes-src.tar.gz) | 006c3d901f3bf417ef5472708aaa47fc857bef6c7e4fe9e89693ec5b9040f06c5b44be3b74130b12a5bd17650855cb101263033100e0d46275eafd489924987b ### Client Binaries filename | sha512 hash -------- | ----------- [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.27.6/kubernetes-client-darwin-amd64.tar.gz) | b47b6213859d1397b55a0000d1c8daa1f630dd8a5ca553fcb541f4645fb01409f2ae2dd85a8c5274f5ec99cd5082de366e036cae72f8716861f83f8e38652fd2 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.27.6/kubernetes-client-darwin-arm64.tar.gz) | 4e1c6db712de967f6d9dc1c21b2fedf99aae3671242a8559c2bad43dc45e22e399e51bf606d2f6b4a943098fb61cf588324b6c9ba5615d18733d319183ed2c10 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.27.6/kubernetes-client-linux-386.tar.gz) | 992074087f3df0681ead508b8f35b173caac235813fe6db778b2e1367ae64cca9ff6a0d47bfae0c90612ac83361385ba82a45d01fc7b86f94752e7df32d006f7 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.6/kubernetes-client-linux-amd64.tar.gz) | 01e57c5d2c92094df17ce079a8944df73a2834362f3e9b051b1e3923b51b9f02bba7f4aab8ffd3183a0f99cfc102bc61e3dde77142e36f428612cced55d33892 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.27.6/kubernetes-client-linux-arm.tar.gz) | 08a51b07f14aaf4f7e9c6a3fa287c70b34195fcf7ca38968ef97cf6f70e9ab7ea17c5724fb94734a8a11ef8254122d8d246a8b777f95c1c753d430d4b64bb0ea [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.6/kubernetes-client-linux-arm64.tar.gz) | fb0cbe29113ba495c45843bb98e644570bf6284c2b689327d1a07aa5cd8d336f997a3f80ce0fd6df16c3a706b511922b13610b86632f07506032e51f0161479e [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.6/kubernetes-client-linux-ppc64le.tar.gz) | 81670992dcd1944881c54c24ef936c8fbbb7f01e7abd2c253f1894137cb114b60845e5a5181ad826c08074196f8c8bd9077322b43b26b72b0b06cee4c296dd45 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.6/kubernetes-client-linux-s390x.tar.gz) | e6e094117f04341bb704fae663eb95efe7273a6752acad1f82ed4dbf830d70eb204b9516dc5cfaea69dc31f979342123eaf7ec64dd88998479d8b06ab9851af5 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.27.6/kubernetes-client-windows-386.tar.gz) | bf869d6c20f1ea94dcfb0240b7d20a9ef5a021eab1f6ab6a840869af6ef420f9fe0cbf38f79a5d9055d27b8dc595cfb6587fad56259bbfb11f5b07a321a6ba9b [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.6/kubernetes-client-windows-amd64.tar.gz) | e503aaade44f3b6edfa532399640dc921e38ff5f0b3b6a71c347dd447065fa74d2d9d683de5325febf1de39737e3d1a8ed07781870d072d85f0223635729bcab [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.27.6/kubernetes-client-windows-arm64.tar.gz) | bfc41e7401ab7ad300a305603130ae7dbcf200c4370bea886d069d2ff4694fcb6cd6772849015c2259b693d90f85f575f7d3ecca54d31cf3a9b53569abac498e ### Server Binaries filename | sha512 hash -------- | ----------- [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.6/kubernetes-server-linux-amd64.tar.gz) | 27d62d0bc05d2f117427004a19e5d319337169c0219cc2055f519af4c356001dd5a37253708e835d4a88f14823adb555c4e2753347d739fece47e76337210ef8 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.6/kubernetes-server-linux-arm64.tar.gz) | a5dc93ca776751a54d4d2fc33b4ad2e3f4cd15610e0e38592a08e82bd3eb3e99852e17a94c002665b60ddd5433cbd1d04eb25d0763117d907595ab620ab9f885 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.6/kubernetes-server-linux-ppc64le.tar.gz) | 3f6aeb89e8455675ba638014a4625ca9a491b557c1b23e59af3c2b917ec4b671960b1180e2318e44d4aa01e9c28bb3394853edd6e41dd4f83d2bff9cb84de676 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.6/kubernetes-server-linux-s390x.tar.gz) | 517ac5b1ad72abf3043e3841aca7ff5b3b26206f3c7c37401a25fa0c74437dcfb0b6cb053bd64a58b7d062821c82156f66190defe5775d2eab7d9b2912f70b99 ### Node Binaries filename | sha512 hash -------- | ----------- [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.6/kubernetes-node-linux-amd64.tar.gz) | d93e9d2995fb9ff0a850c6217c8d47dc55163cc36a87e4820cd1303ae05f8f0be49fedb1a6f193812a5da49e26b0e9fed4c8f3f8856eda6164dfe11ca339b796 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.6/kubernetes-node-linux-arm64.tar.gz) | d634fb1eb577b999179212055fdef04a270e74ed3f5b765ad94dadc6349dc091f69d66b99f12a119b576171a21d2cd7bb4ca4e00c006e782b987764e854e80b3 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.6/kubernetes-node-linux-ppc64le.tar.gz) | d81c05c511c0d1a0e23e9e31e7ca7d5cf2ae854d6e78360cbeb8a7af929796b4f446199c0bad095bf52b31b9541aa8159694e983a757a6a5416b3eb4e75e8404 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.6/kubernetes-node-linux-s390x.tar.gz) | 22a0e8689d26c6b5fd9778930f430f7633a17e4956bc23de33b7fe22afaac1b24aa4422c59c549a285c9e4ca17f3ee12bc8d221eaaf98627816bad815ff3bd1d [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.6/kubernetes-node-windows-amd64.tar.gz) | 566c0848f94e30ee2ef287a83a4d82f6fe93f582e4a6db9de884fdfe33c941a4731c905206d251d9b3e096175198fd18dd211fee34f151e3c651443ecce09b3d ### Container Images All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures ---- | ------------- [registry.k8s.io/conformance:v1.27.6](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) [registry.k8s.io/kube-apiserver:v1.27.6](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) [registry.k8s.io/kube-controller-manager:v1.27.6](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) [registry.k8s.io/kube-proxy:v1.27.6](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) [registry.k8s.io/kube-scheduler:v1.27.6](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) ## Changelog since v1.27.5 ## Changes by Kind ### API Change - Fixed a bug where CEL expressions in CRD validation rules would incorrectly compute a high estimated cost for functions that return strings, lists or maps. The incorrect cost was evident when the result of a function was used in subsequent operations. ([#119809](https://github.com/kubernetes/kubernetes/pull/119809), [@jpbetz](https://github.com/jpbetz)) [SIG API Machinery, Auth and Cloud Provider] - Mark Job onPodConditions as optional in pod failure policy ([#120209](https://github.com/kubernetes/kubernetes/pull/120209), [@mimowo](https://github.com/mimowo)) [SIG API Machinery and Apps] ### Feature - Kubernetes is now built with Go 1.20.8 ([#120494](https://github.com/kubernetes/kubernetes/pull/120494), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing] ### Bug or Regression - Fix OpenAPI v3 not being cleaned up after deleting APIServices ([#120108](https://github.com/kubernetes/kubernetes/pull/120108), [@tnqn](https://github.com/tnqn)) [SIG API Machinery and Testing] - Fix a concurrent map access in TopologyCache's `HasPopulatedHints` method. ([#120323](https://github.com/kubernetes/kubernetes/pull/120323), [@Miciah](https://github.com/Miciah)) [SIG Apps and Network] - Fixed a 1.26 regression scheduling bug by ensuring that preemption is skipped when a PreFilter plugin returns `UnschedulableAndUnresolvable` ([#119952](https://github.com/kubernetes/kubernetes/pull/119952), [@sanposhiho](https://github.com/sanposhiho)) [SIG Scheduling] - Fixed a 1.27 scheduling regression that PostFilter plugin may not function if previous PreFilter plugins return Skip ([#119943](https://github.com/kubernetes/kubernetes/pull/119943), [@Huang-Wei](https://github.com/Huang-Wei)) [SIG Scheduling and Testing] - Fixed a regression in default 1.27 configurations in kube-apiserver: fixed the AggregatedDiscoveryEndpoint feature (beta in 1.27+) to successfully fetch discovery information from aggregated API servers that do not check `Accept` headers when serving the `/apis` endpoint ([#120360](https://github.com/kubernetes/kubernetes/pull/120360), [@Jefftree](https://github.com/Jefftree)) [SIG API Machinery] - Fixes a bug where images pinned by the container runtime can be garbage collected by kubelet. ([#120054](https://github.com/kubernetes/kubernetes/pull/120054), [@ruiwen-zhao](https://github.com/ruiwen-zhao)) [SIG Node] - Fixes a regression exposed in 1.27 by kubectl switching to openapi v3 by making apiregistration.k8s.io discoverable in openapi/v3 ([#119841](https://github.com/kubernetes/kubernetes/pull/119841), [@atiratree](https://github.com/atiratree)) [SIG API Machinery] - Fixes a regression exposed in 1.27 by kubectl switching to openapi v3 by resolving a flake in openapi v3 aggregation ([#119839](https://github.com/kubernetes/kubernetes/pull/119839), [@atiratree](https://github.com/atiratree)) [SIG API Machinery] - Fixes issue https://github.com/kubernetes-sigs/cloud-provider-azure/issues/4230 and removes the additional filtering on `NotReady` nodes by the azure cloud provider code ([#119128](https://github.com/kubernetes/kubernetes/pull/119128), [@alexanderConstantinescu](https://github.com/alexanderConstantinescu)) [SIG Cloud Provider] - Fixes regression in 1.27.2 causing running pods with devices to be terminated if kubelet is restarted ([#119432](https://github.com/kubernetes/kubernetes/pull/119432), [@ffromani](https://github.com/ffromani)) [SIG Node and Testing] - Ignore context canceled from validate and mutate webhook ([#120020](https://github.com/kubernetes/kubernetes/pull/120020), [@divyasri537](https://github.com/divyasri537)) [SIG API Machinery] - Kubeadm: fix nil pointer when etcd member is already removed ([#120011](https://github.com/kubernetes/kubernetes/pull/120011), [@neolit123](https://github.com/neolit123)) [SIG Cluster Lifecycle] ### Other (Cleanup or Flake) - When retrieving event resources, the reportingController and reportingInstance fields in the event will contain values. ([#120067](https://github.com/kubernetes/kubernetes/pull/120067), [@HirazawaUi](https://github.com/HirazawaUi)) [SIG Instrumentation] ## Dependencies ### Added _Nothing has changed._ ### Changed - github.com/google/cel-go: [v0.12.6 → v0.12.7](https://github.com/google/cel-go/compare/v0.12.6...v0.12.7) ### Removed _Nothing has changed._ # v1.27.5 ## Downloads for v1.27.5 ### Source Code filename | sha512 hash -------- | ----------- [kubernetes.tar.gz](https://dl.k8s.io/v1.27.5/kubernetes.tar.gz) | c38254c54938b816edbbbfb104846e5802500b09029719cda914cde334d4372f56a9ad70d01cdcb2983c06b3386cb6af01c04b26dec5e9b51bee772989826fd9 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.27.5/kubernetes-src.tar.gz) | 1e06ed46e530a8fa4cfd928e22008cfdc804473867fcf55c5304277fd36c1265069473a4a4d36ca1f53d1db4c742a7e3823f0910dab82ab82518c4e4d1bc7932 ### Client Binaries filename | sha512 hash -------- | ----------- [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.27.5/kubernetes-client-darwin-amd64.tar.gz) | 62dfc1d11fca2a2cc5b39d72233c94846af57a476984c7cac725f74dd6e3f3a5483de4b910d5c1becacf9ae33aef06de70f78f727c1b5114cd3a92ab120595b0 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.27.5/kubernetes-client-darwin-arm64.tar.gz) | a209d4533602b7fb49d9f850976de26d71b4936b1669726052c22842842e96a402a36ec85dd189bdb367b780f761a41c6272652907b1e7df128fb6bbcb7ea1ca [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.27.5/kubernetes-client-linux-386.tar.gz) | 71e5a5f26ca4b005582189ec9b6711a3e59197e9df268c6cd85c146ae042d97da82a41254df21bfcee2187939dc7a2a413db9ebd228e2a9d1e91f3a244c69d8b [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.5/kubernetes-client-linux-amd64.tar.gz) | 82ed21532b842d2da029eb7d2cbf0630619051d278034493c48b98b1149175f78d80cc8fcba79658384cdc6ed4b236aed1fc8dbe69fd47a0c7811a2f4e54369e [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.27.5/kubernetes-client-linux-arm.tar.gz) | a368c4275045b6a5a7efaa3adf18a8488ca728c689d5d4d0e0d562dd9046fdd3eceb1104b1f2a3f27b9fe1bf7006d5dd11294ee8d3c2468a51fe0c30bac1f0d3 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.5/kubernetes-client-linux-arm64.tar.gz) | 3631bea44d8e745035b044bddb3cb9a22002a61045365ea5485070e90501371ccf249ab6b83a2bc5188cc05a9b5c2adb35da2651ddf024a295fe7f584c56dd70 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.5/kubernetes-client-linux-ppc64le.tar.gz) | 9ca26442c15406e15813ff76a293afbc01b051ee2f5db29a415ff0a6daf9ec4186e0044f8a6cb410d22998167b393b8b65bc3a47a2ac57da44dbb25b4dec6d31 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.5/kubernetes-client-linux-s390x.tar.gz) | 1d39dbaae47cb7b8677010a905896461068ac408d17bfe401114ef08d39fd73affb115d5a86b0ec2fb98d0e6ee3a499460a0f874bc8c998b29346cf46c217712 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.27.5/kubernetes-client-windows-386.tar.gz) | a75f574826b613b71de6b4057ef7e7f2fd7c08053c7f973680c0b96e0659d75baeb34b491c9a0d877477688021b77719d270afe480b590b5c0cb60f834633586 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.5/kubernetes-client-windows-amd64.tar.gz) | fef167cba4f3f6793ca2a70ac33d24e0fae859fdf7eb78cffcd7ea1693bc4ba400c7f7244d1b4d124ddc67b5439bd3ac46b3a887703d6db7be28b553cb028222 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.27.5/kubernetes-client-windows-arm64.tar.gz) | 19583b45d2affba34ac1b3bf7c40fee86591d4f0a06710ea88da5a6345ad32b4ca283e16a06b88af37ecceed78b58b3cc716e70967a35c2a16a018a31848e9c7 ### Server Binaries filename | sha512 hash -------- | ----------- [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.5/kubernetes-server-linux-amd64.tar.gz) | d135dcd85ee02b2e39f5b08e97bc335c1a79f3c98ad17848de258d842c476c9f779c00b32763e99191e7a45eb2c4be02d87efa2ed38c304a49d91fabebb0eb6a [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.5/kubernetes-server-linux-arm64.tar.gz) | 2040380ddaac3039c15b10ae8474f677ecda83fd5489c7d52772038b8b377026f20ecf48998c2b33b355ff541702a896ef71154d935fd4f11f5a6d0c0177881b [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.5/kubernetes-server-linux-ppc64le.tar.gz) | d08827a2ade5407735177b245bb4660f5db3efd44bec14b7613e042aa8d011065548a626cd6af50090c5380384e6bcfb6d1fd21fcd1d2b3039480be634027754 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.5/kubernetes-server-linux-s390x.tar.gz) | a3a01b9aa6d7b826eb0dc6de519d881bbf0273e3fbc62857a328fd23be37cb0749b812ac3a40a739e03ea02ef60808599832237a803770f773bfe277946060b9 ### Node Binaries filename | sha512 hash -------- | ----------- [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.5/kubernetes-node-linux-amd64.tar.gz) | 4560cd0ad15195e6752df67a1a079d49e2254aeef1713459549f13e9b922602e364a22208e9b3a1168a976648583c476c601d88e08dcc8dfeca7bf3955325879 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.5/kubernetes-node-linux-arm64.tar.gz) | 83ec9e500d6a63c646fc488eee0cd5381d295616e0b49ad8e702d0bede8cc163184a77a50817b0b29b949aa25da99ef702d285b39844a92534f513599d1beb86 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.5/kubernetes-node-linux-ppc64le.tar.gz) | 0610be236df7fb50ec4fea5eda50d9d491f174ad9c0d4eff1968501258f69a8059b6d165eed0be8637d86649a5e23a24084916366c95d5b2f27c8c7c13fd24eb [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.5/kubernetes-node-linux-s390x.tar.gz) | 6bf0a266eb9a73800455380c1692e2b630042762a619514e257d1c672f3b6146f3aaf3711e3392802ed0565139819924ccd998c054720a305d8c65c70bd5595b [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.5/kubernetes-node-windows-amd64.tar.gz) | d0476c2cc08472aa73ca921167ed5849b072933553b5e076d6eae86b9a6c0e10816321cba0a5ca0cb51159b2958213c26a2a5c7a518474968ec21d06f425d640 ### Container Images All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures ---- | ------------- [registry.k8s.io/conformance:v1.27.5](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) [registry.k8s.io/kube-apiserver:v1.27.5](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) [registry.k8s.io/kube-controller-manager:v1.27.5](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) [registry.k8s.io/kube-proxy:v1.27.5](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) [registry.k8s.io/kube-scheduler:v1.27.5](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) ## Changelog since v1.27.4 ## Important Security Information This release contains changes that address the following vulnerabilities: ### CVE-2023-3955: Insufficient input sanitization on Windows nodes leads to privilege escalation A security issue was discovered in Kubernetes where a user that can create pods on Windows nodes may be able to escalate to admin privileges on those nodes. Kubernetes clusters are only affected if they include Windows nodes. **Affected Versions**: - kubelet <= v1.28.0 - kubelet <= v1.27.4 - kubelet <= v1.26.7 - kubelet <= v1.25.12 - kubelet <= v1.24.16 **Fixed Versions**: - kubelet v1.28.1 - kubelet v1.27.5 - kubelet v1.26.8 - kubelet v1.25.13 - kubelet v1.24.17 This vulnerability was discovered by James Sturtevant @jsturtevant and Mark Rossetti @marosset during the process of fixing CVE-2023-3676 (that original CVE was reported by Tomer Peled @tomerpeled92) **CVSS Rating:** High (8.8) [CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H](https://www.first.org/cvss/calculator/3.1#CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H) ### CVE-2023-3676: Insufficient input sanitization on Windows nodes leads to privilege escalation A security issue was discovered in Kubernetes where a user that can create pods on Windows nodes may be able to escalate to admin privileges on those nodes. Kubernetes clusters are only affected if they include Windows nodes. **Affected Versions**: - kubelet <= v1.28.0 - kubelet <= v1.27.4 - kubelet <= v1.26.7 - kubelet <= v1.25.12 - kubelet <= v1.24.16 **Fixed Versions**: - kubelet v1.28.1 - kubelet v1.27.5 - kubelet v1.26.8 - kubelet v1.25.13 - kubelet v1.24.17 This vulnerability was reported by Tomer Peled @tomerpeled92 **CVSS Rating:** High (8.8) [CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H](https://www.first.org/cvss/calculator/3.1#CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H) ## Changes by Kind ### API Change - Aggregated discovery now returns `responseKind: {}` for resources which are missing group/version/kind information, to ensure compatibility with v0.26.0-v0.26.3 clients. ([#119835](https://github.com/kubernetes/kubernetes/pull/119835), [@liggitt](https://github.com/liggitt)) [SIG API Machinery and Testing] ### Feature - Kubeadm: generate CA certificates with a start time that is offset 5 minutes in the past relative to the current system time to workaround cases of clock desync. client-go: allow to set NotBefore in NewSelfSignedCACert() ([#119113](https://github.com/kubernetes/kubernetes/pull/119113), [@champtar](https://github.com/champtar)) [SIG API Machinery, Auth and Cluster Lifecycle] - Kubernetes is now built with Go 1.20.7 ([#119828](https://github.com/kubernetes/kubernetes/pull/119828), [@jeremyrickard](https://github.com/jeremyrickard)) [SIG Release and Testing] ### Bug or Regression - Fix Topology Aware Hints not working when the `topology.kubernetes.io/zone` label is added after Node creation - Fix a data race in TopologyCache when `AddHints` and `SetNodes` are called concurrently ([#117269](https://github.com/kubernetes/kubernetes/pull/117269), [@tnqn](https://github.com/tnqn)) [SIG Apps and Network] - Fix computing backoff delay when using Job pod failure policy, by including in the backoff delay calculation pod failures ignored from the backoffLimit counter. Also, compute the backoff delay more accurately for deleted pods. ([#119466](https://github.com/kubernetes/kubernetes/pull/119466), [@mimowo](https://github.com/mimowo)) [SIG Apps] - Fix: After a Node is down and take some time to get back to up again, the mount point of the evicted Pods cannot be cleaned up successfully. (#111933) Meanwhile Kubelet will print the log `Orphaned pod "xxx" found, but error not a directory occurred when trying to remove the volumes dir` every 2 seconds. (#105536) ([#116134](https://github.com/kubernetes/kubernetes/pull/116134), [@cvvz](https://github.com/cvvz)) [SIG Node and Storage] - Fixed kubelet startup getting stuck with `NewVolumeManagerReconstruction` feature enabled and a CSI volume present in /var/lib/kubelet/pods. ([#117804](https://github.com/kubernetes/kubernetes/pull/117804), [@jsafrane](https://github.com/jsafrane)) [SIG Node and Storage] - Revert kubelet prober metrics `pod` tag to include actual pod name ([#118549](https://github.com/kubernetes/kubernetes/pull/118549), [@a7i](https://github.com/a7i)) [SIG Node] - Update kube-apiserver's priority & fairness work estimator such that 'max seats' is MIN(0.15 x nominalCL, nominalCL / handSize) This fixes a bug where clients with requests using hand size x max seats greater than the nominal concurrency limit can starve other requests in the same priority level. ([#118601](https://github.com/kubernetes/kubernetes/pull/118601), [@andrewsykim](https://github.com/andrewsykim)) [SIG API Machinery] ## Dependencies ### Added _Nothing has changed._ ### Changed _Nothing has changed._ ### Removed _Nothing has changed._ # v1.27.4 ## Downloads for v1.27.4 ### Source Code filename | sha512 hash -------- | ----------- [kubernetes.tar.gz](https://dl.k8s.io/v1.27.4/kubernetes.tar.gz) | 9469a8dae4ac11764119c1a8f71b010ff431f92e80fbddf98b9c0c84f009ba85e0a4d5d0dfcb292a0815b707fd1ed21550f76b7dc79b1058cd7ed224e2b4f4cc [kubernetes-src.tar.gz](https://dl.k8s.io/v1.27.4/kubernetes-src.tar.gz) | e908723662d8a7c5dc26865420c2a41e27d68be4018e3a2d02987128930bc027692b37d3a8ea59e03948b4649bd24c21fe37e50bc73fe490af26752d221bfd4c ### Client Binaries filename | sha512 hash -------- | ----------- [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.27.4/kubernetes-client-darwin-amd64.tar.gz) | e83130d975116ad8274a0638b86f11917bc2b47dae84fca3a79f93384387de3e55dad846a4b357df0df3d8004cb4f948a2fdc7434bd6d6b40dc64922acc27734 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.27.4/kubernetes-client-darwin-arm64.tar.gz) | 323206960a0faec4382b5031f5c92ac24e8ff1276282fed57fd7d3c60b88a188e16e6dc99a4afbd41e671fe0bb56044e8d99a59ff962390b900fed33c4de832a [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.27.4/kubernetes-client-linux-386.tar.gz) | d282fbddf9e95f2d06aa1a6efa0e898c50abe2944c415848c47fd024672f6e0b704ff6df0bcef49df9fdbf940726ab8de10466211ca9476237cd565a24be25f0 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.4/kubernetes-client-linux-amd64.tar.gz) | 42ac0cdfb1d961cb14fbdf09370b0798a1cd687d576ded4c85a2574a2adc48e514c7df9c905f1e851fab2f69246efee73971fd39838351d3b194881cfa3b5409 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.27.4/kubernetes-client-linux-arm.tar.gz) | 935c9296d2d2c85862b357f94406876648ceca247e92acb99ba3fb2b98608687cc1ff88d201b4f1cddaa526ad026e96c7af1fccb6e52b7e5ba9307c913e31ed1 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.4/kubernetes-client-linux-arm64.tar.gz) | b9f1a90cb7011d872ab4773afeef752d5d726f33628acc7592978a7ec37b2afb085fc4ebdabd4f8cb25a990eb73289ffda3442d41e6393eadc90c696b7a80be9 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.4/kubernetes-client-linux-ppc64le.tar.gz) | e5cb8236308bb04287564285750673353dfed6d806f4266c36ad3b640bc0959e36bb9b5d65fa75937b73970abcc5571a1ac9af203093d978bc2331bcfa0969e8 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.4/kubernetes-client-linux-s390x.tar.gz) | 586c4c5e31762c9d52303ed86476d6b6e3df0aa590bc3676d50069502a4f3de0808eadb1e20cd18abaf148cb87d73aaa85ad92902f927b29e650d1f6534dc78b [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.27.4/kubernetes-client-windows-386.tar.gz) | e74259bcc5980ac370cbbcd91590cb36982c1af638fe6da7175c4ca3ab7d2686df1ca0d540fda6ef1d4c4978ac1f9c46a1a998852cf9a903ddc0be3852a82dbc [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.4/kubernetes-client-windows-amd64.tar.gz) | d5fc29cf85eceed250f366dafb182c516934008c99212a8f417eef414fbfc3bac715450e4e497a01a3c378e2bfe795ca34601e7684166b2cfcfab1422f911754 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.27.4/kubernetes-client-windows-arm64.tar.gz) | 0fc5de119e6d43958e24dec587b5253edcc9a38aa27478d220b1b231b1bd847fb59a18f27b0fb7305872d2a7ecbde11ff07058094d8f267c67c24ced0ea4af52 ### Server Binaries filename | sha512 hash -------- | ----------- [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.4/kubernetes-server-linux-amd64.tar.gz) | a200bb097870297c6fd8f60329958e5e21c45df277778812e6217b0130753cdedd1f03589ade6b944d9fabf0da19921c8a4e02d0f19a731d2fc7df9489f0d28e [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.4/kubernetes-server-linux-arm64.tar.gz) | 5584594ae1a221fb50eb372ca9d8d7f1a1fa8c89a1e946660d232aa0c2fe646d5bcf8c123e874b76111768efff6192ac943143a547669ccc85579b32edf0a08f [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.4/kubernetes-server-linux-ppc64le.tar.gz) | d9285e3e483d0fd4a1ff5acbe86a7a2ccc206dc005ee09120c8eea16c5311b97fcf26bd40b2ff4b30bc924f284cec59e44fd733cf0b33ac83fe2c017bc9025c3 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.4/kubernetes-server-linux-s390x.tar.gz) | e36e5ab5e6b98ec84a96f1846e84784012c53b166bad7fc4fc61f5f4d030d384f2d25d007929cb5373474bae38f4ad35491600fc5a71f498801870603d9f9a1d ### Node Binaries filename | sha512 hash -------- | ----------- [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.4/kubernetes-node-linux-amd64.tar.gz) | 271633c9411066295941201cb861e2ea4759a212059786d5120e75bb7221f84b443afc631bd49f0957c1ec67e0a16f0308ddee061187afc75c1a1910968a59cc [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.4/kubernetes-node-linux-arm64.tar.gz) | 69ddbbbbb0c23ad7b86bbdbaaf5748e259978436730de1902ce047999b9c456a735ee92c63ba4616e0411ca99565efe4da727a6356485bbe2d66f0ef2b2516ea [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.4/kubernetes-node-linux-ppc64le.tar.gz) | c0758d63ade901e1fe6258cac4e5e951c7f8101a4a31ceb91f77ec0b2317f89fb2b4df64f672085dd03bc83fad27af4de9b886bc3de85f06e329e27d6e4f57bd [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.4/kubernetes-node-linux-s390x.tar.gz) | f3a2af663abd1e06f7ad828351fabe370f26ef8f1224b8a5738bbfd311df71d13ec72260e00e1e38bfe3dd1302ec7b5e9df13d8c8e874262f2b0607096deee75 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.4/kubernetes-node-windows-amd64.tar.gz) | bb05e9426d02f1a6593a14fef0671803a374dc6e98a4ff1de837b93559a1164275d5e313b08806ce09baa36d3177e59038de9487d6ee1d7a8ae898342e82ae89 ### Container Images All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures ---- | ------------- [registry.k8s.io/conformance:v1.27.4](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) [registry.k8s.io/kube-apiserver:v1.27.4](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) [registry.k8s.io/kube-controller-manager:v1.27.4](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) [registry.k8s.io/kube-proxy:v1.27.4](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) [registry.k8s.io/kube-scheduler:v1.27.4](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) ## Changelog since v1.27.3 ## Changes by Kind ### Feature - Fixes the alpha `CloudDualStackNodeIPs` feature. ([#118329](https://github.com/kubernetes/kubernetes/pull/118329), [@danwinship](https://github.com/danwinship)) [SIG Network and Node] - Kubernetes is now built with Go 1.20.6 ([#119366](https://github.com/kubernetes/kubernetes/pull/119366), [@xmudrii](https://github.com/xmudrii)) [SIG Release and Testing] ### Bug or Regression - Fix component status calling etcd health endpoint over http which exposed kubernetes to the risk of complete watch starvation and is inconsistent with other etcd probing done by kube-apiserver. ([#118683](https://github.com/kubernetes/kubernetes/pull/118683), [@serathius](https://github.com/serathius)) [SIG API Machinery] - Fix cronjob controller handling of complex schedules, like "30 6-16/4 * * 1-5", for example ([#119139](https://github.com/kubernetes/kubernetes/pull/119139), [@kmala](https://github.com/kmala)) [SIG Apps] - Fix deletion of non-admissible pods that are deleted during Kubelet restart ([#118841](https://github.com/kubernetes/kubernetes/pull/118841), [@bobbypage](https://github.com/bobbypage)) [SIG Node and Testing] - Fixed #118052: nodeAffinity on pods can change prior to scheduling gates being removed even when podSpec.affinity is nil in the initial spec, this matches the 1.28 behavior to allow consistent integrators to be written. ([#118199](https://github.com/kubernetes/kubernetes/pull/118199), [@aleskandro](https://github.com/aleskandro)) [SIG Apps, Scheduling and Testing] - Fixed a performance issue where pods weren't created/deleted in parallel for a StatefulSet with podManagementPolicy: Parallel. ([#119096](https://github.com/kubernetes/kubernetes/pull/119096), [@aleksandra-malinowska](https://github.com/aleksandra-malinowska)) [SIG Apps] - Fixed vSphere cloud provider not to skip detach volumes from nodes at kube-controller-startup. ([#117243](https://github.com/kubernetes/kubernetes/pull/117243), [@jsafrane](https://github.com/jsafrane)) [SIG Cloud Provider] - Kubectl explain should correctly work for all resources ([#118876](https://github.com/kubernetes/kubernetes/pull/118876), [@atiratree](https://github.com/atiratree)) [SIG CLI] - Only declare Job as finished after removing all Pod finalizers to avoid orphan Pods ([#119160](https://github.com/kubernetes/kubernetes/pull/119160), [@alculquicondor](https://github.com/alculquicondor)) [SIG Apps and Testing] - The Daemonset controller creates replacements for terminal Pods, which can appear during VM preemptions or when using Pod finalizers ([#118911](https://github.com/kubernetes/kubernetes/pull/118911), [@alculquicondor](https://github.com/alculquicondor)) [SIG Apps and Testing] - The `pod_scheduling_duration_seconds` metrics won't consider the time when a Pod fails PreEnqueue (like being gated). ([#118821](https://github.com/kubernetes/kubernetes/pull/118821), [@helayoty](https://github.com/helayoty)) [SIG Scheduling] - This PR adds additional validation for endpoint ip configuration while iterating through queried endpoint list. ([#117226](https://github.com/kubernetes/kubernetes/pull/117226), [@princepereira](https://github.com/princepereira)) [SIG Network and Windows] - Updated cAdvisor to v0.47.2 - Fix metrics in cri-o when a container restarts ([#118797](https://github.com/kubernetes/kubernetes/pull/118797), [@harche](https://github.com/harche)) [SIG Node] ## Dependencies ### Added _Nothing has changed._ ### Changed - github.com/google/cadvisor: [v0.47.1 → v0.47.2](https://github.com/google/cadvisor/compare/v0.47.1...v0.47.2) - github.com/rogpeppe/go-internal: [v1.10.0 → v1.11.0](https://github.com/rogpeppe/go-internal/compare/v1.10.0...v1.11.0) ### Removed _Nothing has changed._ # v1.27.3 ## Downloads for v1.27.3 ### Source Code filename | sha512 hash -------- | ----------- [kubernetes.tar.gz](https://dl.k8s.io/v1.27.3/kubernetes.tar.gz) | dd9174340ea3773db4c8a19d0a958fa22046863f270e31add0b7fad07ec285c6f2d09d335121697f6cd879c91589204b73b6631100c95c027bddbdef391fe5e0 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.27.3/kubernetes-src.tar.gz) | b3ecffc433df2dcaea9287e45da74f23c784f18bf22a33b65a196563077b01ba61b73f5761c16ae723ae821f27644f98c22be69ea16b8b14deb8b6e35d35b018 ### Client Binaries filename | sha512 hash -------- | ----------- [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.27.3/kubernetes-client-darwin-amd64.tar.gz) | fb8f708135406f1ba9116d2fecd6fadcc2e5be7e56c78e0620840352e236178835e4e162b76c8b545faf02700347f01792c2b9c83c6e3819a020e3ba0aef7afe [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.27.3/kubernetes-client-darwin-arm64.tar.gz) | 1c5b9ba8d9e41b61d519ede27c7d65320ababa7ac922f4b425553922cb319d825e0ca8a4bb38440408c8f99395c981f682887e7a0d8e20217feafafc2cbfa75c [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.27.3/kubernetes-client-linux-386.tar.gz) | 6ccc8cce4c26eacd27499b20a0017b4059aa133438b26f0576828d57c1810095f5fae67ca023047d44cdc9535605cf95a084af509138087616cd8d56a7e5c2fd [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.3/kubernetes-client-linux-amd64.tar.gz) | 14339c478f3c2d169d2e9fc387a62b32ba86be6cc9d9fc73f50bfe3a2225bbefd5463a09f665155ec36b04e52676831e7b010a1fcb9158e7f10a55dde393502d [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.27.3/kubernetes-client-linux-arm.tar.gz) | d40738d863188768089cf14fe31c060f578182283f8d4070e279cdabd9eccaf5a8e33f20025281e394de95a76ddc4a9835a6d1eac2b131d64f137a8a5d4cd667 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.3/kubernetes-client-linux-arm64.tar.gz) | 79aa1ea8d763c5258d43499a55ed4c88cf9aec36353246c5de797bc399023ffcb110c4b69da419b085a259b8b7253de2956480e6734a9ce1edd674821756cb54 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.3/kubernetes-client-linux-ppc64le.tar.gz) | c9e89f6032853874dce84df467c9f8ade7800af2c7b29fb070628e16350ba49b086520903508c711f6970b3fe37a3e88065be705d8950c024c006929abec6459 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.3/kubernetes-client-linux-s390x.tar.gz) | 927f54417e2e177ba811b08db45677a7a5693c99da4eaa268fc2026d68769d49bb036c1b4dc2d376a2080f17f207399d9a1e83bcf17c747647bd86aa3e1bab77 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.27.3/kubernetes-client-windows-386.tar.gz) | 78e6a307237f66386856a217ed82a63e2d1f9a1b674fe1f5fd26d6169193490d0aaeb8597f2ac9ea876ff984ca51e4f6810cb6587aff3d6af4eaf1a4b8590330 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.3/kubernetes-client-windows-amd64.tar.gz) | 83252ef0c7491a637fc45b696c35cdafd3bd168608c1cb6fa0cc380b20dd68e456a63615d5b42074c3a8a5e1f8cc988e746793b598b5e658f1d598f8da29b2d4 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.27.3/kubernetes-client-windows-arm64.tar.gz) | c4f364d16a3fdf3885d2176beb058b85f8f1f603d25c0e87b2c97c122543c7f1f0f6c1d7377a145a3cf893ae3af864779f555f0bdfddfa8d6cf2a5d442a57e1f ### Server Binaries filename | sha512 hash -------- | ----------- [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.3/kubernetes-server-linux-amd64.tar.gz) | 4493a9c8e64579f0904c5dddb5967397f7ad9f8ec1d16d34eaa9692a716170f8392d414ee8d2adce0effdfc875bce5e70567df51d538d520eb4d284ff538ed3c [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.3/kubernetes-server-linux-arm64.tar.gz) | 06f0440bf4fdfe19a27d4ee5e2ae71df9d8697b37422133a487b15ddb3861fde4e705367374daa1b726adea432d4e23cab3477741583dd5f9fdc7ddc4633081b [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.3/kubernetes-server-linux-ppc64le.tar.gz) | 6886265f0839ecd83988dbe535d0c15613f828697b2a2948c79fcb5eabc7f4177b08d9e58a93c80d6f8fb2f463688ab043a4d0bf354f4bc410785d2d2e5b94a4 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.3/kubernetes-server-linux-s390x.tar.gz) | a7a595f914393f776809555f22831d7c5ca91d2de117b69f392ae6f7f3611199074abd373a973b5d4d5aaddfcce60c17ba9ecb8eeeeabfe0a4ccbc7c6f0735c7 ### Node Binaries filename | sha512 hash -------- | ----------- [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.3/kubernetes-node-linux-amd64.tar.gz) | 5950f42effc3b48697142cc6012de0921a575252e1eb860eb89941be7f22ef9005d590fd825f0b2e9025583a7912ecae10f3a848932d16981afc9a36dac4d337 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.3/kubernetes-node-linux-arm64.tar.gz) | 9dedf1d677a9a57d8ec28d09ae70ea9bf5deb4869a33e15391978347e7f1e9d51ac4fcd82e0411803670d84d161104575fd84e4ea90458c877b9247c748d12b4 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.3/kubernetes-node-linux-ppc64le.tar.gz) | 54c1018641d2ffb075fa6c3674a0c69a8509332f13db569c937c649c4c678db418512266d6373aca9892776fc090e060a51c1f6e54e4ab8018f44c3a6bfad4d4 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.3/kubernetes-node-linux-s390x.tar.gz) | 3d0e771b812c8aff468cced3bb29619cc15ee2f2bd5980c8bb5880391e0076eab54bfea6bdd77953ed929462024349ba5fc07edad85126ae5a6e81c399e8b333 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.3/kubernetes-node-windows-amd64.tar.gz) | 3614c4df2074714a74ce392f3f14ff91b15d97a7759cc33755c0c1f0b96e728cef04d8e55068911a64611a46cc40f0d25bee2ac974e4cc78b7428b2f1e1f5ecc ### Container Images All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures ---- | ------------- [registry.k8s.io/conformance:v1.27.3](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) [registry.k8s.io/kube-apiserver:v1.27.3](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) [registry.k8s.io/kube-controller-manager:v1.27.3](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) [registry.k8s.io/kube-proxy:v1.27.3](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) [registry.k8s.io/kube-scheduler:v1.27.3](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) ## Changelog since v1.27.2 ## Important Security Information This release contains changes that address the following vulnerabilities: ### CVE-2023-2728: Bypassing enforce mountable secrets policy imposed by the ServiceAccount admission plugin A security issue was discovered in Kubernetes where users may be able to launch containers that bypass the mountable secrets policy enforced by the ServiceAccount admission plugin when using ephemeral containers. The policy ensures pods running with a service account may only reference secrets specified in the service account's secrets field. Kubernetes clusters are only affected if the ServiceAccount admission plugin and the `kubernetes.io/enforce-mountable-secrets` annotation are used together with ephemeral containers. **Note**: This only impacts the cluster if the ServiceAccount admission plugin is used (most cluster should have this on by default as recommended in https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#serviceaccount), the `kubernetes.io/enforce-mountable-secrets` annotation is used by a service account (this annotation is not added by default), and Pods are using ephemeral containers. **Affected Versions**: - kube-apiserver v1.27.0 - v1.27.2 - kube-apiserver v1.26.0 - v1.26.5 - kube-apiserver v1.25.0 - v1.25.10 - kube-apiserver <= v1.24.14 **Fixed Versions**: - kube-apiserver v1.27.3 - kube-apiserver v1.26.6 - kube-apiserver v1.25.11 - kube-apiserver v1.24.15 **CVSS Rating:** Medium (6.5) [CVSS:3.1/AV:N/AC:L/PR:H/UI:N/S:U/C:H/I:H/A:N](https://www.first.org/cvss/calculator/3.1#CVSS:3.1/AV:N/AC:L/PR:H/UI:N/S:U/C:H/I:H/A:N) ## Changes by Kind ### Feature - Kubernetes is now built with Go 1.20.5 ([#118553](https://github.com/kubernetes/kubernetes/pull/118553), [@puerco](https://github.com/puerco)) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node, Release, Storage and Testing] ### Bug or Regression - Add DisruptionTarget condition to the pod preempted by Kubelet to make room for a critical pod ([#118219](https://github.com/kubernetes/kubernetes/pull/118219), [@mimowo](https://github.com/mimowo)) [SIG Node and Testing] - Fixes a bug at kube-apiserver start where APIService objects for custom resources could be deleted and recreated. ([#118104](https://github.com/kubernetes/kubernetes/pull/118104), [@liggitt](https://github.com/liggitt)) [SIG API Machinery and Testing] - If `kubeadm reset` finds no etcd member ID for the peer it removes during the `remove-etcd-member` phase, it continues immediately to other phases, instead of retrying the phase for up to 3 minutes before continuing. ([#117948](https://github.com/kubernetes/kubernetes/pull/117948), [@dlipovetsky](https://github.com/dlipovetsky)) [SIG Cluster Lifecycle] - Kubeadm: fix a bug where the static pod changes detection logic is inconsistent with kubelet ([#118069](https://github.com/kubernetes/kubernetes/pull/118069), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle] - Kubeadm: fix etc version support for Kubernetes v1.27 ([#118307](https://github.com/kubernetes/kubernetes/pull/118307), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle] ## Dependencies ### Added _Nothing has changed._ ### Changed _Nothing has changed._ ### Removed _Nothing has changed._ # v1.27.2 ## Downloads for v1.27.2 ### Source Code filename | sha512 hash -------- | ----------- [kubernetes.tar.gz](https://dl.k8s.io/v1.27.2/kubernetes.tar.gz) | c46c9c1c4cdb0b1532630ce0e01295c7185f725e494d4fd190bae0540283c679b1c8b0a1ad1f0f5d320ddbf439e5fdd6f925700080cdf810158e1f41b8c5d9c9 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.27.2/kubernetes-src.tar.gz) | b9be38a5506071a362864661b369c71b0a02e66df0a77a2afc68040fa9634751a189e3c6c94771aee3e17b50a73228ad992a08f31cf4b322ebd7003e7676d381 ### Client Binaries filename | sha512 hash -------- | ----------- [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.27.2/kubernetes-client-darwin-amd64.tar.gz) | 7e4c0a207e505f6966999e0efb293c0e885d5975ad02f8e534b60ab1a94e0fdbcefd72c18fd536bb23e9356735098dd3fb85c6034e3ec07879f9511deb254f1b [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.27.2/kubernetes-client-darwin-arm64.tar.gz) | 41a51f588a9c19d0377921b66c21d7b406244aca4fbd32c0d7dcdd9b1cf80712c8f26a1134c2ae1d57612025b943e87a7bb55ee01c22838c0deb2754cf4a43cd [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.27.2/kubernetes-client-linux-386.tar.gz) | 729310d48d34fd21805869f849492227d3a74d1feafa4969d2aa5e0336c85d51f379865eda7b20c92b2f5122094884de5947a715f2fc9b6cb32e8a4e79dcd16c [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.2/kubernetes-client-linux-amd64.tar.gz) | cc34cffb3ec65a1b29dc3998341c8317dd1bf34d45b230a2379b1676d4d9a600cb662cde7caa7c8253e4cf2320d40b9581f97c0a04ac81037643b4fd105c6103 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.27.2/kubernetes-client-linux-arm.tar.gz) | b85927b9ff2f5871dac6814800c0fda43c4e69c27dd5c9d5cb3c73551c147ad2501675ffc5775cb70af4b643e88e784d034c3eec714de66c7ddf163a2ae4f500 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.2/kubernetes-client-linux-arm64.tar.gz) | cdf09ad3150c702e84c22158e95f164cffcbfa5e06af65e33dafada0d0d00fd6c160f41eb72d7966b0659e49705f7197d92dc3dc7153cf907fcdf318071138bc [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.2/kubernetes-client-linux-ppc64le.tar.gz) | 7219c79d43cc57a0866c854183dbed2629866e4ce081b62eef6c3034094bf0d3143e9e2eb7cf819a2f49bd98566cf4ac56cc9f2989f4c49906a71e2df68767e4 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.2/kubernetes-client-linux-s390x.tar.gz) | 9a5c0d13732ebf2d69f714ee953cc57f1c7ca2a27cb26333336a0da225414a96976f6a383cbe89aee80bcdd47a59dc17784acce377f656a6f959ed65f638a82d [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.27.2/kubernetes-client-windows-386.tar.gz) | 7bad7610f5a000cf40f68451d4cf94395d43907271aa98132cb6a52eae541e25cce7f40b5dfb1b45c79da5bbf54ce49cfbcd04f819635e16d91e03c63b48b8f4 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.2/kubernetes-client-windows-amd64.tar.gz) | 3d1c4f023867e8289d19dba43d702d40f3a8d8583e2c436e447af43127da9b0e90b5ca4ac055c3256c92d8fcaaa3734f0a83039480b35a012aed86ecd377da59 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.27.2/kubernetes-client-windows-arm64.tar.gz) | 0b08b36d4869b6b1de0314bb365ae45f85719297088f12b723a535a61f7b2c648969c12a4e1ecbd29a6deb804551815ed21c3b8ae9ed6813aa26d625723a273e ### Server Binaries filename | sha512 hash -------- | ----------- [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.2/kubernetes-server-linux-amd64.tar.gz) | 53f1533aa8f493ebbdfb07ba59eaf971cf865c60d1ac9a5ad9f61e6d5f670e9d86e0dc70c6d3057953da2968d100de8d8bf50d5863ad2decb69c397aa6f185b9 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.2/kubernetes-server-linux-arm64.tar.gz) | 66466de2b1b5ad7ce09fba95da00d3451ae13b28b89755a64cc4e18e1254c5dfed290df5f8509f312396679bcc90eec98eb84e333ea9bd206ecb8bf00eeeba71 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.2/kubernetes-server-linux-ppc64le.tar.gz) | 283dd8c6391d62b1f11102ce3a252d78b1dd3268dd2b8c5f08276c9c764ced6f0f8e8056b5d302045c464efc063a81d815e6fc3f804b997770b40bc1b2a89f8c [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.2/kubernetes-server-linux-s390x.tar.gz) | ea30de775e794eb738a3c10c730c0e291ae1460fdaba984d4eb00bf52b552f192608e8213b382ac8161f1f13486ddf13b7f20e4f5838a1e38f08da9288c01a3c ### Node Binaries filename | sha512 hash -------- | ----------- [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.2/kubernetes-node-linux-amd64.tar.gz) | 2a05c3ebcec8bce9ca2a7c835617f0b85dbf11a07d39c5b002b5389a45465807c437dd22d975c60e680ee34f3bc00e460d848927a8fa2c1543ec97fb66f50477 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.2/kubernetes-node-linux-arm64.tar.gz) | 9d6c45fb54c01ac9a106d356d8c9ed1c6564f4b86bb1ba55ced628bbbf5c4fb1f78d68a15c3055680c6dff6c85a32726300d09b7a85bdcb7ba263a434b148826 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.2/kubernetes-node-linux-ppc64le.tar.gz) | b029fe744619f9649e42c80ca2f0bb14ae72c934d4687100ee7f041cbcd72cc5567ae01acef9e2c7b5c579228b2af9a419bdbf2af64420754a5f6049cdd391bb [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.2/kubernetes-node-linux-s390x.tar.gz) | d3319d9a4a205cd1fa9da590407fe2be3b149c50e77422ff2b2f1e803c24e0d496fdc89d16c658fbef7a0bc59d1b0e295dfa5354ce3c3c5d9a6749e60e1580ee [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.2/kubernetes-node-windows-amd64.tar.gz) | a745fe1b46ec6c3bd27e72c2774a01fd53c27361de6ad7281c2adeaf88ab59ec725d60dc99a4de5f3579428ef4e923860a85143f14b51590ce43bfdee7a36a10 ### Container Images All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures ---- | ------------- [registry.k8s.io/conformance:v1.27.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) [registry.k8s.io/kube-apiserver:v1.27.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) [registry.k8s.io/kube-controller-manager:v1.27.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) [registry.k8s.io/kube-proxy:v1.27.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) [registry.k8s.io/kube-scheduler:v1.27.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) ## Changelog since v1.27.1 ## Changes by Kind ### API Change - Added error handling for seccomp localhost configurations that do not properly set a localhostProfile ([#117020](https://github.com/kubernetes/kubernetes/pull/117020), [@cji](https://github.com/cji)) [SIG API Machinery and Node] - Fixed an issue where kubelet does not set case-insensitive headers for http probes. (#117182, @dddddai) ([#117324](https://github.com/kubernetes/kubernetes/pull/117324), [@dddddai](https://github.com/dddddai)) [SIG API Machinery, Apps and Node] - Revised the comment about the feature-gate level for PodFailurePolicy from alpha to beta ([#117815](https://github.com/kubernetes/kubernetes/pull/117815), [@kerthcet](https://github.com/kerthcet)) [SIG Apps] ### Feature - Kubernetes is now built with Go 1.20.4 ([#117773](https://github.com/kubernetes/kubernetes/pull/117773), [@xmudrii](https://github.com/xmudrii)) [SIG Release and Testing] ### Failing Test - Allow Azure Disk e2es to use newer topology labels if available from nodes ([#117216](https://github.com/kubernetes/kubernetes/pull/117216), [@gnufied](https://github.com/gnufied)) [SIG Storage and Testing] ### Bug or Regression - CVE-2023-27561 CVE-2023-25809 CVE-2023-28642: Bump fix runc v1.1.4 -> v1.1.5 ([#117242](https://github.com/kubernetes/kubernetes/pull/117242), [@haircommander](https://github.com/haircommander)) [SIG Node] - During device plugin allocation, resources requested by the pod can only be allocated if the device plugin has registered itself to kubelet AND healthy devices are present on the node to be allocated. If these conditions are not sattsfied, the pod would fail with `UnexpectedAdmissionError` error. ([#117719](https://github.com/kubernetes/kubernetes/pull/117719), [@swatisehgal](https://github.com/swatisehgal)) [SIG Node and Testing] - Fallback from OpenAPI V3 to V2 when the OpenAPI V3 document is invalid or incomplete. ([#117980](https://github.com/kubernetes/kubernetes/pull/117980), [@seans3](https://github.com/seans3)) [SIG CLI] - Fix bug where `listOfStrings.join()` in CEL expressions resulted in an unexpected internal error. ([#117596](https://github.com/kubernetes/kubernetes/pull/117596), [@jpbetz](https://github.com/jpbetz)) [SIG API Machinery] - Fix incorrect calculation for ResourceQuota with PriorityClass as its scope. ([#117825](https://github.com/kubernetes/kubernetes/pull/117825), [@Huang-Wei](https://github.com/Huang-Wei)) [SIG API Machinery] - Fix performance regression in scheduler caused by frequent metric lookup on critical code path. ([#117617](https://github.com/kubernetes/kubernetes/pull/117617), [@tosi3k](https://github.com/tosi3k)) [SIG Scheduling] - Fix: the volume is not detached after the pod and PVC objects are deleted ([#117236](https://github.com/kubernetes/kubernetes/pull/117236), [@cvvz](https://github.com/cvvz)) [SIG Storage] - Fixed a memory leak in the Kubernetes API server that occurs during APIService processing. ([#117310](https://github.com/kubernetes/kubernetes/pull/117310), [@enj](https://github.com/enj)) [SIG API Machinery] - Fixes a race condition serving OpenAPI content ([#117708](https://github.com/kubernetes/kubernetes/pull/117708), [@Jefftree](https://github.com/Jefftree)) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Instrumentation and Node] - Fixes a regression in kubectl and client-go discovery when configured with a server URL other than the root of a server. ([#117685](https://github.com/kubernetes/kubernetes/pull/117685), [@ardaguclu](https://github.com/ardaguclu)) [SIG API Machinery] - Fixes bug where an incomplete OpenAPI V3 document can cause a nil-pointer crash. Ensures fallback to OpenAPI V2 endpoint for errors retrieving OpenAPI V3 document. ([#117918](https://github.com/kubernetes/kubernetes/pull/117918), [@seans3](https://github.com/seans3)) [SIG CLI] - Kubeadm: fix a bug where file copy(backup) could not be executed correctly on Windows platform during upgrade ([#117861](https://github.com/kubernetes/kubernetes/pull/117861), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle] - Kubelet terminates pods correctly upon restart, fixing an issue where pods may have not been fully terminated if the kubelet was restarted during pod termination. ([#117433](https://github.com/kubernetes/kubernetes/pull/117433), [@bobbypage](https://github.com/bobbypage)) [SIG Node and Testing] - Number of errors reported to the metric `storage_operation_duration_seconds_count` for emptyDir decreased significantly because previously one error was reported for each projected volume created. ([#117022](https://github.com/kubernetes/kubernetes/pull/117022), [@mpatlasov](https://github.com/mpatlasov)) [SIG Storage] - Resolves a spurious "Unknown discovery response content-type" error in client-go discovery requests by tolerating extra content-type parameters in API responses ([#117637](https://github.com/kubernetes/kubernetes/pull/117637), [@seans3](https://github.com/seans3)) [SIG API Machinery] - Reverted NewVolumeManagerReconstruction and SELinuxMountReadWriteOncePod feature gates to disabled by default to resolve a regression of volume reconstruction on kubelet/node restart ([#117752](https://github.com/kubernetes/kubernetes/pull/117752), [@liggitt](https://github.com/liggitt)) [SIG Storage] - Static pods were taking extra time to be restarted after being updated. Static pods that are waiting to restart were not correctly counted in `kubelet_working_pods`. ([#116995](https://github.com/kubernetes/kubernetes/pull/116995), [@smarterclayton](https://github.com/smarterclayton)) [SIG Node] - [KCCM] service controller: change the cloud controller manager to make `providerID` a predicate when synchronizing nodes. This change allows load balancer integrations to ensure that the `providerID` is set when configuring load balancers and targets. ([#117450](https://github.com/kubernetes/kubernetes/pull/117450), [@alexanderConstantinescu](https://github.com/alexanderConstantinescu)) [SIG Cloud Provider and Network] ### Other (Cleanup or Flake) - A v2-level info log will be added, which will output the details of the pod being preempted, including victim and preemptor ([#117214](https://github.com/kubernetes/kubernetes/pull/117214), [@HirazawaUi](https://github.com/HirazawaUi)) [SIG Scheduling] - Structured logging of NamespacedName was inconsistent with klog.KObj. Now both use lower case field names and namespace is optional. ([#117238](https://github.com/kubernetes/kubernetes/pull/117238), [@pohly](https://github.com/pohly)) [SIG API Machinery, Architecture and Instrumentation] ## Dependencies ### Added _Nothing has changed._ ### Changed - github.com/opencontainers/runc: [v1.1.4 → v1.1.6](https://github.com/opencontainers/runc/compare/v1.1.4...v1.1.6) - k8s.io/kube-openapi: 15aac26 → 8b0f38b - sigs.k8s.io/apiserver-network-proxy/konnectivity-client: v0.1.1 → v0.1.2 ### Removed _Nothing has changed._ # v1.27.1 ## Downloads for v1.27.1 ### Source Code filename | sha512 hash -------- | ----------- [kubernetes.tar.gz](https://dl.k8s.io/v1.27.1/kubernetes.tar.gz) | 01e0cc312fa04847174bf63eb888b0247ee92788801b77d642480226b112be6e3989169b9a51209187878ac6dd9c7475fd6550f16dc34b8bc52ae2f0d82a8830 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.27.1/kubernetes-src.tar.gz) | e5b55fe1e6262c60cf14229ac6dabc12b6108c7c75a454aa53919fa779bd9566412bcb31122405faa7326a19b3506a2e21904225bae1d63b23c0479ef909324d ### Client Binaries filename | sha512 hash -------- | ----------- [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.27.1/kubernetes-client-darwin-amd64.tar.gz) | a85b973c88f5684d6ea73753b8c6b99b3e64e0cd2507db24b2bd7c25e22b55cf725c6e7ee80795884a34135434bb19b676e3572d587a13d2400ba7ddc5ae4935 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.27.1/kubernetes-client-darwin-arm64.tar.gz) | 83aa7154c08bbe92c6850e1c9669f6a074019ec42253b45cf0a29d58526ce7f3c3845f9504266c01cceef5013f3977eed6bd5c7a4926ee3433194c8ea8984e6f [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.27.1/kubernetes-client-linux-386.tar.gz) | 0723e0b8a1fdc6aea6028bc2f4e25addd20dca83060c12b8b7869dbd49e9eb852264957383dcf6069b542d4db7a2e3a5c5408091d58709d676afa899df25bce2 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.1/kubernetes-client-linux-amd64.tar.gz) | 98caa662a63d7f9ba36761caaf997be4d214ea2b921a4387965a67d168b52ea29ae9185de20192f7b4b9169a887beb19d22e5776ff0bb0b68907e177b11a8043 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.27.1/kubernetes-client-linux-arm.tar.gz) | 773171a17712112850332effaf64c901b9eace6b52c003eed79032178bbbbc2b9f7ed6725bf95533e574511c70e75cd03ff1b3a1e22bcc3871d50887f04b7121 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.1/kubernetes-client-linux-arm64.tar.gz) | 9dd09ca0738b1b7e287271cf16f5527d3a1abc7878e4259dc4208f74e82e3db16e5fed07a3dca75550ca531be0c504f6942841ab65e67687a94648e1b0e382a5 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.1/kubernetes-client-linux-ppc64le.tar.gz) | 976499635c9b7507350edeebff6d997bbff8c9f66aecea55d549eeab51b89bc3e415a703fc49a37f284f8180055d1d36d0722932d7d19729aabb6ed39c08b8ff [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.1/kubernetes-client-linux-s390x.tar.gz) | ad642b631eab5a68d8c88670f7880d6cafa2fdfb04e6e69e50a7c264b309ed2b04ff63b045f5133db9c3aecd99e7ac556577a74d57a64b705f3c6367d998c323 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.27.1/kubernetes-client-windows-386.tar.gz) | bc3a685dd07085119672a6ccabc2dd832a990cf50304eea8396771006ef0bcb1fc8e08b6db9d2f455dc64c5f3e9c48fdb2ff19ec7802f35257742465fbc083a7 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.1/kubernetes-client-windows-amd64.tar.gz) | 55d2a9e5ca066166592d778697d657eb04d828ffd6e15b177a399b44574dcae972d109e83bc2e33f0effe679e61538d5b1b4fea66ec1d812e678a9b1c50494fc [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.27.1/kubernetes-client-windows-arm64.tar.gz) | adc10b48334f2a4fae6d7ace0e1485b8a874f98187b3b5491ca056b25f9096b58a0e3e37f096c4e9045bd7c56aa7efb4d2544570153864288095ddf6d0515172 ### Server Binaries filename | sha512 hash -------- | ----------- [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.1/kubernetes-server-linux-amd64.tar.gz) | 0752a63510a6d0ae06d8e24e42faa1641e093c25061f60b4c8355f735788ddd4bd25ae2a47a796064b6eb2ea15f0d451852fddc3c1b82b0e1afd279df700cea2 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.1/kubernetes-server-linux-arm64.tar.gz) | 63d086447a7abcd37a7d4e8d36609dc1a8217e6a7d2f45efd6bf35444321bf96f181d621d7a12224e55034fae1ece62a6c3109204dfd3b40577afb7c61a3ead7 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.1/kubernetes-server-linux-ppc64le.tar.gz) | 0b477fb3af66976f7aafba0b97c089e5dda8c21fc314ba3fb2d54ed1ff453389af8d3aba6bbe14fdd8db02f1a309bff2291abbdbdd64b2174878bec6997a889b [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.1/kubernetes-server-linux-s390x.tar.gz) | bf198f3099663ead089c42221f60d4dee53c6ac0cefab37e316d2b6449a46a1708426782aff946f6cb735ab7141f61df1e33144115845aa277c45b1ea8741a6c ### Node Binaries filename | sha512 hash -------- | ----------- [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.1/kubernetes-node-linux-amd64.tar.gz) | 484993a2478aa7d734fe711789f661b74152a96eff540b883e8603a92d91b89d870c4bfd0b482e72fd0ab2a95b7766725a0e8627676e99d051db0bf7c97d5791 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.1/kubernetes-node-linux-arm64.tar.gz) | 1d0d9a393b513f32e35fd609fbdd61179402421674a51ceb25b6516f2d5a5d596a451a285a038e4fe31e175727b75e30ff2e9bb368a7aa82b363018634b55a6d [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.1/kubernetes-node-linux-ppc64le.tar.gz) | 00c40568cc8e255b557f9a69f704f393d9ad8cfce75aaadc4df5485551106064d62fa58a6bb8a2bacd4cc363c09424eeb0acc156fe7172df057df46f4cc57109 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.1/kubernetes-node-linux-s390x.tar.gz) | 0615a6be0ea7ca1c90418690633013872f5f56404a6eecfedb677276b2653e22f600ad3863c80b592ff7ff2107fc4ce407ab526793464abcb0f93b65d671f6dc [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.1/kubernetes-node-windows-amd64.tar.gz) | 87ab85b2d784124c945c4b92ff037ecdc12800176c22d7c06d38285386d32b3fc4fea0b00d6dcdd3db77f4b67123aaeb1dc62cca6c9ff381c0d2155ee4c1ec4e ### Container Images All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures ---- | ------------- [registry.k8s.io/conformance:v1.27.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) [registry.k8s.io/kube-apiserver:v1.27.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) [registry.k8s.io/kube-controller-manager:v1.27.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) [registry.k8s.io/kube-proxy:v1.27.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) [registry.k8s.io/kube-scheduler:v1.27.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) ## Changelog since v1.27.0 ## Changes by Kind ### Bug or Regression - Fixes a regression in 1.27.0 that resulted in "missing metadata in converted object" errors when modifying objects for multi-version custom resource definitions with a conversion strategy of `None`. ([#117305](https://github.com/kubernetes/kubernetes/pull/117305), [@ncdc](https://github.com/ncdc)) [SIG API Machinery] - Known issue: fixed that the PreEnqueue plugins aren't executed for Pods proceeding to activeQ through backoffQ. ([#117194](https://github.com/kubernetes/kubernetes/pull/117194), [@sanposhiho](https://github.com/sanposhiho)) [SIG Release and Scheduling] - Setting a mirror pod's phase to Succeeded or Failed can prevent the corresponding static pod from restarting due mutation of a Kubelet cache. ([#116482](https://github.com/kubernetes/kubernetes/pull/116482), [@smarterclayton](https://github.com/smarterclayton)) [SIG Node] ## Dependencies ### Added _Nothing has changed._ ### Changed _Nothing has changed._ ### Removed _Nothing has changed._ # v1.27.0 [Documentation](https://docs.k8s.io) ## Downloads for v1.27.0 ### Source Code filename | sha512 hash -------- | ----------- [kubernetes.tar.gz](https://dl.k8s.io/v1.27.0/kubernetes.tar.gz) | `78dbb72f270ab70d0ad70d2da6727eed64bdc54a11892fd6c2157882865f93ab41fedf5fced2f3e71dc0eda5679d06884c262a7960277face4510eed30a3678e` [kubernetes-src.tar.gz](https://dl.k8s.io/v1.27.0/kubernetes-src.tar.gz) | `4080d2452ff4fd316a823c1c495e7e9a39d364e24225020a91bf0bc0289c3ef90ade746ef5a05172d6e355af9014cbddf144ca71839ec65fc57f3eaf553fb7ab` ### Client Binaries filename | sha512 hash -------- | ----------- [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.27.0/kubernetes-client-darwin-amd64.tar.gz) | `faa0e340f1829ba694326c6ff71f8527249af03d8d78f784289be4122b6ceb0829fa70ee1eab25f64bbb9f5972ae30f3cfdfefe617ce3360b2897d4f6259bd81` [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.27.0/kubernetes-client-darwin-arm64.tar.gz) | `9c4fe911e41ab9c355d39b21d77372bd2a070cc376fdfceac362eb6cc3e8f616754cc61593ea140030a81961b40fa6344b7628d7a4edf7e6dcdef29711bbd064` [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.27.0/kubernetes-client-linux-386.tar.gz) | `ba522302624ac7b3a9e5c1a5c80857bdde4c47b44394dbfa8da597ee07b2e1975409e8eac514516329826f593fa82d143a03185ef3c30a97cb1f8011ffb96060` [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.0/kubernetes-client-linux-amd64.tar.gz) | `3ea3b4a866815cc08f1897771d63bf4e4f75b481e1d70417e34581d079a58b647b077382a264224acb52e6a76474d6e92efd22a0d4f7fdfde0c244006beef76a` [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.27.0/kubernetes-client-linux-arm.tar.gz) | `5fd69b567ab835b35b8156c66eec02ee109f731acf7d68250b05a1f43a56458be68654f95107cd28859b4b8e73d5f64c78aca2f4b1dc74fff3ca8d942c60d2db` [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.0/kubernetes-client-linux-arm64.tar.gz) | `f20e579ab71b1cdace22bec0a11314ec44534f0e7040a436c63eb18a47d839e070e5134917ef2b531fe7b8bfee12133fa14de4dac7c0ac7798b4d9fa5679f193` [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.0/kubernetes-client-linux-ppc64le.tar.gz) | `c56a2d021b1a99fde0871bbe8e71427b8c4f03847e2bf6cbf526a71f6d7d1060481bd0f00d7dce2bd8afa1c969e02422ac1a2283ab58facd3db43f0713c10212` [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.0/kubernetes-client-linux-s390x.tar.gz) | `4ad879e2ab2b952cc0fdfcd738b6264db60b72174057947737ab07f40dd0c4c727fb042c24323be3accaf8fbc320973821c915fd1bb3c4ea8a22eb16c03ce4a3` [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.27.0/kubernetes-client-windows-386.tar.gz) | `befab85193ce017c647b391606d45d3626e71bf7ea6bbca7f955985e0f505a9c8ca27898ee41c4f3124b7a3788b4a4eab602994415b24b8b0bfc154b938c547e` [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.0/kubernetes-client-windows-amd64.tar.gz) | `0fbba06f00713c32c74d9b62733dfb83a597e3a33ee62bfb3a93de7cd883c460a0c56f25cd1577dd7923ef73312788d9b805020297fcf784722783ac1890253f` [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.27.0/kubernetes-client-windows-arm64.tar.gz) | `b7edbb25dbbf5b0bd9839f93d43f08262cf5f6e138599c034da0ff402c763a0cff18c1e9b42631d250389ff6b865dec4aa35b577fac75a51e65c825ec8efe234` ### Server Binaries filename | sha512 hash -------- | ----------- [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.0/kubernetes-server-linux-amd64.tar.gz) | `9726ba173084adade1c1b0de014ccedc5dc5317a80076cbf20d15fdcd6296dd1e9efcf1b1349456757a5c186fa52293f60411397cb6c79765adff335391add9e` [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.0/kubernetes-server-linux-arm64.tar.gz) | `657726cd4ae93a9696371717a280689af76c488586c49273086bef4e712228025c6e179c2a5c93b8a33640ac42347dd821053485659f9383dbb1b3e2a17f022c` [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.0/kubernetes-server-linux-ppc64le.tar.gz) | `2ff2464453ca8ca2e9e4a024ad730c12fa506379b4a7bd749431fe64ddb13c2dccea05c37dba119799940eac2dc57635e9d70b908d1786a3cdc031a5b70504a5` [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.0/kubernetes-server-linux-s390x.tar.gz) | `44ce8faa8710832593b656e3b053207e05def556ad821b8e08e0c2f33b73f280a455fbef933ea70e9efbe8a085ef7deb47139d4e9af43417d8242029a2b60c35` ### Node Binaries filename | sha512 hash -------- | ----------- [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.0/kubernetes-node-linux-amd64.tar.gz) | `812f5adfafe778200558678af6510f9f315f75b46f7bb4482e92b57d1bed08c4b7f236850bf8e4dcac7018879736d614fc482e3641da06c6f8d0554af4f4ef45` [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.0/kubernetes-node-linux-arm64.tar.gz) | `a5e353205a93ebaade50dfd652ee5623b28ee4f6fd8ca949fb2303d708468026ba66c10b70f1761f4099706baad8959993a9ec0053259b94b5f4793aeda27adb` [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.0/kubernetes-node-linux-ppc64le.tar.gz) | `b5dbcf8131bad7ef897c64ac482599ac3bedc99e5c211d189e0566543a13c89d0812d7a7b1e4e9655d8d884ee24dc553616c96cf74df19f4d2cce0ea552015ce` [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.0/kubernetes-node-linux-s390x.tar.gz) | `81662d6e14a7500bf2714ae3c0b9070031ea5ef2628c84aaf2a8fe96fed07a52c3677babc74247160cb71ce1fc77b728f549f2c18dfc7dc6a65dfadb7ec17cd7` [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.0/kubernetes-node-windows-amd64.tar.gz) | `a8b1a53ba6ee416fb9939961d8290ae1f5e0c21117f1cd6cebbc9ba01cafa235730a2887fd91b92552d07eec78aeb65aefac92899eaf9b2f4f195c61f20d05d7` ### Container Images All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures ---- | ------------- [registry.k8s.io/conformance:v1.27.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) [registry.k8s.io/kube-apiserver:v1.27.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) [registry.k8s.io/kube-controller-manager:v1.27.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) [registry.k8s.io/kube-proxy:v1.27.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) [registry.k8s.io/kube-scheduler:v1.27.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) ## Changelog since v1.26.0 ## Known Issues ### The PreEnqueue extension point doesn't work for Pods going to activeQ through backoffQ In v1.26.0, we've found the bug that the PreEnqueue extension point doesn't work for Pods going to activeQ through backoffQ. It doesn't affect any of the vanilla Kubernetes behavior, but, may break custom PreEnqueue plugins. The cause PR is [reverted](https://github.com/kubernetes/kubernetes/pull/117194) by v1.26.1. ## Urgent Upgrade Notes ### (No, really, you MUST read this before you upgrade) - 'The `IPv6DualStack` feature gate for external cloud providers was removed. (The feature became GA in 1.23 and the gate was removed for all other components several releases ago.) If you were still manually enabling it you must stop now.' ([#116255](https://github.com/kubernetes/kubernetes/pull/116255), [@danwinship](https://github.com/danwinship)) - Give terminal phase correctly to all pods that will not be restarted. In particular, assign Failed phase to pods which are deleted while pending. Also, assign a terminal phase (Succeeded or Failed, depending on the exit statuses of the pod containers) to pods which are deleted while running. This fixes the issue for jobs using pod failure policy (with JobPodFailurePolicy and PodDisruptionConditions feature gates enabled) that their pods could get stuck in the pending phase when deleted. Users who maintain controllers which relied on the fact that pods with RestartPolicy=Always never enter the Succeeded phase may need to adapt their controllers. This is because as a consequence of the change pods which use RestartPolicy=Always may end up in the Succeeded phase in two scenarios: pod deletion and graceful node shutdown. ([#115331](https://github.com/kubernetes/kubernetes/pull/115331), [@mimowo](https://github.com/mimowo)) [SIG Cloud Provider, Node and Testing] - The in-tree cloud provider for AWS (and the EBS storage plugin) has now been removed. Please use the external cloud provider and CSI driver from https://github.com/kubernetes/cloud-provider-aws instead. ([#115838](https://github.com/kubernetes/kubernetes/pull/115838), [@torredil](https://github.com/torredil)) [SIG API Machinery, Apps, Architecture, Auth, CLI , Cloud Provider, Cluster Lifecycle, Instrumentation, Node, Release, Scheduling, Storage, and Testing] ## Changes by Kind ### Deprecation - Added a [warning](https://k8s.io/blog/2020/09/03/warnings/) response when handling requests that set the deprecated `spec.externalID` field for a Node. ([#115944](https://github.com/kubernetes/kubernetes/pull/115944), [@SataQiu](https://github.com/SataQiu)) [SIG Node] - Added warnings to the Services API. Kubernetes now warns for Services in the case of: - IPv4 addresses with leading zeros - IPv6 address in non-canonical format (RFC 5952) ([#114505](https://github.com/kubernetes/kubernetes/pull/114505), [@aojea](https://github.com/aojea)) - Support for the alpha seccomp annotations `seccomp.security.alpha.kubernetes.io/pod` and `container.seccomp.security.alpha.kubernetes.io` were deprecated since v1.19, now have been completely removed. The seccomp fields are no longer auto-populated when pods with seccomp annotations are created. Pods should use the corresponding pod or container `securityContext.seccompProfile` field instead. ([#114947](https://github.com/kubernetes/kubernetes/pull/114947), [@saschagrunert](https://github.com/saschagrunert)) - The `SecurityContextDeny` admission plugin is going deprecated and will be removed in future versions. ([#115879](https://github.com/kubernetes/kubernetes/pull/115879), [@mtardy](https://github.com/mtardy)) ### API Change - A fix in the `resource.k8s.io/v1alpha1/ResourceClaim` API avoids harmless (?) ".status.reservedFor: element 0: associative list without keys has an element that's a map type" errors in the apiserver. Validation now rejects the incorrect reuse of the same UID in different entries. ([#115354](https://github.com/kubernetes/kubernetes/pull/115354), [@pohly](https://github.com/pohly)) - A terminating pod on a node that is not caused by preemption no longer prevents `kube-scheduler` from preempting pods on that node - Rename `PreemptionByKubeScheduler` to `PreemptionByScheduler` ([#114623](https://github.com/kubernetes/kubernetes/pull/114623), [@Huang-Wei](https://github.com/Huang-Wei)) - API: resource.k8s.io/v1alpha1.PodScheduling was renamed to resource.k8s.io/v1alpha2.PodSchedulingContext. ([#116556](https://github.com/kubernetes/kubernetes/pull/116556), [@pohly](https://github.com/pohly)) [SIG API Machinery, Apps, Auth, CLI, Node, Scheduling and Testing] - Added CEL runtime cost calculation into ValidatingAdmissionPolicy, matching the evaluation cost restrictions that already apply to CustomResourceDefinition. If rule evaluation uses more compute than the limit, the API server aborts the evaluation and the admission check that was being performed is aborted; the `failurePolicy` for the ValidatingAdmissionPolicy determines the outcome. ([#115747](https://github.com/kubernetes/kubernetes/pull/115747), [@cici37](https://github.com/cici37)) - Added `auditAnnotations` to `ValidatingAdmissionPolicy`, enabling CEL to be used to add audit annotations to request audit events. Added `validationActions` to `ValidatingAdmissionPolicyBinding`, enabling validation failures to be handled by any combination of the warn, audit and deny enforcement actions. ([#115973](https://github.com/kubernetes/kubernetes/pull/115973), [@jpbetz](https://github.com/jpbetz)) - Added `messageExpression` field to `ValidationRule`. ([#115969](https://github.com/kubernetes/kubernetes/pull/115969), [@DangerOnTheRanger](https://github.com/DangerOnTheRanger)) - Added `messageExpression` to `ValidatingAdmissionPolicy`, to set custom failure message via CEL expression. ([#116397](https://github.com/kubernetes/kubernetes/pull/116397), [@jiahuif](https://github.com/jiahuif)) [SIG API Machinery] - Added a new IPAddress object kind - Added a new ClusterIP allocator. The new allocator removes previous Service CIDR block size limitations for IPv4, and limits IPv6 size to a /64 ([#115075](https://github.com/kubernetes/kubernetes/pull/115075), [@aojea](https://github.com/aojea)) [SIG API Machinery, Apps, Auth, CLI, Cluster Lifecycle, Network and Testing] - Added a new alpha API: ClusterTrustBundle (`certificates.k8s.io/v1alpha1`). A ClusterTrustBundle may be used to distribute [X.509](https://www.itu.int/rec/T-REC-X.509) trust anchors to workloads within the cluster. ([#113218](https://github.com/kubernetes/kubernetes/pull/113218), [@ahmedtd](https://github.com/ahmedtd)) [SIG API Machinery, Auth and Testing] - Added authorization check support to the CEL expressions of ValidatingAdmissionPolicy via a `authorizer` variable with expressions. The new variable provides a builder that allows expressions such `authorizer.group('').resource('pods').check('create').allowed()`. ([#116054](https://github.com/kubernetes/kubernetes/pull/116054), [@jpbetz](https://github.com/jpbetz)) [SIG API Machinery and Testing] - Added matchConditions field to ValidatingAdmissionPolicy and enabled support for CEL based custom match criteria. ([#116350](https://github.com/kubernetes/kubernetes/pull/116350), [@maxsmythe](https://github.com/maxsmythe)) - Added new option to the `InterPodAffinity` scheduler plugin to ignore existing pods` preferred inter-pod affinities if the incoming pod has no preferred inter-pod affinities. This option can be used as an optimization for higher scheduling throughput (at the cost of an occasional pod being scheduled non-optimally/violating existing pods preferred inter-pod affinities). To enable this scheduler option, set the `InterPodAffinity` scheduler plugin arg `ignorePreferredTermsOfExistingPods: true` ([#114393](https://github.com/kubernetes/kubernetes/pull/114393), [@danielvegamyhre](https://github.com/danielvegamyhre)) - Added the `MatchConditions` field to `ValidatingWebhookConfiguration` and `MutatingWebhookConfiguration` for the v1beta and v1 apis. The `AdmissionWebhookMatchConditions` featuregate is now in Alpha ([#116261](https://github.com/kubernetes/kubernetes/pull/116261), [@ivelichkovich](https://github.com/ivelichkovich)) [SIG API Machinery and Testing] - Added validation to ensure that if `service.kubernetes.io/topology-aware-hints` and `service.kubernetes.io/topology-mode` annotations are both set, they are set to the same value.Also Added deprecation warning if `service.kubernetes.io/topology-aware-hints` annotation is used. ([#116612](https://github.com/kubernetes/kubernetes/pull/116612), [@robscott](https://github.com/robscott)) - Added warnings about workload resources (Pods, ReplicaSets, Deployments, Jobs, CronJobs, or ReplicationControllers) whose names are not valid DNS labels. ([#114412](https://github.com/kubernetes/kubernetes/pull/114412), [@thockin](https://github.com/thockin)) - Adds feature gate `NodeLogQuery` which provides cluster administrators with a streaming view of logs using kubectl without them having to implement a client side reader or logging into the node. ([#96120](https://github.com/kubernetes/kubernetes/pull/96120), [@LorbusChris](https://github.com/LorbusChris)) - Api: validation of a `PodSpec` now rejects invalid `ResourceClaim` and `ResourceClaimTemplate` names. For a pod, the name generated for the `ResourceClaim` when using a template also must be valid. ([#116576](https://github.com/kubernetes/kubernetes/pull/116576), [@pohly](https://github.com/pohly)) - Bump default API QPS limits for Kubelet. ([#116121](https://github.com/kubernetes/kubernetes/pull/116121), [@wojtek-t](https://github.com/wojtek-t)) - Enabled the `StatefulSetStartOrdinal` feature gate in beta ([#115260](https://github.com/kubernetes/kubernetes/pull/115260), [@pwschuurman](https://github.com/pwschuurman)) - Enabled usage of `kube-proxy`, `kube-scheduler` and `kubelet` HTTP APIs for changing the logging verbosity at runtime for JSON output. ([#114609](https://github.com/kubernetes/kubernetes/pull/114609), [@pohly](https://github.com/pohly)) - Encryption of API Server at rest configuration now allows the use of wildcards in the list of resources. For example, *.* can be used to encrypt all resources, including all current and future custom resources. ([#115149](https://github.com/kubernetes/kubernetes/pull/115149), [@nilekhc](https://github.com/nilekhc)) - Extended the kubelet's PodResources API to include resources allocated in `ResourceClaims` via `DynamicResourceAllocation`. Additionally, added a new `Get()` method to query a specific pod for its resources. ([#115847](https://github.com/kubernetes/kubernetes/pull/115847), [@moshe010](https://github.com/moshe010)) [SIG Node] - Forbid to set matchLabelKeys when labelSelector is not set in topologySpreadConstraints ([#116535](https://github.com/kubernetes/kubernetes/pull/116535), [@denkensk](https://github.com/denkensk)) - GCE does not support LoadBalancer Services with ports with different protocols (TCP and UDP) ([#115966](https://github.com/kubernetes/kubernetes/pull/115966), [@aojea](https://github.com/aojea)) [SIG Apps and Cloud Provider] - GRPC probes are now a GA feature. `GRPCContainerProbe` feature gate was locked to default value and will be removed in v1.29. If you were setting this feature gate explicitly, please remove it now. ([#116233](https://github.com/kubernetes/kubernetes/pull/116233), [@SergeyKanzhelev](https://github.com/SergeyKanzhelev)) - Graduated `Kubelet Topology Manager` to GA. ([#116093](https://github.com/kubernetes/kubernetes/pull/116093), [@swatisehgal](https://github.com/swatisehgal)) - Graduated `KubeletTracing` to beta, which means that the feature gate is now enabled by default. ([#115750](https://github.com/kubernetes/kubernetes/pull/115750), [@saschagrunert](https://github.com/saschagrunert)) - Graduated seccomp profile defaulting to GA. Set the kubelet `--seccomp-default` flag or `seccompDefault` kubelet configuration field to `true` to make pods on that node default to using the `RuntimeDefault` seccomp profile. Enabling seccomp for your workload can have a negative performance impact depending on the kernel and container runtime version in use. Guidance for identifying and mitigating those issues is outlined in the Kubernetes [seccomp tutorial](https://k8s.io/docs/tutorials/security/seccomp). ([#115719](https://github.com/kubernetes/kubernetes/pull/115719), [@saschagrunert](https://github.com/saschagrunert)) [SIG API Machinery, Node, Storage and Testing] - Graduated the container resource metrics feature on `HPA` to beta. ([#116046](https://github.com/kubernetes/kubernetes/pull/116046), [@sanposhiho](https://github.com/sanposhiho)) - Implemented API streaming for the `watch-cache` When `sendInitialEvents` `ListOption` is set together with `watch=true`, it begins the watch stream with synthetic init events followed by a synthetic "Bookmark" after which the server continues streaming events. ([#110960](https://github.com/kubernetes/kubernetes/pull/110960), [@p0lyn0mial](https://github.com/p0lyn0mial)) - Introduced API for streaming. Added `SendInitialEvents` field to the `ListOptions`. When the new option is set together with `watch=true`, it begins the watch stream with synthetic init events followed by a synthetic "Bookmark" after which the server continues streaming events. ([#115402](https://github.com/kubernetes/kubernetes/pull/115402), [@p0lyn0mial](https://github.com/p0lyn0mial)) - Introduced a breaking change to the `resource.k8s.io` API in its `AllocationResult` struct. This change allows a kubelet plugin for the `DynamicResourceAllocation` feature to service allocations from multiple resource driver controllers. ([#116332](https://github.com/kubernetes/kubernetes/pull/116332), [@klueska](https://github.com/klueska)) - Introduces new alpha functionality to the reflector, allowing user to enable API streaming. To activate this feature, users can set the `ENABLE_CLIENT_GO_WATCH_LIST_ALPHA` environmental variable. It is important to note that the server must support streaming for this feature to function properly. If streaming is not supported by the server, the reflector will revert to the previous method of obtaining data through LIST/WATCH semantics. ([#110772](https://github.com/kubernetes/kubernetes/pull/110772), [@p0lyn0mial](https://github.com/p0lyn0mial)) [SIG API Machinery] - K8s.io/client-go/tools/record.EventBroadcaster: after Shutdown() is called, the broadcaster now gives up immediately after a failure to write an event to a sink. Previously it tried multiple times for 12 seconds in a goroutine. ([#115514](https://github.com/kubernetes/kubernetes/pull/115514), [@pohly](https://github.com/pohly)) [SIG API Machinery] - K8s.io/component-base/logs: usage of the pflag values in a normal Go flag set led to panics when printing the help message ([#114680](https://github.com/kubernetes/kubernetes/pull/114680), [@pohly](https://github.com/pohly)) [SIG Instrumentation] - Kubeadm: explicitly set `priority` for static pods with `priorityClassName: system-node-critical` ([#114338](https://github.com/kubernetes/kubernetes/pull/114338), [@champtar](https://github.com/champtar)) [SIG Cluster Lifecycle] - Kubelet: a "maxParallelImagePulls" field can now be specified in the kubelet configuration file to control how many image pulls the kubelet can perform in parallel. ([#115220](https://github.com/kubernetes/kubernetes/pull/115220), [@ruiwen-zhao](https://github.com/ruiwen-zhao)) [SIG API Machinery, Node and Scalability] - Kubelet: changed `MemoryThrottlingFactor` default value to `0.9` and formulas to calculate `memory.high` ([#115371](https://github.com/kubernetes/kubernetes/pull/115371), [@pacoxu](https://github.com/pacoxu)) - Kubernetes components that perform leader election now only support using `Leases` for this. ([#114055](https://github.com/kubernetes/kubernetes/pull/114055), [@aimuz](https://github.com/aimuz)) - Migrated the `DaemonSet` controller (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging) ([#113622](https://github.com/kubernetes/kubernetes/pull/113622), [@249043822](https://github.com/249043822)) - New `service.kubernetes.io/topology-mode` annotation has been introduced as a replacement for the `service.kubernetes.io/topology-aware-hints` annotation. - `service.kubernetes.io/topology-aware-hints` annotation has been deprecated. - kube-proxy now accepts any value that is not "disabled" for these annotations, enabling custom implementation-specific and/or future built-in heuristics to be used. ([#116522](https://github.com/kubernetes/kubernetes/pull/116522), [@robscott](https://github.com/robscott)) [SIG Apps, Network and Testing] - Pods owned by a Job now uses the labels `batch.kubernetes.io/job-name` and `batch.kubernetes.io/controller-uid`. The legacy labels `job-name` and `controller-uid` are still added for compatibility. ([#114930](https://github.com/kubernetes/kubernetes/pull/114930), [@kannon92](https://github.com/kannon92)) - Promoted `CronJobTimeZone` feature to GA ([#115904](https://github.com/kubernetes/kubernetes/pull/115904), [@soltysh](https://github.com/soltysh)) - Promoted `SelfSubjectReview` to Beta ([#116274](https://github.com/kubernetes/kubernetes/pull/116274), [@nabokihms](https://github.com/nabokihms)) [SIG API Machinery, Auth, CLI and Testing] - Relaxed API validation to allow pod node selector to be mutable for gated pods (additions only, no deletions or mutations). ([#116161](https://github.com/kubernetes/kubernetes/pull/116161), [@danielvegamyhre](https://github.com/danielvegamyhre)) - Remove `kubernetes.io/grpc` standard appProtocol ([#116866](https://github.com/kubernetes/kubernetes/pull/116866), [@LiorLieberman](https://github.com/LiorLieberman)) [SIG API Machinery and Apps] - Remove deprecated `--enable-taint-manager` and `--pod-eviction-timeout` CLI ([#115840](https://github.com/kubernetes/kubernetes/pull/115840), [@atosatto](https://github.com/atosatto)) - Removed support for the `v1alpha1` kubeletplugin API of `DynamicResourceManagement`. All plugins must be updated to `v1alpha2` in order to function properly. ([#116558](https://github.com/kubernetes/kubernetes/pull/116558), [@klueska](https://github.com/klueska)) - The API server now re-uses data encryption keys while the kms v2 plugin key ID is stable. Data encryption keys are still randomly generated on server start but an atomic counter is used to prevent nonce collisions. ([#116155](https://github.com/kubernetes/kubernetes/pull/116155), [@enj](https://github.com/enj)) - The PodDisruptionBudget `spec.unhealthyPodEvictionPolicy` field has graduated to beta and is enabled by default. On servers with the feature enabled, this field may be set to `AlwaysAllow` to always allow unhealthy pods covered by the PodDisruptionBudget to be evicted. ([#115363](https://github.com/kubernetes/kubernetes/pull/115363), [@ravisantoshgudimetla](https://github.com/ravisantoshgudimetla)) [SIG Apps, Auth and Node] - The `DownwardAPIHugePages` kubelet feature graduated to stable / GA. ([#115721](https://github.com/kubernetes/kubernetes/pull/115721), [@saschagrunert](https://github.com/saschagrunert)) [SIG Apps and Node] - The following feature gates for volume expansion GA features have now been removed and must no longer be referenced in `--feature-gates` flags: `ExpandCSIVolumes`, `ExpandInUsePersistentVolumes`, `ExpandPersistentVolumes` ([#113942](https://github.com/kubernetes/kubernetes/pull/113942), [@mengjiao-liu](https://github.com/mengjiao-liu)) - The list-type of the alpha `resourceClaims` field introduced to `Pods` in `1.26.0` was modified from `set` to `map`, resolving an incompatibility with use of this schema in `CustomResourceDefinitions` and with server-side apply. ([#114585](https://github.com/kubernetes/kubernetes/pull/114585), [@JoelSpeed](https://github.com/JoelSpeed)) - Updated API reference for Requests, specifying they must not exceed limits ([#115434](https://github.com/kubernetes/kubernetes/pull/115434), [@ehashman](https://github.com/ehashman)) - Updated `KMSv2` to beta ([#115123](https://github.com/kubernetes/kubernetes/pull/115123), [@aramase](https://github.com/aramase)) - Updated: Redefine AppProtocol field description and add new standard values ([#115433](https://github.com/kubernetes/kubernetes/pull/115433), [@LiorLieberman](https://github.com/LiorLieberman)) [SIG API Machinery, Apps and Network] - `/metrics/slis` is now available for control plane components allowing you to scrape health check metrics. ([#114997](https://github.com/kubernetes/kubernetes/pull/114997), [@Richabanker](https://github.com/Richabanker)) - `APIServerTracing` feature gate is now enabled by default. Tracing in the API Server is still disabled by default, and requires a config file to enable. ([#116144](https://github.com/kubernetes/kubernetes/pull/116144), [@dashpole](https://github.com/dashpole)) - `NodeResourceFit` and `NodeResourcesBalancedAllocation` implement the `PreScore` extension point for a more performant calculation. ([#115655](https://github.com/kubernetes/kubernetes/pull/115655), [@tangwz](https://github.com/tangwz)) - `PodSchedulingReadiness` is graduated to beta. ([#115815](https://github.com/kubernetes/kubernetes/pull/115815), [@Huang-Wei](https://github.com/Huang-Wei)) - `PodSpec.Container.Resources` became mutable for CPU and memory resource types. - `PodSpec.Container.ResizePolicy` (new object) gives users control over how their containers are resized. - `PodStatus.Resize` status describes the state of a requested Pod resize. - `PodStatus.ResourcesAllocated` describes node resources allocated to Pod. - `PodStatus.Resources` describes node resources applied to running containers by CRI. - `UpdateContainerResources` CRI API now supports both Linux and Windows. ([#102884](https://github.com/kubernetes/kubernetes/pull/102884), [@vinaykul](https://github.com/vinaykul)) - `SELinuxMountReadWriteOncePod` graduated to Beta. ([#116425](https://github.com/kubernetes/kubernetes/pull/116425), [@jsafrane](https://github.com/jsafrane)) - `StatefulSetAutoDeletePVC` feature gate promoted to beta. ([#116501](https://github.com/kubernetes/kubernetes/pull/116501), [@mattcary](https://github.com/mattcary)) - `StatefulSet` names must be DNS labels, rather than subdomains. Any `StatefulSet` which took advantage of subdomain validation (by having dots in the name) can't possibly have worked, because we eventually set `pod.spec.hostname` from the `StatefulSetName`, and that is validated as a DNS label. ([#114172](https://github.com/kubernetes/kubernetes/pull/114172), [@thockin](https://github.com/thockin)) - `ValidatingAdmissionPolicy` now provides a status field that contains results of type checking the validation expression. The type checking is fully informational, and the behavior of the policy is unchanged. ([#115668](https://github.com/kubernetes/kubernetes/pull/115668), [@jiahuif](https://github.com/jiahuif)) - `cacheSize` field in `EncryptionConfiguration` is not supported for KMSv2 provider ([#113121](https://github.com/kubernetes/kubernetes/pull/113121), [@aramase](https://github.com/aramase)) - `k8s.io/component-base/logs` now also supports adding command line flags to a `flag.FlagSet`. ([#114731](https://github.com/kubernetes/kubernetes/pull/114731), [@pohly](https://github.com/pohly)) - `kubelet`: migrated `--container-runtime-endpoint` and `--image-service-endpoint` to kubelet config ([#112136](https://github.com/kubernetes/kubernetes/pull/112136), [@pacoxu](https://github.com/pacoxu)) - `resource.k8s.io/v1alpha1` was replaced with `resource.k8s.io/v1alpha2`. Before upgrading a cluster, all objects in resource.k8s.io/v1alpha1 (ResourceClaim, ResourceClaimTemplate, ResourceClass, PodScheduling) must be deleted. The changes are internal, so YAML files which create pods and resource claims don't need changes except for the newer `apiVersion`. ([#116299](https://github.com/kubernetes/kubernetes/pull/116299), [@pohly](https://github.com/pohly)) - `volumes`: `resource.claims` is now cleared for PVC specs during create or update of a pod spec with inline PVC template or of a PVC because it has no effect. ([#115928](https://github.com/kubernetes/kubernetes/pull/115928), [@pohly](https://github.com/pohly)) ### Feature - A new client side metric `rest_client_request_retries_total` has been added that tracks the number of retries sent to the server, partitioned by status code, verb and host ([#108396](https://github.com/kubernetes/kubernetes/pull/108396), [@tkashem](https://github.com/tkashem)) - A new feature was enabled to improve the performance of the iptables mode of `kube-proxy` in large clusters. No action was required, however: 1. If you experienced problems with Services not syncing to iptables correctly, you can disable the feature by passing `--feature-gates=MinimizeIPTablesRestore=false` to kube-proxy (and file a bug if this fixes it). (This might also be detected by seeing the value of kube-proxy's `sync_proxy_rules_iptables_partial_restore_failures_total` metric rising.) 2. If you were previously overriding the kube-proxy configuration for performance reasons, this may no longer be necessary. See https://kubernetes.io/docs/reference/networking/virtual-ips/#optimizing-iptables-mode-performance. ([#115138](https://github.com/kubernetes/kubernetes/pull/115138), [@danwinship](https://github.com/danwinship)) - API validation relaxed allowing Indexed Jobs to be scaled up/down by changing parallelism and completions in tandem, such that parallelism == completions. ([#115236](https://github.com/kubernetes/kubernetes/pull/115236), [@danielvegamyhre](https://github.com/danielvegamyhre)) [SIG Apps and Testing] - Added "general", "baseline", and "restricted" debugging profiles for kubectl debug. ([#114280](https://github.com/kubernetes/kubernetes/pull/114280), [@sding3](https://github.com/sding3)) [SIG CLI] - Added "netadmin" debugging profiles for kubectl debug. ([#115712](https://github.com/kubernetes/kubernetes/pull/115712), [@wedaly](https://github.com/wedaly)) [SIG CLI] - Added `--output plaintext-openapiv2` argument to kubectl explain to use old openapiv2 `explain` implementation. ([#115480](https://github.com/kubernetes/kubernetes/pull/115480), [@alexzielenski](https://github.com/alexzielenski)) - Added `NewVolumeManagerReconstruction` feature gate and enabled it by default to enable updated discovery of mounted volumes during kubelet startup. Please watch for kubelet getting stuck at startup and / or not unmounting volumes from deleted Pods and report any issues in this area. ([#115268](https://github.com/kubernetes/kubernetes/pull/115268), [@jsafrane](https://github.com/jsafrane)) - Added `kubelet` Topology Manager metrics to track admission requests processed and occured admission errors. ([#115137](https://github.com/kubernetes/kubernetes/pull/115137), [@swatisehgal](https://github.com/swatisehgal)) - Added apiserver_envelope_encryption_invalid_key_id_from_status_total to measure number of times an invalid keyID is returned by the Status RPC call. ([#115846](https://github.com/kubernetes/kubernetes/pull/115846), [@ritazh](https://github.com/ritazh)) [SIG API Machinery and Auth] - Added apiserver_envelope_encryption_kms_operations_latency_seconds metric to measure the KMSv2 grpc calls latency. ([#115649](https://github.com/kubernetes/kubernetes/pull/115649), [@aramase](https://github.com/aramase)) [SIG API Machinery, Auth and Testing] - Added e2e test to node expand volume with secret ([#115451](https://github.com/kubernetes/kubernetes/pull/115451), [@zhucan](https://github.com/zhucan)) - Added e2e tests for kubectl `--subresource` for beta graduation ([#116590](https://github.com/kubernetes/kubernetes/pull/116590), [@MadhavJivrajani](https://github.com/MadhavJivrajani)) - Added kubelet Topology Manager metric to measure topology manager admission latency. ([#115590](https://github.com/kubernetes/kubernetes/pull/115590), [@swatisehgal](https://github.com/swatisehgal)) - Added logging-format option to CCMs based on `k8s.io/cloud-provider` ([#108984](https://github.com/kubernetes/kubernetes/pull/108984), [@LittleFox94](https://github.com/LittleFox94)) - Added metrics for volume reconstruction during kubelet startup. ([#115965](https://github.com/kubernetes/kubernetes/pull/115965), [@jsafrane](https://github.com/jsafrane)) [SIG Node and Storage] - Added new -f flag into debug command to be used passing pod or node files instead explicit names. ([#111453](https://github.com/kubernetes/kubernetes/pull/111453), [@ardaguclu](https://github.com/ardaguclu)) - Added new feature gate `ServiceNodePortStaticSubrange`, to enable the new strategy in the `NodePort` Service port allocators, so the node port range is subdivided and dynamic allocated `NodePort` port for Services are allocated preferentially from the upper range. ([#114418](https://github.com/kubernetes/kubernetes/pull/114418), [@xuzhenglun](https://github.com/xuzhenglun)) - Added scheduler preemption support for pods using `ReadWriteOncePod` PVCs ([#114051](https://github.com/kubernetes/kubernetes/pull/114051), [@chrishenzie](https://github.com/chrishenzie)) - Added the `applyconfiguration` generator to the code-generator script that generates server-side apply configuration and client APIs ([#114987](https://github.com/kubernetes/kubernetes/pull/114987), [@astefanutti](https://github.com/astefanutti)) - Added the ability to host webhooks in the cloud controller manager. ([#108838](https://github.com/kubernetes/kubernetes/pull/108838), [@nckturner](https://github.com/nckturner)) - Apiserver_storage_transformation_operations_total metric has been updated to include labels transformer_prefix and status. ([#115394](https://github.com/kubernetes/kubernetes/pull/115394), [@ritazh](https://github.com/ritazh)) [SIG API Machinery, Auth, Instrumentation and Testing] - By enabling the `UserNamespacesStatelessPodsSupport` feature gate in kubelet, you can now run a stateless pod in a separate user namespace ([#116377](https://github.com/kubernetes/kubernetes/pull/116377), [@giuseppe](https://github.com/giuseppe)) [SIG Apps, Node and Storage] - By enabling the alpha `CloudNodeIPs` feature gate in kubelet and the cloud provider, you can now specify a dual-stack `--node-ip` value (when using an external cloud provider that supports that functionality). ([#116305](https://github.com/kubernetes/kubernetes/pull/116305), [@danwinship](https://github.com/danwinship)) [SIG API Machinery, Cloud Provider, Network and Node] - Changed kubectl `--subresource` flag to beta ([#116595](https://github.com/kubernetes/kubernetes/pull/116595), [@MadhavJivrajani](https://github.com/MadhavJivrajani)) - Changed metrics for aggregated discovery to publish new time series (alpha). ([#115630](https://github.com/kubernetes/kubernetes/pull/115630), [@Jefftree](https://github.com/Jefftree)) [SIG API Machinery and Testing] - Dynamic Resource Allocation framework can be used for network devices ([#114364](https://github.com/kubernetes/kubernetes/pull/114364), [@bart0sh](https://github.com/bart0sh)) [SIG Node] - Enable external plugins can be used as subcommands for kubectl create command if subcommand does not exist as builtin only when KUBECTL_ENABLE_CMD_SHADOW environment variable is exported. ([#116293](https://github.com/kubernetes/kubernetes/pull/116293), [@ardaguclu](https://github.com/ardaguclu)) - GRPC probes now set a linger option of 1s to improve the TIME-WAIT state. ([#115321](https://github.com/kubernetes/kubernetes/pull/115321), [@rphillips](https://github.com/rphillips)) [SIG Network and Node] - Graduated CRI Events driven Pod LifeCycle Event Generator (Evented PLEG) to Beta ([#115967](https://github.com/kubernetes/kubernetes/pull/115967), [@harche](https://github.com/harche)) - Graduated `matchLabelKeys` in `podTopologySpread` to Beta ([#116291](https://github.com/kubernetes/kubernetes/pull/116291), [@denkensk](https://github.com/denkensk)) - Graduated the `CSINodeExpandSecret` feature to Beta. This feature facilitates passing secrets to CSI driver as part of Node Expansion CSI operation. ([#115621](https://github.com/kubernetes/kubernetes/pull/115621), [@humblec](https://github.com/humblec)) - Graduated the `LegacyServiceAccountTokenTracking` feature gate to Beta. The usage of auto-generated secret-based service account token now produces warnings by default, and relevant Secrets are labeled with a last-used timestamp (label key `kubernetes.io/legacy-token-last-used`). ([#114523](https://github.com/kubernetes/kubernetes/pull/114523), [@zshihang](https://github.com/zshihang)) [SIG API Machinery and Auth] - HPA controller exposes the following metrics from the kube-controller-manager. - `metric_computation_duration_seconds`: The time(seconds) that the HPA controller takes to calculate one metric. - `metric_computation_total`: Number of metric computations. ([#116326](https://github.com/kubernetes/kubernetes/pull/116326), [@sanposhiho](https://github.com/sanposhiho)) [SIG Apps, Autoscaling and Instrumentation] - HPA controller starts to expose metrics from the kube-controller-manager.\n- `reconciliations_total`: Number of reconciliation of HPA controller. \n- `reconciliation_duration_seconds`: The time(seconds) that the HPA controller takes to reconcile once. ([#116010](https://github.com/kubernetes/kubernetes/pull/116010), [@sanposhiho](https://github.com/sanposhiho)) - Kube-up now includes `CoreDNS` version `v1.9.3` ([#114279](https://github.com/kubernetes/kubernetes/pull/114279), [@pacoxu](https://github.com/pacoxu)) - Kubeadm: added the experimental (alpha) feature gate `EtcdLearnerMode` that allows etcd members to be joined as learner and only then promoted as voting members ([#113318](https://github.com/kubernetes/kubernetes/pull/113318), [@pacoxu](https://github.com/pacoxu)) - Kubectl will now display `SeccompProfile` for pods, containers and ephemeral containers, if values were set. ([#113284](https://github.com/kubernetes/kubernetes/pull/113284), [@williamyeh](https://github.com/williamyeh)) - Kubectl: added e2e test for default container annotation ([#115046](https://github.com/kubernetes/kubernetes/pull/115046), [@pacoxu](https://github.com/pacoxu)) - Kubelet TCP and HTTP probes are now more effective using networking resources: conntrack entries, sockets. This is achieved by reducing the `TIME-WAIT` state of the connection to 1 second, instead of the defaults 60 seconds. This allows kubelet to free the socket, and free conntrack entry and ephemeral port associated. ([#115143](https://github.com/kubernetes/kubernetes/pull/115143), [@aojea](https://github.com/aojea)) - Kubelet allows pods to use the `net.ipv4.ip_local_reserved_ports` sysctl by default and the minimal kernel version is 3.16; Pod Security admission allows this sysctl in v1.27+ versions of the baseline and restricted policies. ([#115374](https://github.com/kubernetes/kubernetes/pull/115374), [@pacoxu](https://github.com/pacoxu)) [SIG Auth, Network and Node] - Kubelet config file will be backed up to `/etc/kubernetes/tmp/` folder with `kubeadm-kubelet-config` append with a random suffix as the filename ([#114695](https://github.com/kubernetes/kubernetes/pull/114695), [@chendave](https://github.com/chendave)) [SIG Cluster Lifecycle] - Kubernetes is now built with Go `1.19.5` ([#115010](https://github.com/kubernetes/kubernetes/pull/115010), [@cpanato](https://github.com/cpanato)) - Kubernetes is now built with go 1.20 ([#114502](https://github.com/kubernetes/kubernetes/pull/114502), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing] - Kubernetes is now built with go 1.20.1 ([#115828](https://github.com/kubernetes/kubernetes/pull/115828), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing] - Kubernetes is now built with go 1.20.2 ([#116404](https://github.com/kubernetes/kubernetes/pull/116404), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing] - Locked `CSIMigrationvSphere` feature gate. ([#116610](https://github.com/kubernetes/kubernetes/pull/116610), [@xing-yang](https://github.com/xing-yang)) - Made `apiextensions-apiserver` binary linking static (also affects the deb and rpm packages). ([#114226](https://github.com/kubernetes/kubernetes/pull/114226), [@saschagrunert](https://github.com/saschagrunert)) - Made `kube-aggregator` binary linking static (also affects the deb and rpm packages). ([#114227](https://github.com/kubernetes/kubernetes/pull/114227), [@saschagrunert](https://github.com/saschagrunert)) - Made `kubectl-convert` binary linking static (also affects the deb and rpm packages). ([#114228](https://github.com/kubernetes/kubernetes/pull/114228), [@saschagrunert](https://github.com/saschagrunert)) - Migrated controller helper functions to use contextual logging. ([#115049](https://github.com/kubernetes/kubernetes/pull/115049), [@fatsheep9146](https://github.com/fatsheep9146)) - Migrated the ResourceQuota controller (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#113315](https://github.com/kubernetes/kubernetes/pull/113315), [@ncdc](https://github.com/ncdc)) [SIG API Machinery, Apps and Testing] - Migrated the StatefulSet controller (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging) ([#113840](https://github.com/kubernetes/kubernetes/pull/113840), [@249043822](https://github.com/249043822)) - Migrated the `ClusterRole` aggregation controller (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#113910](https://github.com/kubernetes/kubernetes/pull/113910), [@mengjiao-liu](https://github.com/mengjiao-liu)) - Migrated the `Deployment` controller (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging) ([#113525](https://github.com/kubernetes/kubernetes/pull/113525), [@249043822](https://github.com/249043822)) - Migrated the `ReplicaSet` controller (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#114871](https://github.com/kubernetes/kubernetes/pull/114871), [@Namanl2001](https://github.com/Namanl2001)) - Migrated the bootstrap signer controller and the token cleaner controller (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#113464](https://github.com/kubernetes/kubernetes/pull/113464), [@mengjiao-liu](https://github.com/mengjiao-liu)) [SIG API Machinery, Apps and Instrumentation] - Migrated the defaultbinder scheduler plugin to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#116571](https://github.com/kubernetes/kubernetes/pull/116571), [@mengjiao-liu](https://github.com/mengjiao-liu)) [SIG Instrumentation and Scheduling] - Migrated the main kube-controller-manager binary to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#116529](https://github.com/kubernetes/kubernetes/pull/116529), [@pohly](https://github.com/pohly)) - Migrated the namespace controller (within `kube-controller-manager`) to support [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#113443](https://github.com/kubernetes/kubernetes/pull/113443), [@yangjunmyfm192085](https://github.com/yangjunmyfm192085)) - Migrated the service-account controller (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#114918](https://github.com/kubernetes/kubernetes/pull/114918), [@Namanl2001](https://github.com/Namanl2001)) [SIG API Machinery, Apps, Auth, Instrumentation and Testing] - Migrated the volume attach/detach controller (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). Migrated the `PersistentVolumeClaim` protection controller (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). Migrated the `PersistentVolume` protection controller (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#113584](https://github.com/kubernetes/kubernetes/pull/113584), [@yangjunmyfm192085](https://github.com/yangjunmyfm192085)) - Migrated the “TTL after finished” controller (within `kube-controller-manager`)to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#113916](https://github.com/kubernetes/kubernetes/pull/113916), [@songxiao-wang87](https://github.com/songxiao-wang87)) - Migrated the `cronjob` controller (within `kube-controller-manager`)to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging) ([#113428](https://github.com/kubernetes/kubernetes/pull/113428), [@mengjiao-liu](https://github.com/mengjiao-liu)) - New plugin_evaluation_total is added to the scheduler.This metric counts how many times the specific plugin affects the scheduling result. The metric does not get incremented when the plugin has nothing to do with an incoming Pod. ([#115082](https://github.com/kubernetes/kubernetes/pull/115082), [@sanposhiho](https://github.com/sanposhiho)) - Node `ipam` controller now exposes metrics `cidrset_cidrs_max_total` and `multicidrset_cidrs_max_total` with information about the max number of CIDRs that can be allocated. ([#112260](https://github.com/kubernetes/kubernetes/pull/112260), [@aryan9600](https://github.com/aryan9600)) - Performance improvements in `klog` ([#115277](https://github.com/kubernetes/kubernetes/pull/115277), [@pohly](https://github.com/pohly)) - Pod template `schedulingGates` are now mutable for Jobs that are suspended and have never been started ([#115940](https://github.com/kubernetes/kubernetes/pull/115940), [@ahg-g](https://github.com/ahg-g)) [SIG Apps] - Pods which have an invalid negative `spec.terminationGracePeriodSeconds` value will now be treated as having a `terminationGracePeriodSeconds` of `1` ([#115606](https://github.com/kubernetes/kubernetes/pull/115606), [@wzshiming](https://github.com/wzshiming)) - Profiling can now be served on a unix-domain socket by using the `--profiling-path` option (when profiling is enabled) for security purposes. ([#114191](https://github.com/kubernetes/kubernetes/pull/114191), [@apelisse](https://github.com/apelisse)) [SIG API Machinery] - Promote aggregated discovery endpoint to beta and it will be enabled by default ([#116108](https://github.com/kubernetes/kubernetes/pull/116108), [@Jefftree](https://github.com/Jefftree)) - Promoted `OpenAPIV3` to GA ([#116235](https://github.com/kubernetes/kubernetes/pull/116235), [@Jefftree](https://github.com/Jefftree)) - Promoted `whoami` kubectl command. ([#116510](https://github.com/kubernetes/kubernetes/pull/116510), [@nabokihms](https://github.com/nabokihms)) - Scheduler no longer runs the plugin's `Filter` method when its `PreFilter` method returned a Skip status. In other words, your `PreFilter`/`Filter` plugin can return a Skip status in `PreFilter` if the plugin does nothing in Filter for that Pod. Scheduler skips `NodeAffinity` Filter plugin when `NodeAffinity` Filter plugin has nothing to do with a Pod. It may affect some metrics values related to the `NodeAffinity` Filter plugin. ([#114125](https://github.com/kubernetes/kubernetes/pull/114125), [@sanposhiho](https://github.com/sanposhiho)) - Scheduler now skips `InterPodAffinity` Filter plugin when `InterPodAffinity` Filter plugin has nothing to do with a Pod. It may affect some metrics values related to the `InterPodAffinity` Filter plugin. ([#114889](https://github.com/kubernetes/kubernetes/pull/114889), [@sanposhiho](https://github.com/sanposhiho)) - Scheduler volumebinding: leveraged `PreFilterResult` to reduce down to only eligible node(s) for pod with bound claim(s) to local `PersistentVolume(s)` ([#109877](https://github.com/kubernetes/kubernetes/pull/109877), [@yibozhuang](https://github.com/yibozhuang)) - Scheduling cycle now terminates immediately when any scheduler plugin returns an `unschedulableAndUnresolvable` status in `PostFilter`. ([#114699](https://github.com/kubernetes/kubernetes/pull/114699), [@kerthcet](https://github.com/kerthcet)) - Since Kubernetes v1.5, `kubectl apply` has had an alpha-stage `--prune` flag to support deleting previously applied objects that have been removed from the input manifest. This feature has remained in alpha ever since due to performance and correctness issues inherent in its design. This PR exposes a second, independent pruning alpha powered by a new standard named `ApplySets`. An `ApplySet` is a server-side object (by default, a Secret; ConfigMaps are also allowed) that kubectl can use to accurately and efficiently track set membership across `apply` operations. The format used for `ApplySet` is set out in [KEP 3659](https://github.com/kubernetes/enhancements/issues/3659) as a low-level specification. Other tools in the ecosystem can also build on this specification for improved interoperability. To try the ApplySet-based pruning alpha, set `KUBECTL_APPLYSET=true` and use the flags `--prune --applyset=secret-name` with `kubectl apply`. ([#116205](https://github.com/kubernetes/kubernetes/pull/116205), [@justinsb](https://github.com/justinsb)) - Switched kubectl explain to use OpenAPIV3 information published by the server. OpenAPIV2 backend can still be used with the `--output plaintext-openapiv2` argument ([#116390](https://github.com/kubernetes/kubernetes/pull/116390), [@alexzielenski](https://github.com/alexzielenski)) [SIG API Machinery, CLI and Testing] - The Pod API field `.spec.schedulingGates[*].name` now requires qualified names (like `example.com/mygate`), matching validation for names of `.spec.readinessGates[*].name`. Any uses of the alpha scheduling gate feature prior to 1.27 that do not match that validation must be renamed or deleted before upgrading to 1.27. ([#115821](https://github.com/kubernetes/kubernetes/pull/115821), [@lianghao208](https://github.com/lianghao208)) [SIG Apps and Scheduling] - The Scheduler did not run the plugin Score method when its PreScore method returned a Skip status. In other words, the PreScore/Score plugin could return a Skip status in PreScore if the plugin did nothing in Score for that Pod. ([#115652](https://github.com/kubernetes/kubernetes/pull/115652), [@AxeZhan](https://github.com/AxeZhan)) - The `AdvancedAuditing` feature gate was locked to _true_ in v1.27, and will be removed completely in v1.28 ([#115163](https://github.com/kubernetes/kubernetes/pull/115163), [@SataQiu](https://github.com/SataQiu)) [SIG API Machinery] - The `JobMutableNodeSchedulingDirectives` feature gate has graduated to GA. ([#116116](https://github.com/kubernetes/kubernetes/pull/116116), [@ahg-g](https://github.com/ahg-g)) [SIG Apps, Scheduling and Testing] - The `ReadWriteOncePod` feature gate has been graduated to beta. ([#114494](https://github.com/kubernetes/kubernetes/pull/114494), [@chrishenzie](https://github.com/chrishenzie)) - The bug which caused the status of Indexed Jobs to only update when new indexes were completed was fixed. Now, completed indexes are updated even if the `.status.completedIndexes` values are outside the `[0, .spec.completions> range`. ([#115349](https://github.com/kubernetes/kubernetes/pull/115349), [@danielvegamyhre](https://github.com/danielvegamyhre)) - The go version defined in `.go-version` is now fetched when invoking test, build, and code generation targets if the current go version does not match it. Set $FORCE_HOST_GO=y while testing or building to skip this behavior, or set $GO_VERSION to override the selected go version. ([#115377](https://github.com/kubernetes/kubernetes/pull/115377), [@liggitt](https://github.com/liggitt)) [SIG Testing] - The job controller back-off logic is now decoupled from workqueue. In case of parallelism > 1, if there are multiple new failures in a reconciliation cycle, all the failures are taken into account to compute the back-off. Previously, the back-off kicked in for all types of failures; with this change, only pod failures are taken into account. If the back-off limits exceeds, the job is marked as failed immediately; before this change, the job is marked as failed in the next back-off. ([#114768](https://github.com/kubernetes/kubernetes/pull/114768), [@sathyanarays](https://github.com/sathyanarays)) [SIG Apps and Testing] - The mount-utils mounter now provides an option to limit the number of concurrent format operations. ([#115379](https://github.com/kubernetes/kubernetes/pull/115379), [@artemvmin](https://github.com/artemvmin)) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node and Storage] - The scheduler's metric `plugin_execution_duration_seconds` now records `PreEnqueue` plugins execution seconds. ([#116201](https://github.com/kubernetes/kubernetes/pull/116201), [@sanposhiho](https://github.com/sanposhiho)) - Two changes to the `/debug/api_priority_and_fairness/dump_priority_levels` endpoint of API Priority and Fairness: added total number of dispatched, timed-out, rejected and cancelled requests; output now sorted by `PriorityLevelName`. ([#112393](https://github.com/kubernetes/kubernetes/pull/112393), [@borgerli](https://github.com/borgerli)) - Unlocked the `CSIMigrationvSphere` feature gate. The change allow users to continue using the in-tree vSphere driver,pending a vSphere CSI driver release that has with GA support for Windows, XFS, and raw block access. ([#116342](https://github.com/kubernetes/kubernetes/pull/116342), [@msau42](https://github.com/msau42)) [SIG Storage] - Updated `cAdvisor` to `v0.47.0` ([#114883](https://github.com/kubernetes/kubernetes/pull/114883), [@bobbypage](https://github.com/bobbypage)) - Updated `kube-apiserver` SLO/SLI latency metrics to exclude priority & fairness queue wait times ([#116420](https://github.com/kubernetes/kubernetes/pull/116420), [@andrewsykim](https://github.com/andrewsykim)) - Updated distroless iptables to use released image `registry.k8s.io/build-image/distroless-iptables:v0.2.2` - Updated setcap to use released image `registry.k8s.io/build-image/setcap:bullseye-v1.4.2` ([#116509](https://github.com/kubernetes/kubernetes/pull/116509), [@cpanato](https://github.com/cpanato)) [SIG Testing] - Updated distroless iptables to use released image `registry.k8s.io/distroless-iptables:v0.2.1` ([#115905](https://github.com/kubernetes/kubernetes/pull/115905), [@cpanato](https://github.com/cpanato)) [SIG Testing] - Upgrades functionality of `kubectl kustomize` as described at https://github.com/kubernetes-sigs/kustomize/releases/tag/kustomize%2Fv5.0.0 and https://github.com/kubernetes-sigs/kustomize/releases/tag/kustomize%2Fv5.0.1. This is a new major release of kustomize, so there are a few backwards-incompatible changes, most of which are rare use cases, bug fixes with side effects, or things that have been deprecated for multiple releases already: - https://github.com/kubernetes-sigs/kustomize/pull/4911: Drop support for a very old, legacy style of patches. patches used to be allowed to be used as an alias for patchesStrategicMerge in kustomize v3. You now have to use patchesStrategicMerge explicitly, or update to the new syntax supported by patches. See examples in the PR description of https://github.com/kubernetes-sigs/kustomize/pull/4911. - https://github.com/kubernetes-sigs/kustomize/issues/4731: Remove a potential build-time side-effect in ConfigMapGenerator and SecretGenerator, which loaded values from the local environment under some circumstances, breaking kustomize build's side-effect-free promise. While this behavior was never intended, we deprecated it and are announcing it as a breaking change since it existed for a long time. See also the Eschewed Features documentation. - https://github.com/kubernetes-sigs/kustomize/pull/4985: If you previously included .git in an AWS or Azure URL, we will no longer automatically remove that suffix. You may need to add an extra / to replace the .git for the URL to properly resolve. - https://github.com/kubernetes-sigs/kustomize/pull/4954: Drop support for using gh: as a host (e.g. gh:kubernetes-sigs/kustomize). We were unable to find any usage of or basis for this and believe it may have been targeting a custom gitconfig shorthand syntax. ([#116598](https://github.com/kubernetes/kubernetes/pull/116598), [@natasha41575](https://github.com/natasha41575)) [SIG CLI] - When an unsupported PodDisruptionBudget configuration is found, an event and log will be emitted to inform users of the misconfiguration. ([#115861](https://github.com/kubernetes/kubernetes/pull/115861), [@JayKayy](https://github.com/JayKayy)) [SIG Apps] - [E2E] Pods spawned by E2E tests can now pull images from the private registry using the new --e2e-docker-config-file flag ([#114625](https://github.com/kubernetes/kubernetes/pull/114625), [@Divya063](https://github.com/Divya063)) [SIG Node and Testing] - [alpha: kubectl apply --prune --applyset] Enabled certain custom resources (CRs) to be used as `ApplySet` parent objects. To enable this for a given CR, apply the label `applyset.kubernetes.io/is-parent-type: true` to the CustomResourceDefinition (CRD) that defines it. ([#116353](https://github.com/kubernetes/kubernetes/pull/116353), [@KnVerey](https://github.com/KnVerey)) - `Kubelet` no longer creates certain legacy iptables rules by default. It is possible that this will cause problems with some third-party components that improperly depended on those rules. If this affects you, you can run `kubelet` with `--feature-gates=IPTablesOwnershipCleanup=false`, but a bug should also be filed against the third-party component. ([#114472](https://github.com/kubernetes/kubernetes/pull/114472), [@danwinship](https://github.com/danwinship)) - `MinDomainsInPodTopologySpread` feature gate is enabled by default as a Beta feature in 1.27. ([#114445](https://github.com/kubernetes/kubernetes/pull/114445), [@mengjiao-liu](https://github.com/mengjiao-liu)) - `Secret` of `kubernetes.io/tls` type now verifies that the private key matches the cert ([#113581](https://github.com/kubernetes/kubernetes/pull/113581), [@aimuz](https://github.com/aimuz)) - `StorageVersionGC` (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#113986](https://github.com/kubernetes/kubernetes/pull/113986), [@songxiao-wang87](https://github.com/songxiao-wang87)) - `client-go`: `sharedInformerFactory` now waits for goroutines during shutdown for metadatainformer and dynamicinformer. ([#114434](https://github.com/kubernetes/kubernetes/pull/114434), [@howardjohn](https://github.com/howardjohn)) - `kube-proxy` now accepts the `ContextualLogging`, `LoggingAlphaOptions`, `LoggingBetaOptions` ([#115233](https://github.com/kubernetes/kubernetes/pull/115233), [@pohly](https://github.com/pohly)) - `kube-scheduler`: Optimized implementation of null `labelSelector` in topology spreading. ([#116607](https://github.com/kubernetes/kubernetes/pull/116607), [@alculquicondor](https://github.com/alculquicondor)) - `kubeadm`: now shows a warning message when detecting that the sandbox image of the container runtime is inconsistent with that used by kubeadm ([#115610](https://github.com/kubernetes/kubernetes/pull/115610), [@SataQiu](https://github.com/SataQiu)) - `kubectl` now uses `HorizontalPodAutoscaler` `v2` by default. ([#114886](https://github.com/kubernetes/kubernetes/pull/114886), [@a7i](https://github.com/a7i)) - Kubernetes is now built with Go 1.20.3 ([#117125](https://github.com/kubernetes/kubernetes/pull/117125), [@xmudrii](https://github.com/xmudrii)) [SIG Release and Testing] - Updated distroless iptables to use released image `registry.k8s.io/build-image/distroless-iptables:v0.2.3` ([#117126](https://github.com/kubernetes/kubernetes/pull/117126), [@xmudrii](https://github.com/xmudrii)) [SIG Testing] ### Documentation - Documented the reason field in CRI API to ensure it equals `OOMKilled` for the containers terminated by OOM killer ([#112977](https://github.com/kubernetes/kubernetes/pull/112977), [@mimowo](https://github.com/mimowo)) - Error message for Pods with requests exceeding limits will have a limit value printed. ([#112925](https://github.com/kubernetes/kubernetes/pull/112925), [@SergeyKanzhelev](https://github.com/SergeyKanzhelev)) - The change affects the following CLI command: kubectl create rolebinding -h ([#107124](https://github.com/kubernetes/kubernetes/pull/107124), [@ptux](https://github.com/ptux)) [SIG CLI] ### Failing Test - Deflaked a preemption test that may patch Nodes incorrectly. ([#114350](https://github.com/kubernetes/kubernetes/pull/114350), [@Huang-Wei](https://github.com/Huang-Wei)) - Fixed panic in vSphere e2e tests. ([#115863](https://github.com/kubernetes/kubernetes/pull/115863), [@jsafrane](https://github.com/jsafrane)) [SIG Storage and Testing] - Setting the Kubelet config option `--resolv-conf=Host` on Windows will now result in Kubelet applying the Pod DNS Policies as intended. ([#110566](https://github.com/kubernetes/kubernetes/pull/110566), [@claudiubelu](https://github.com/claudiubelu)) ### Bug or Regression - Added (dry run) and (server dry run) suffixes to `kubectl scale` command when `dry-run` is passed ([#114252](https://github.com/kubernetes/kubernetes/pull/114252), [@ardaguclu](https://github.com/ardaguclu)) - Applied configurations can be generated for types with `non-builtin` map fields ([#114920](https://github.com/kubernetes/kubernetes/pull/114920), [@astefanutti](https://github.com/astefanutti)) - Changed the error message of `kubectl rollout restart` when subsequent `kubectl rollout restart` commands are executed within a second ([#113040](https://github.com/kubernetes/kubernetes/pull/113040), [@ardaguclu](https://github.com/ardaguclu)) - Changed the error message to `cannot exec into multiple objects at a time` when file passed to `kubectl exec` contains multiple resources ([#114249](https://github.com/kubernetes/kubernetes/pull/114249), [@ardaguclu](https://github.com/ardaguclu)) - Client-go: fixed potential data races retrying requests using a custom `io.Reader` body; with this fix, only requests with no body or with `string` / `[]byte` / `runtime.Object` bodies can be retried ([#113933](https://github.com/kubernetes/kubernetes/pull/113933), [@liggitt](https://github.com/liggitt)) - Describing the CRs will now hide `.metadata.managedFields` ([#114584](https://github.com/kubernetes/kubernetes/pull/114584), [@soltysh](https://github.com/soltysh)) - Discovery document will correctly return the resources for aggregated apiservers that do not implement aggregated disovery ([#115770](https://github.com/kubernetes/kubernetes/pull/115770), [@Jefftree](https://github.com/Jefftree)) - Excluded preemptor pod metadata in the event message ([#114923](https://github.com/kubernetes/kubernetes/pull/114923), [@mimowo](https://github.com/mimowo)) - Expanded the partial fix for https://github.com/kubernetes/kubernetes/issues/111539 which was already started in https://github.com/kubernetes/kubernetes/pull/109706 Specifically, we will now reduce the amount of syncs for `ETP=local` services even further in the CCM and avoid re-configuring LBs to an even greater extent. ([#111658](https://github.com/kubernetes/kubernetes/pull/111658), [@alexanderConstantinescu](https://github.com/alexanderConstantinescu)) - File content check for IPV4 is now enabled by default, and the check of IPV4 or IPV6 is done for `kubeadm init` or `kubeadm join` only in case the user intends to create a cluster to support that kind of IP address family ([#115420](https://github.com/kubernetes/kubernetes/pull/115420), [@chendave](https://github.com/chendave)) - Fixed CSI `PersistentVolumes` to allow Secrets names longer than 63 characters. ([#114776](https://github.com/kubernetes/kubernetes/pull/114776), [@jsafrane](https://github.com/jsafrane)) - Fixed Route controller to update routes when NodeIP changes ([#108095](https://github.com/kubernetes/kubernetes/pull/108095), [@lzhecheng](https://github.com/lzhecheng)) - Fixed `DaemonSet` to update the status even if it fails to create a pod. ([#113787](https://github.com/kubernetes/kubernetes/pull/113787), [@gjkim42](https://github.com/gjkim42)) - Fixed `SELinux` label for host path volumes created by host path provisioner ([#112021](https://github.com/kubernetes/kubernetes/pull/112021), [@mrunalp](https://github.com/mrunalp)) - Fixed `StatefulSetAutoDeletePVC` feature when `OwnerReferencesPermissionEnforcement` admission plugin is enabled. ([#114116](https://github.com/kubernetes/kubernetes/pull/114116), [@jsafrane](https://github.com/jsafrane)) - Fixed a bug on the `EndpointSlice` mirroring controller that generated multiple slices in some cases for custom endpoints in non canonical format. ([#114155](https://github.com/kubernetes/kubernetes/pull/114155), [@aojea](https://github.com/aojea)) - Fixed a bug that caused the `apiserver` to panic when trying to allocate a Service with a dynamic `ClusterIP` and was configured with Service CIDRs with a /28 mask for IPv4 and a /124 mask for IPv6 ([#115322](https://github.com/kubernetes/kubernetes/pull/115322), [@aojea](https://github.com/aojea)) - Fixed a bug where Kubernetes would apply a default StorageClass to a PersistentVolumeClaim, even when the deprecated annotation `volume.beta.kubernetes.io/storage-class` was set. ([#116089](https://github.com/kubernetes/kubernetes/pull/116089), [@cvvz](https://github.com/cvvz)) [SIG Apps and Storage] - Fixed a bug where `events/v1` `Events` with similar event type and reporting instance were not aggregated by `client-go`. ([#112365](https://github.com/kubernetes/kubernetes/pull/112365), [@dgrisonnet](https://github.com/dgrisonnet)) - Fixed a bug where when emitting similar Events consecutively, some were rejected by the apiserver. ([#114237](https://github.com/kubernetes/kubernetes/pull/114237), [@dgrisonnet](https://github.com/dgrisonnet)) - Fixed a data race when emitting similar Events consecutively ([#114236](https://github.com/kubernetes/kubernetes/pull/114236), [@dgrisonnet](https://github.com/dgrisonnet)) - Fixed a log line in scheduler that inaccurately implies that volume binding has finalized ([#116018](https://github.com/kubernetes/kubernetes/pull/116018), [@TommyStarK](https://github.com/TommyStarK)) - Fixed a rare race condition in `kube-apiserver` that could lead to missing events when a watch API request was created at the same time `kube-apiserver` was re-initializing its internal watch. ([#116172](https://github.com/kubernetes/kubernetes/pull/116172), [@wojtek-t](https://github.com/wojtek-t)) - Fixed a regression in the pod binding subresource to honor the `metadata.uid` precondition. This allows kube-scheduler to ensure it is assigns node names to the same instances of pods it made scheduling decisions for. ([#116550](https://github.com/kubernetes/kubernetes/pull/116550), [@alculquicondor](https://github.com/alculquicondor)) - Fixed a regression that the scheduler always goes through all Filter plugins. ([#114518](https://github.com/kubernetes/kubernetes/pull/114518), [@Huang-Wei](https://github.com/Huang-Wei)) - Fixed an EndpointSlice Controller hashing bug that could cause EndpointSlices to incorrectly handle Pods with duplicate IP addresses. For example this could happen when a new Pod reused an IP that was also assigned to a Pod in a completed state. ([#115907](https://github.com/kubernetes/kubernetes/pull/115907), [@qinqon](https://github.com/qinqon)) [SIG Apps and Network] - Fixed an issue where a CSI migrated volume may be prematurely detached when the CSI driver is not running on the node. If CSI migration is enabled on the node, even the csi-driver is not up and ready, we will still add this volume to DSW. ([#115464](https://github.com/kubernetes/kubernetes/pull/115464), [@sunnylovestiramisu](https://github.com/sunnylovestiramisu)) - Fixed an issue where failed pods associated with a job with `parallelism = 1` are recreated by the job controller honoring exponential backoff delay again. However, for jobs with `parallelism > 1`, pods might be created without exponential backoff delay. ([#114516](https://github.com/kubernetes/kubernetes/pull/114516), [@nikhita](https://github.com/nikhita)) - Fixed an issue with Winkernel Proxier - ClusterIP Loadbalancers missing if the `ExternalTrafficPolicy` is set to Local and the available endpoints are all `remoteEndpoints`. ([#115919](https://github.com/kubernetes/kubernetes/pull/115919), [@princepereira](https://github.com/princepereira)) - Fixed bug in CRD Validation Rules (beta) and `ValidatingAdmissionPolicy` (alpha) where all admission requests could result in `internal error: runtime error: index out of range [3] with length 3 evaluating rule: ` under certain circumstances. ([#114857](https://github.com/kubernetes/kubernetes/pull/114857), [@jpbetz](https://github.com/jpbetz)) - Fixed bug in beta aggregated discovery endpoint which caused CRD discovery information to be temporarily missing when an Aggregated APIService with the same GroupVersion is deleted (and vice versa). ([#116770](https://github.com/kubernetes/kubernetes/pull/116770), [@alexzielenski](https://github.com/alexzielenski)) - Fixed bug in reflector that couldn't recover from `Too large resource version` errors with API servers before 1.17.0. ([#115093](https://github.com/kubernetes/kubernetes/pull/115093), [@xuzhenglun](https://github.com/xuzhenglun)) - Fixed clearing of rate-limiter for the queue of checks for cleaning stale pod disruption conditions. The bug could result in the PDB synchronization updates firing too often or the pod disruption cleanups taking too long to happen. ([#114770](https://github.com/kubernetes/kubernetes/pull/114770), [@mimowo](https://github.com/mimowo)) - Fixed data race in `kube-scheduler` when preemption races with a Pod update. ([#116395](https://github.com/kubernetes/kubernetes/pull/116395), [@alculquicondor](https://github.com/alculquicondor)) [SIG Scheduling] - Fixed file permission issues that happened during update of `Secret`/`ConfigMap`/`projected volume` when `fsGroup` is used. The problem caused a race condition where application gets intermittent permission denied error when reading files that were just updated, before the correct permissions were applied. ([#114464](https://github.com/kubernetes/kubernetes/pull/114464), [@tsaarni](https://github.com/tsaarni)) - Fixed incorrect watch events when a watch is initialized simultanously with a reinitializing watchcache. ([#116436](https://github.com/kubernetes/kubernetes/pull/116436), [@wojtek-t](https://github.com/wojtek-t)) - Fixed issue in `Winkernel` Proxier - Unexpected active TCP connection drops while horizontally scaling the endpoints for a LoadBalancer Service with Internal Traffic Policy: `Local` ([#113742](https://github.com/kubernetes/kubernetes/pull/113742), [@princepereira](https://github.com/princepereira)) - Fixed issue on Windows when calculating cpu limits on nodes with more than 64 logical processors ([#114231](https://github.com/kubernetes/kubernetes/pull/114231), [@mweibel](https://github.com/mweibel)) - Fixed issue with Winkernel Proxier - IPV6 load balancer policies were missing when service was configured with `ipFamilyPolicy`: `RequireDualStack` ([#115503](https://github.com/kubernetes/kubernetes/pull/115503), [@princepereira](https://github.com/princepereira)) - Fixed issue with Winkernel Proxier - IPV6 load balancer policies were missing when service was configured with `ipFamilyPolicy`: `RequireDualStack` ([#115577](https://github.com/kubernetes/kubernetes/pull/115577), [@princepereira](https://github.com/princepereira)) - Fixed issue with `Winkernel Proxier` - No ingress load balancer rules with endpoints to support load balancing when all the endpoints are terminating. ([#113776](https://github.com/kubernetes/kubernetes/pull/113776), [@princepereira](https://github.com/princepereira)) - Fixed missing delete events on informer re-lists to ensure all delete events were correctly emitted and using the latest known object state, so that all event handlers and stores always reflect the actual apiserver state as best as possible ([#115620](https://github.com/kubernetes/kubernetes/pull/115620), [@odinuge](https://github.com/odinuge)) - Fixed nil pointer error in `NodeVolumeLimits` csi logging ([#115179](https://github.com/kubernetes/kubernetes/pull/115179), [@sunnylovestiramisu](https://github.com/sunnylovestiramisu)) - Fixed panic validating custom resource definition schemas that set `multipleOf` to 0 ([#114869](https://github.com/kubernetes/kubernetes/pull/114869), [@liggitt](https://github.com/liggitt)) - Fixed performance regression in scheduler caused by frequent metric lookup on critical code path. ([#116428](https://github.com/kubernetes/kubernetes/pull/116428), [@mborsz](https://github.com/mborsz)) [SIG Scheduling] - Fixed stuck apiserver if an aggregated apiservice returned `304 Not Modified` for aggregated discovery information ([#114459](https://github.com/kubernetes/kubernetes/pull/114459), [@alexzielenski](https://github.com/alexzielenski)) - Fixed the problem Pod terminating stuck because of trying to umount not actual mounted dir. ([#115769](https://github.com/kubernetes/kubernetes/pull/115769), [@mochizuki875](https://github.com/mochizuki875)) - Fixed the regression that introduced 34s timeout for DELETECOLLECTION calls ([#115341](https://github.com/kubernetes/kubernetes/pull/115341), [@tkashem](https://github.com/tkashem)) - Fixed two regressions introduced by the `PodDisruptionConditions` feature (on by default in 1.26): - pod eviction API calls returned spurious precondition errors and required a second evict API call to succeed - dry-run eviction API calls persisted a DisruptionTarget condition into the pod being evicted ([#116554](https://github.com/kubernetes/kubernetes/pull/116554), [@atiratree](https://github.com/atiratree)) - Fixes #115825. Kube-proxy will now include the `healthz` state in its response to the LB HC as to avoid indicating to the LB that it should use the node in question when Kube-proxy is not healthy. ([#111661](https://github.com/kubernetes/kubernetes/pull/111661), [@alexanderConstantinescu](https://github.com/alexanderConstantinescu)) [SIG Network] - Flag `--concurrent-node-syncs` has been added to cloud node controller which defines how many workers in parallel will be initialising and synchronising nodes. ([#113104](https://github.com/kubernetes/kubernetes/pull/113104), [@pawbana](https://github.com/pawbana)) [SIG API Machinery, Cloud Provider and Scalability] - Force deleted pods may fail to terminate until the kubelet is restarted when the container runtime returns an error during termination. We have strengthened testing for runtime failures and now perform a more rigorous reconciliation to ensure static pods (especially those that use fixed UIDs) are restarted. As a side effect of these changes static pods will be restarted with lower latency than before (2s vs 4s, on average) and rapid updates to pod configuration should take effect sooner. A new metric `kubelet_known_pods` has been added at ALPHA stability to report the number of pods a Kubelet is tracking in a number of internal states. Operators may use the metrics to track an excess of pods in the orphaned state that may not be completing. ([#113145](https://github.com/kubernetes/kubernetes/pull/113145), [@smarterclayton](https://github.com/smarterclayton)) [SIG API Machinery, Auth, Cloud Provider, Node and Testing] - From now on, the HPA controller will return an error for the container resource metrics when the feature gate `HPAContainerMetrics` is disabled. As a result, HPA with a container resource metric performs no scale-down and performs only. ([#116043](https://github.com/kubernetes/kubernetes/pull/116043), [@sanposhiho](https://github.com/sanposhiho)) - IPVS: Any ipvs scheduler can now be configured. If a un-usable scheduler is configured `kube-proxy` will re-start and the logs must be checked (same as before but different log printouts). ([#114878](https://github.com/kubernetes/kubernetes/pull/114878), [@uablrek](https://github.com/uablrek)) - If a user attempts to add an ephemeral container to a static pod, they will now get a visible validation error. ([#114086](https://github.com/kubernetes/kubernetes/pull/114086), [@xmcqueen](https://github.com/xmcqueen)) - Ingress with `ingressClass` annotation and `IngressClassName` both set can be created now. ([#115447](https://github.com/kubernetes/kubernetes/pull/115447), [@AxeZhan](https://github.com/AxeZhan)) - Kube-apiserver: errors decoding objects in etcd are now recorded in an `apiserver_storage_decode_errors_total` counter metric ([#114376](https://github.com/kubernetes/kubernetes/pull/114376), [@baomingwang](https://github.com/baomingwang)) [SIG API Machinery and Instrumentation] - Kube-apiserver: regular expressions specified with the `--cors-allowed-origins` option are now validated to match the entire `hostname` inside the `Origin` header of the request and must contain '^' or the '//' prefix to anchor to the start, and '$' or the port separator ':' to anchor to the end. ([#112809](https://github.com/kubernetes/kubernetes/pull/112809), [@tkashem](https://github.com/tkashem)) [SIG API Machinery] - Kube-apiserver: removed N^2 behavior loading webhook configurations. ([#114794](https://github.com/kubernetes/kubernetes/pull/114794), [@lavalamp](https://github.com/lavalamp)) [SIG API Machinery, Architecture, CLI, Cloud Provider and Node] - Kubeadm: fixed an etcd learner-mode bug by preparing an etcd static pod manifest before promoting ([#115038](https://github.com/kubernetes/kubernetes/pull/115038), [@tobiasgiese](https://github.com/tobiasgiese)) - Kubeadm: fixed the bug where `kubeadm` always does CRI detection even if it is not required by a phase subcommand ([#114455](https://github.com/kubernetes/kubernetes/pull/114455), [@SataQiu](https://github.com/SataQiu)) - Kubeadm: improved retries when updating node information, in case `kube-apiserver` is temporarily unavailable ([#114176](https://github.com/kubernetes/kubernetes/pull/114176), [@QuantumEnergyE](https://github.com/QuantumEnergyE)) - Kubeadm`: modified `--config` flag from required to optional for `kubeadm kubeconfig user` command ([#116074](https://github.com/kubernetes/kubernetes/pull/116074), [@SataQiu](https://github.com/SataQiu)) - Kubectl: enabled usage of label selector for filtering out resources when pruning for kubectl diff ([#114863](https://github.com/kubernetes/kubernetes/pull/114863), [@danlenar](https://github.com/danlenar)) - Kubelet startup now fails CRI connection if service or image endpoint is throwing any error ([#115102](https://github.com/kubernetes/kubernetes/pull/115102), [@saschagrunert](https://github.com/saschagrunert)) - Kubelet: fix recording issue when pulling image did finish ([#114904](https://github.com/kubernetes/kubernetes/pull/114904), [@TommyStarK](https://github.com/TommyStarK)) [SIG Node] - Kubelet`: fixed a bug in `kubelet` that stopped rendering the `ConfigMaps` when `fsquota` monitoring is enabled ([#112624](https://github.com/kubernetes/kubernetes/pull/112624), [@pacoxu](https://github.com/pacoxu)) - Messages of `DisruptionTarget` condition now excludes preemptor pod metadata ([#114914](https://github.com/kubernetes/kubernetes/pull/114914), [@mimowo](https://github.com/mimowo)) - Optimized `LoadBalancer` creation with the help of attribute Internal Traffic Policy: `Local` ([#114407](https://github.com/kubernetes/kubernetes/pull/114407), [@princepereira](https://github.com/princepereira)) - PVCs will automatically be recreated if they are missing for a pending Pod. ([#113270](https://github.com/kubernetes/kubernetes/pull/113270), [@rrangith](https://github.com/rrangith)) [SIG Apps and Testing] - PersistentVolume API objects which set NodeAffinities using beta Kubernetes labels for OS, architecture, zone, region, and instance type may now be modified to use the stable Kubernetes labels. ([#115391](https://github.com/kubernetes/kubernetes/pull/115391), [@haoruan](https://github.com/haoruan)) - Potentially breaking change - Updating the polling interval for Windows stats collection from 1 second to 10 seconds ([#116546](https://github.com/kubernetes/kubernetes/pull/116546), [@marosset](https://github.com/marosset)) [SIG Node and Windows] - Relaxed API validation for usage `key encipherment` and `kubelet` uses requested usages accordingly ([#111660](https://github.com/kubernetes/kubernetes/pull/111660), [@pacoxu](https://github.com/pacoxu)) - Removed scheduler names from preemption event messages. ([#114980](https://github.com/kubernetes/kubernetes/pull/114980), [@mimowo](https://github.com/mimowo)) - Shared informers now correctly propagate whether they are synced or not. Individual informer handlers may now check if they are synced or not (new `HasSynced` method). Library support is added to assist controllers in tracking whether their own work is completed for items in the initial list (`AsyncTracker`). ([#113985](https://github.com/kubernetes/kubernetes/pull/113985), [@lavalamp](https://github.com/lavalamp)) - The Kubernetes API server now correctly detects and closes existing TLS connections when its client certificate file for kubelet authentication has been rotated. ([#115315](https://github.com/kubernetes/kubernetes/pull/115315), [@enj](https://github.com/enj)) [SIG API Machinery, Auth, Node and Testing] - Total test spec is now available by `ProgressReporter`, it will be reported before test suite got executed. ([#114417](https://github.com/kubernetes/kubernetes/pull/114417), [@chendave](https://github.com/chendave)) - Updated the Event series starting count when emitting isomorphic events from 1 to 2. ([#112334](https://github.com/kubernetes/kubernetes/pull/112334), [@dgrisonnet](https://github.com/dgrisonnet)) - When GCing pods, `kube-controller-manager` will delete Evicted pods first. ([#116167](https://github.com/kubernetes/kubernetes/pull/116167), [@borgerli](https://github.com/borgerli)) - When describing deployments, `OldReplicaSets` now always shows all replicasets controlled the deployment, not just those that still have replicas available. ([#113083](https://github.com/kubernetes/kubernetes/pull/113083), [@llorllale](https://github.com/llorllale)) [SIG CLI] - Windows CPU usage node stats are now correctly calculated for nodes with multiple Processor Groups. ([#110864](https://github.com/kubernetes/kubernetes/pull/110864), [@claudiubelu](https://github.com/claudiubelu)) [SIG Node, Testing and Windows] - `LabelSelectors` specified in `topologySpreadConstraints` were validated to ensure that pods are scheduled as expected. Existing pods with invalid `LabelSelectors` could be updated, but new pods were required to specify valid `LabelSelectors`. ([#111802](https://github.com/kubernetes/kubernetes/pull/111802), [@maaoBit](https://github.com/maaoBit)) - `PodGC` for pods which are in terminal phase now do not add the `DisruptionTarget` condition. ([#115056](https://github.com/kubernetes/kubernetes/pull/115056), [@mimowo](https://github.com/mimowo)) - `Service` of type `ExternalName` do not create an `Endpoint` anymore. ([#114814](https://github.com/kubernetes/kubernetes/pull/114814), [@panslava](https://github.com/panslava)) - `cacher`: If `ResourceVersion` is unset, the watch is now served from the underlying storage as documented. ([#115096](https://github.com/kubernetes/kubernetes/pull/115096), [@MadhavJivrajani](https://github.com/MadhavJivrajani)) - `client-go`: fixed the wait time for trying to acquire the leader lease ([#114872](https://github.com/kubernetes/kubernetes/pull/114872), [@Iceber](https://github.com/Iceber)) - `etcd`: Updated to `v3.5.7` ([#115310](https://github.com/kubernetes/kubernetes/pull/115310), [@mzaian](https://github.com/mzaian)) - `golang.org/x/net` updated to `v0.7.0` to fix CVE-2022-41723 ([#115786](https://github.com/kubernetes/kubernetes/pull/115786), [@liggitt](https://github.com/liggitt)) - `kube-controller-manager` will not run nodeipam controller when allocator type is `CloudAllocator` and the cloud provider is not enabled. ([#114596](https://github.com/kubernetes/kubernetes/pull/114596), [@andrewsykim](https://github.com/andrewsykim)) - `kube-controller-manager`: fixed a bug that the `kubeconfig` field of `kubecontrollermanager.config.k8s.io` configuration is not populated correctly ([#116219](https://github.com/kubernetes/kubernetes/pull/116219), [@SataQiu](https://github.com/SataQiu)) - `kube-proxy` with `--proxy-mode=ipvs` can be used with statically linked kernels. The reseved IPv4 range `TEST-NET-2` in `rfc5737` MUST NOT be used for `ClusterIP` or `loadBalancerIP` since address `198.51.100.0` is used for probing. ([#114669](https://github.com/kubernetes/kubernetes/pull/114669), [@uablrek](https://github.com/uablrek)) - `kubeadm`: fixed a bug where the uploaded kubelet configuration in `kube-system/kubelet-config` `ConfigMap` does not respect user patch ([#115575](https://github.com/kubernetes/kubernetes/pull/115575), [@SataQiu](https://github.com/SataQiu)) - `kubeadm`: now respects user provided `kubeconfig` during discovery process ([#113998](https://github.com/kubernetes/kubernetes/pull/113998), [@SataQiu](https://github.com/SataQiu)) - `kubectl port-forward` now exits with exit code 1 when remote connection is lost ([#114460](https://github.com/kubernetes/kubernetes/pull/114460), [@brianpursley](https://github.com/brianpursley)) - `nodeName` being set along with non-empty `schedulingGates` is now enforced. ([#115569](https://github.com/kubernetes/kubernetes/pull/115569), [@Huang-Wei](https://github.com/Huang-Wei)) - `node_stage_path` is now set whenever available for expansion during mount ([#115346](https://github.com/kubernetes/kubernetes/pull/115346), [@gnufied](https://github.com/gnufied)) - `statefulset` status will now be consistent on API errors ([#113834](https://github.com/kubernetes/kubernetes/pull/113834), [@atiratree](https://github.com/atiratree)) - `tryUnmount` now respects `mounter.withSafeNotMountedBehavior` ([#114736](https://github.com/kubernetes/kubernetes/pull/114736), [@andyzhangx](https://github.com/andyzhangx)) - The encryption response from KMS v2 plugins is now validated earlier at DEK generation time instead of waiting until an encryption is performed. ([#116877](https://github.com/kubernetes/kubernetes/pull/116877), [@enj](https://github.com/enj)) [SIG API Machinery and Auth] - Recreate DaemonSet pods completed with Succeeded phase ([#117073](https://github.com/kubernetes/kubernetes/pull/117073), [@mimowo](https://github.com/mimowo)) [SIG Apps and Testing] ### Other (Cleanup or Flake) - Added basic Denial Of Service prevention for the the node-local kubelet `podresource` API ([#116459](https://github.com/kubernetes/kubernetes/pull/116459), [@ffromani](https://github.com/ffromani)) [SIG Node and Testing] - Callers of `wait.ExponentialBackoffWithContext` now must pass a `ConditionWithContextFunc` to be consistent with the signature and avoid creating a duplicate context. If your condition does not need a context you can use the `ConditionFunc.WithContext()` helper to ignore the context, or use `ExponentialBackoff` directly. ([#115113](https://github.com/kubernetes/kubernetes/pull/115113), [@smarterclayton](https://github.com/smarterclayton)) - Changed docs for `--contention-profiling` flag to reflect it performed block profiling ([#114490](https://github.com/kubernetes/kubernetes/pull/114490), [@MadhavJivrajani](https://github.com/MadhavJivrajani)) - E2e framework: added `--report-complete-ginkgo` and `--report-complete-junit` parameters. They work like `ginkgo --json-report /ginkgo/report.json --junit-report /ginkgo/report.xml`. ([#115678](https://github.com/kubernetes/kubernetes/pull/115678), [@pohly](https://github.com/pohly)) [SIG Testing] - Fixed incorrect log information in the `iptables` utility. ([#110723](https://github.com/kubernetes/kubernetes/pull/110723), [@yangjunmyfm192085](https://github.com/yangjunmyfm192085)) - Improved FormatMap: Improves performance by about 4x, or nearly 2x in the worst case ([#112661](https://github.com/kubernetes/kubernetes/pull/112661), [@aimuz](https://github.com/aimuz)) [SIG Node] - Improved misleading message, in case of no metrics received for the `HPA` controlled pods. ([#114740](https://github.com/kubernetes/kubernetes/pull/114740), [@kushagra98](https://github.com/kushagra98)) - Introduced new metrics removing the redundant subsystem in kube-apiserver pod logs metrics and deprecate the original ones: - kube_apiserver_pod_logs_pods_logs_backend_tls_failure_total becomes kube_apiserver_pod_logs_backend_tls_failure_total - kube_apiserver_pod_logs_pods_logs_insecure_backend_total becomes kube_apiserver_pod_logs_insecure_backend_total ([#114497](https://github.com/kubernetes/kubernetes/pull/114497), [@dgrisonnet](https://github.com/dgrisonnet)) - Kubeadm: removed the deprecated `v1beta2` API. kubeadm 1.26's `config migrate` command can be used to migrate a `v1beta2` configuration file to `v1beta3` ([#114540](https://github.com/kubernetes/kubernetes/pull/114540), [@pacoxu](https://github.com/pacoxu)) - Kubelet: remove deprecated flag `--container-runtime` ([#114017](https://github.com/kubernetes/kubernetes/pull/114017), [@calvin0327](https://github.com/calvin0327)) [SIG Cloud Provider and Node] - Kubelet: the deprecated `--master-service-namespace` flag is removed in v1.27 ([#116015](https://github.com/kubernetes/kubernetes/pull/116015), [@SataQiu](https://github.com/SataQiu)) - Linux/arm will not ship in Kubernetes 1.27 as we are running into issues with building artifacts using golang 1.20.2 (please see issue #116492) ([#115742](https://github.com/kubernetes/kubernetes/pull/115742), [@dims](https://github.com/dims)) [SIG Architecture, Release and Testing] - Migrated `pkg/controller/nodeipam/ipam/cloud_cidr_allocator.go, pkg/controller/nodeipam/ipam/multi_cidr_range_allocator.go pkg/controller/nodeipam/ipam/range_allocator.go pkg/controller/nodelifecycle/node_lifecycle_controller.go` to structured logging ([#112670](https://github.com/kubernetes/kubernetes/pull/112670), [@yangjunmyfm192085](https://github.com/yangjunmyfm192085)) - Migrated the Kubernetes object garbage collector (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#113471](https://github.com/kubernetes/kubernetes/pull/113471), [@ncdc](https://github.com/ncdc)) - Migrated the ttlafterfinished controller (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#115332](https://github.com/kubernetes/kubernetes/pull/115332), [@obaranov1](https://github.com/obaranov1)) [SIG Apps] - Migrated the “sample-controller” controller to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#113879](https://github.com/kubernetes/kubernetes/pull/113879), [@pchan](https://github.com/pchan)) [SIG API Machinery and Instrumentation] - Promoted pod resource `limit/request` metrics to stable. ([#115454](https://github.com/kubernetes/kubernetes/pull/115454), [@dgrisonnet](https://github.com/dgrisonnet)) - Removed AWS kubelet credential provider. Please use the external kubelet credential provider binary named `ecr-credential-provider` instead. ([#116329](https://github.com/kubernetes/kubernetes/pull/116329), [@dims](https://github.com/dims)) [SIG Node, Storage and Testing] - Removed Azure disk in-tree storage plugin ([#116301](https://github.com/kubernetes/kubernetes/pull/116301), [@andyzhangx](https://github.com/andyzhangx)) - Removed flag `master-service-namespace` from `api-server` arguments ([#114446](https://github.com/kubernetes/kubernetes/pull/114446), [@lengrongfu](https://github.com/lengrongfu)) - Removed the following deprecated metrics: - node_collector_evictions_number replaced by node_collector_evictions_total - scheduler_e2e_scheduling_duration_seconds replaced by scheduler_scheduling_attempt_duration_seconds ([#115209](https://github.com/kubernetes/kubernetes/pull/115209), [@dgrisonnet](https://github.com/dgrisonnet)) - Removed unused rule for `nodes/spec` from `ClusterRole` `system:kubelet-api-admin` ([#113267](https://github.com/kubernetes/kubernetes/pull/113267), [@hoskeri](https://github.com/hoskeri)) - Renamed API server identity Lease labels to use the key `apiserver.kubernetes.io/identity` ([#114586](https://github.com/kubernetes/kubernetes/pull/114586), [@andrewsykim](https://github.com/andrewsykim)) [SIG API Machinery, Apps, Cloud Provider and Testing] - Storage.k8s.io/v1beta1 API version of CSIStorageCapacity will no longer be served ([#116523](https://github.com/kubernetes/kubernetes/pull/116523), [@pacoxu](https://github.com/pacoxu)) [SIG API Machinery] - The `CSIMigrationAzureFile` feature gate (for the feature which graduated to GA in v1.26) is now unconditionally enabled and will be removed in v1.28. ([#114953](https://github.com/kubernetes/kubernetes/pull/114953), [@enj](https://github.com/enj)) - The `ControllerManagerLeaderMigration` feature, GA since `1.24`, is now unconditionally enabled and the feature gate option has been removed. ([#113534](https://github.com/kubernetes/kubernetes/pull/113534), [@pacoxu](https://github.com/pacoxu)) - The `WaitFor` and `WaitForWithContext` functions in the wait package have now been marked private. Callers should use the equivalent `Poll*` method with a zero duration interval. ([#115116](https://github.com/kubernetes/kubernetes/pull/115116), [@smarterclayton](https://github.com/smarterclayton)) - The `wait.Poll*` and `wait.ExponentialBackoff*` functions have been deprecated and will be removed in a future release. Callers should switch to using `wait.PollUntilContextCancel`, `wait.PollUntilContextTimeout`, or `wait.ExponentialBackoffWithContext` as appropriate. `PollWithContext(Cancel|Deadline)` will no longer return `ErrWaitTimeout` - use the `Interrupted(error) bool` helper to replace checks for `err == ErrWaitTimeout`, or compare specifically to context errors as needed. A future release will make the `ErrWaitTimeout` error private and callers must use `Interrupted()` instead. If you are returning `ErrWaitTimeout` from your own methods, switch to creating a location specific `cause err` and pass it to the new method `wait.ErrorInterrupted(cause) error` which will ensure `Interrupted()` returns true for your loop. The `wait.NewExponentialBackoffManager` and `wait.NewJitteringBackoffManager` functions have been marked as deprecated. Callers should switch to using the `Backoff{...}.DelayWithReset(clock, resetInterval)` method and must set the `Steps` field when using `Factor`. As a short term change, callers may use the `Timer()` method on the `BackoffManager` until the backoff managers are deprecated and removed. Please see the godoc of the deprecated functions for examples of how to replace usage of this function. ([#107826](https://github.com/kubernetes/kubernetes/pull/107826), [@smarterclayton](https://github.com/smarterclayton)) [SIG API Machinery, Auth, Cloud Provider, Storage and Testing] - The feature gates `CSIInlineVolume`, `CSIMigration`, `DaemonSetUpdateSurge`, `EphemeralContainers`, `IdentifyPodOS`, `LocalStorageCapacityIsolation`, `NetworkPolicyEndPort` and `StatefulSetMinReadySeconds` that graduated to GA in v1.25 and were unconditionally enabled have been removed in v1.27 ([#114410](https://github.com/kubernetes/kubernetes/pull/114410), [@SataQiu](https://github.com/SataQiu)) [SIG Node] - Upgraded `coredns` to `v1.10.1` ([#115603](https://github.com/kubernetes/kubernetes/pull/115603), [@pacoxu](https://github.com/pacoxu)) - Upgraded `go-jose` to `v2.6.0` ([#115893](https://github.com/kubernetes/kubernetes/pull/115893), [@mgoltzsche](https://github.com/mgoltzsche)) - [KCCM - service controller]: enabled connection draining for terminating pods upon node downscale by the cluster autoscaler. This is done by not reacting to the taint used by the cluster autoscaler to indicate that the node is going away soon, thus keeping the node referenced by the load balancer until the VM has been completely deleted. ([#115204](https://github.com/kubernetes/kubernetes/pull/115204), [@alexanderConstantinescu](https://github.com/alexanderConstantinescu)) - `apiserver_admission_webhook_admission_duration_seconds` buckets have been expanded, 25s is now the largest bucket size to match the webhook default timeout. ([#115802](https://github.com/kubernetes/kubernetes/pull/115802), [@logicalhan](https://github.com/logicalhan)) [SIG API Machinery and Instrumentation] - `wait.ContextForChannel()` now implements the context.Context interface and does not return a cancellation function. ([#115140](https://github.com/kubernetes/kubernetes/pull/115140), [@smarterclayton](https://github.com/smarterclayton)) ## Dependencies ### Added - github.com/a8m/tree: [10a5fd5](https://github.com/a8m/tree/tree/10a5fd5) - github.com/dougm/pretty: [2ee9d74](https://github.com/dougm/pretty/tree/2ee9d74) - github.com/rasky/go-xdr: [4930550](https://github.com/rasky/go-xdr/tree/4930550) - github.com/vmware/vmw-guestinfo: [25eff15](https://github.com/vmware/vmw-guestinfo/tree/25eff15) - sigs.k8s.io/kustomize/kustomize/v5: v5.0.1 ### Changed - github.com/Microsoft/hcsshim: [v0.8.22 → v0.8.25](https://github.com/Microsoft/hcsshim/compare/v0.8.22...v0.8.25) - github.com/aws/aws-sdk-go: [v1.44.116 → v1.35.24](https://github.com/aws/aws-sdk-go/compare/v1.44.116...v1.35.24) - github.com/coredns/corefile-migration: [v1.0.17 → v1.0.20](https://github.com/coredns/corefile-migration/compare/v1.0.17...v1.0.20) - github.com/coreos/go-systemd/v22: [v22.3.2 → v22.4.0](https://github.com/coreos/go-systemd/v22/compare/v22.3.2...v22.4.0) - github.com/creack/pty: [v1.1.11 → v1.1.18](https://github.com/creack/pty/compare/v1.1.11...v1.1.18) - github.com/docker/docker: [v20.10.18+incompatible → v20.10.21+incompatible](https://github.com/docker/docker/compare/v20.10.18...v20.10.21) - github.com/go-errors/errors: [v1.0.1 → v1.4.2](https://github.com/go-errors/errors/compare/v1.0.1...v1.4.2) - github.com/go-openapi/jsonpointer: [v0.19.5 → v0.19.6](https://github.com/go-openapi/jsonpointer/compare/v0.19.5...v0.19.6) - github.com/go-openapi/jsonreference: [v0.20.0 → v0.20.1](https://github.com/go-openapi/jsonreference/compare/v0.20.0...v0.20.1) - github.com/go-openapi/swag: [v0.19.14 → v0.22.3](https://github.com/go-openapi/swag/compare/v0.19.14...v0.22.3) - github.com/golang-jwt/jwt/v4: [v4.2.0 → v4.4.2](https://github.com/golang-jwt/jwt/v4/compare/v4.2.0...v4.4.2) - github.com/golang/protobuf: [v1.5.2 → v1.5.3](https://github.com/golang/protobuf/compare/v1.5.2...v1.5.3) - github.com/google/cadvisor: [v0.46.0 → v0.47.1](https://github.com/google/cadvisor/compare/v0.46.0...v0.47.1) - github.com/google/cel-go: [v0.12.5 → v0.12.6](https://github.com/google/cel-go/compare/v0.12.5...v0.12.6) - github.com/google/uuid: [v1.1.2 → v1.3.0](https://github.com/google/uuid/compare/v1.1.2...v1.3.0) - github.com/kr/pretty: [v0.2.1 → v0.3.0](https://github.com/kr/pretty/compare/v0.2.1...v0.3.0) - github.com/mailru/easyjson: [v0.7.6 → v0.7.7](https://github.com/mailru/easyjson/compare/v0.7.6...v0.7.7) - github.com/moby/ipvs: [v1.0.1 → v1.1.0](https://github.com/moby/ipvs/compare/v1.0.1...v1.1.0) - github.com/moby/term: [39b0c02 → 1aeaba8](https://github.com/moby/term/compare/39b0c02...1aeaba8) - github.com/onsi/ginkgo/v2: [v2.4.0 → v2.9.1](https://github.com/onsi/ginkgo/v2/compare/v2.4.0...v2.9.1) - github.com/onsi/gomega: [v1.23.0 → v1.27.4](https://github.com/onsi/gomega/compare/v1.23.0...v1.27.4) - github.com/opencontainers/runtime-spec: [1c3f411 → 494a5a6](https://github.com/opencontainers/runtime-spec/compare/1c3f411...494a5a6) - github.com/rogpeppe/go-internal: [v1.3.0 → v1.10.0](https://github.com/rogpeppe/go-internal/compare/v1.3.0...v1.10.0) - github.com/sirupsen/logrus: [v1.8.1 → v1.9.0](https://github.com/sirupsen/logrus/compare/v1.8.1...v1.9.0) - github.com/stretchr/objx: [v0.4.0 → v0.5.0](https://github.com/stretchr/objx/compare/v0.4.0...v0.5.0) - github.com/stretchr/testify: [v1.8.0 → v1.8.1](https://github.com/stretchr/testify/compare/v1.8.0...v1.8.1) - github.com/tmc/grpc-websocket-proxy: [e5319fd → 673ab2c](https://github.com/tmc/grpc-websocket-proxy/compare/e5319fd...673ab2c) - github.com/vishvananda/netns: [db3c7e5 → v0.0.2](https://github.com/vishvananda/netns/compare/db3c7e5...v0.0.2) - github.com/vmware/govmomi: [v0.20.3 → v0.30.0](https://github.com/vmware/govmomi/compare/v0.20.3...v0.30.0) - go.etcd.io/etcd/api/v3: v3.5.5 → v3.5.7 - go.etcd.io/etcd/client/pkg/v3: v3.5.5 → v3.5.7 - go.etcd.io/etcd/client/v2: v2.305.5 → v2.305.7 - go.etcd.io/etcd/client/v3: v3.5.5 → v3.5.7 - go.etcd.io/etcd/pkg/v3: v3.5.5 → v3.5.7 - go.etcd.io/etcd/raft/v3: v3.5.5 → v3.5.7 - go.etcd.io/etcd/server/v3: v3.5.5 → v3.5.7 - go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp: v0.35.0 → v0.35.1 - go.uber.org/goleak: v1.2.0 → v1.2.1 - golang.org/x/mod: v0.6.0 → v0.9.0 - golang.org/x/net: 1e63c2f → v0.8.0 - golang.org/x/sync: 886fb93 → v0.1.0 - golang.org/x/sys: v0.3.0 → v0.6.0 - golang.org/x/term: v0.3.0 → v0.6.0 - golang.org/x/text: v0.5.0 → v0.8.0 - golang.org/x/tools: v0.2.0 → v0.7.0 - golang.org/x/xerrors: 5ec99f8 → 04be3eb - google.golang.org/grpc: v1.49.0 → v1.51.0 - gopkg.in/check.v1: 8fa4692 → 10cb982 - gopkg.in/square/go-jose.v2: v2.2.2 → v2.6.0 - k8s.io/klog/v2: v2.80.1 → v2.90.1 - k8s.io/kube-openapi: 172d655 → 15aac26 - k8s.io/utils: 1a15be2 → a36077c - sigs.k8s.io/apiserver-network-proxy/konnectivity-client: v0.0.33 → v0.1.1 - sigs.k8s.io/json: f223a00 → bc3834c - sigs.k8s.io/kustomize/api: v0.12.1 → v0.13.2 - sigs.k8s.io/kustomize/cmd/config: v0.10.9 → v0.11.1 - sigs.k8s.io/kustomize/kyaml: v0.13.9 → v0.14.1 ### Removed - github.com/PuerkitoBio/purell: [v1.1.1](https://github.com/PuerkitoBio/purell/tree/v1.1.1) - github.com/PuerkitoBio/urlesc: [de5bf2a](https://github.com/PuerkitoBio/urlesc/tree/de5bf2a) - github.com/elazarl/goproxy: [947c36d](https://github.com/elazarl/goproxy/tree/947c36d) - github.com/form3tech-oss/jwt-go: [v3.2.3+incompatible](https://github.com/form3tech-oss/jwt-go/tree/v3.2.3) - github.com/mattn/go-runewidth: [v0.0.7](https://github.com/mattn/go-runewidth/tree/v0.0.7) - github.com/mindprince/gonvml: [9ebdce4](https://github.com/mindprince/gonvml/tree/9ebdce4) - github.com/niemeyer/pretty: [a10e7ca](https://github.com/niemeyer/pretty/tree/a10e7ca) - github.com/olekukonko/tablewriter: [v0.0.4](https://github.com/olekukonko/tablewriter/tree/v0.0.4) - sigs.k8s.io/kustomize/kustomize/v4: v4.5.7 # v1.27.0-rc.1 ## Downloads for v1.27.0-rc.1 ### Source Code filename | sha512 hash -------- | ----------- [kubernetes.tar.gz](https://dl.k8s.io/v1.27.0-rc.1/kubernetes.tar.gz) | f6a57401347cb6c6329f4334d0f5b9408125784c13cac8c69288e49fac9fcf057ba9b38340e170c9ee24840bfd9c6e63df5706760837e321efc2ce4da795d6cb [kubernetes-src.tar.gz](https://dl.k8s.io/v1.27.0-rc.1/kubernetes-src.tar.gz) | 62cca03a925930f58083070e3877df2d3de0fc5a2a96ce4079931fab77c77f10cdd739d1ac9f64c16c3dac107f075c6f112d8ed063e3f466d662e55271487e10 ### Client Binaries filename | sha512 hash -------- | ----------- [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.27.0-rc.1/kubernetes-client-darwin-amd64.tar.gz) | 3cf168843bd207f0c277465560c900e09487a3ff5eca6877afc5ca947bf164b7fc20f33c5379783ddf55380ca8370a44e400b37216229496a89309242a6f9bbf [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.27.0-rc.1/kubernetes-client-darwin-arm64.tar.gz) | 1b6c7eab5bb7cfe437400049c1d5bb7320828f975c63f3ac07d9e0ef943799f5fc7e0cd283aa6486da5ada075367447963b1346c96ebdb49632535e2f90dd664 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.27.0-rc.1/kubernetes-client-linux-386.tar.gz) | 7d4e84fd2fa4a06890c9a5a8fc2af2e01c1f3516eb8d164ef4c97a554536f845993f096828469e845392d39576baa7ba5fa125dabeb17c049f8556ff07d941e6 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.0-rc.1/kubernetes-client-linux-amd64.tar.gz) | f300c80e465f32b3c365e9c036c02aaaa3e713fa9e7318ce9e426406e07be6f3e6166664d9d940e2c3f073c3044f2d677f32a14e87a1e40acd973625634baf59 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.27.0-rc.1/kubernetes-client-linux-arm.tar.gz) | 5152fddf3a01b83c89070403d37c43b46daad2ccf566f484f9f9eaa776b3d022672d902dbef1edd7b8f4241c035208f83d7049c2cf6b76ef1da8d7ffb41d86b3 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.0-rc.1/kubernetes-client-linux-arm64.tar.gz) | 955c4019d6e52475bb6cf1df7d3db70ea6dbe226b54e145260f4470c53a0c12128e747ac0fae9843b2ed9d598eb3fbe964df2999a999f89726abd63feadb7ccf [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.0-rc.1/kubernetes-client-linux-ppc64le.tar.gz) | feed26863b79c6a67c3ab7842e11663fa4861982666b451ff13d89c7659482ed08c174b2d70517ef542ba17423e5aff172e30806e17e473815c05d5f6d1c431e [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.0-rc.1/kubernetes-client-linux-s390x.tar.gz) | 6d3704b1edc07ff244b85e9a387f283524cc9975a527a23c567af0554ebe61f564d10068fe6af5635d71ed2d9e01ca1dc79716177c449493cbbd32903b4c11fd [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.27.0-rc.1/kubernetes-client-windows-386.tar.gz) | 4abff8190b3e843b603fae1b6236815c7f15510735ed25f24ffe2ed31a0e904495431b9314366a042eadd1312a3c2e7d97ab4748ac0b67bdb0d0c3b0322a4b36 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.0-rc.1/kubernetes-client-windows-amd64.tar.gz) | f903c4b8397954a11c10463e1fde15c95805fc048988e66c5d59cba5155e244c12cea98e0502365705c3cae138d251a1a87c2bb29da0e1762e84d08ff225ccc4 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.27.0-rc.1/kubernetes-client-windows-arm64.tar.gz) | 4ac1bcba49774c9eba9a562ebc007bbe5963575c9e55917128171b955cc95bc2447352668fe768130b3013fbc425dfc86ed17e6767d59441e3566a456d6356c7 ### Server Binaries filename | sha512 hash -------- | ----------- [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.0-rc.1/kubernetes-server-linux-amd64.tar.gz) | 3c726e0583497813ba93953e17ec93805d18129401ad3e851e44c74d4a2ecb45c2340bf82b53e28c4e480222e7cf85fa4270506103233fbad7f38ac751b16c2d [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.0-rc.1/kubernetes-server-linux-arm64.tar.gz) | 17ef3d22f86328c2faea3df3b476f5b949e0bab2964f878e2b7b61965381ce3dfe43702aea2cfb9ed191e4e3d41ebbe780eccaa57b943d019cabb2adddafe458 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.0-rc.1/kubernetes-server-linux-ppc64le.tar.gz) | d35ba5854c853d095f7588372677f1da2a5b3ef5fde846b8cce2925e7ff4fca21b9f3c9afc8eb82a973027a35918887f2d838cf5c3ec29fb2c7af576a2493efb [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.0-rc.1/kubernetes-server-linux-s390x.tar.gz) | 79568358644e993126aaaf14a4126de5abae4ba2cbf59f264de57045189b53c1ead4ce8bdc75592164537ed6b836261feab1f2f5aca07f218ba5c2db1c80318c ### Node Binaries filename | sha512 hash -------- | ----------- [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.0-rc.1/kubernetes-node-linux-amd64.tar.gz) | dd1feed73c4d0ea7be79338500ae413dc5f55a69da0e2a27cf8e44d14f49a514b262c69b6069d1cd709564fac6e030cef581204b3d48e5208b9967a9d42b69af [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.0-rc.1/kubernetes-node-linux-arm64.tar.gz) | 23421b4f6c4b21168302fa22ddf193f9d2db6a582edfded4cbf88dfa646b1f5f65817165f49bf251cb5c32aaf51a92e425cd92051745f890bfd314a82e292573 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.0-rc.1/kubernetes-node-linux-ppc64le.tar.gz) | 9149c0bd4765484351e703d074e1554ca0735bed13bc7da3ba511ba8b6e547337f2c2c904e6db8256306b8089018ac66857a212bcb555034487111c07170c5b6 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.0-rc.1/kubernetes-node-linux-s390x.tar.gz) | 1fc2f78314fc7781f5296ebe82cce905eb841bb551919261571f549f3d39f53dd8b9546f060f31209e129ff9a4dafbb50ebfc68741525b01d82639ef8a48e12c [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.0-rc.1/kubernetes-node-windows-amd64.tar.gz) | 3ab964ee2fa017e96adf606ce8e8540bc3b79323141548247415e5865dd7447dca4146706cf21a1e36512d6f7950ed866351f9e6427e27c20b27118f81bf5d6c ### Container Images All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures ---- | ------------- [registry.k8s.io/conformance:v1.27.0-rc.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) [registry.k8s.io/kube-apiserver:v1.27.0-rc.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) [registry.k8s.io/kube-controller-manager:v1.27.0-rc.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) [registry.k8s.io/kube-proxy:v1.27.0-rc.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) [registry.k8s.io/kube-scheduler:v1.27.0-rc.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) ## Changelog since v1.27.0-rc.0 ## Changes by Kind ### Feature - Kubernetes is now built with Go 1.20.3 ([#117125](https://github.com/kubernetes/kubernetes/pull/117125), [@xmudrii](https://github.com/xmudrii)) [SIG Release and Testing] - Updated distroless iptables to use released image `registry.k8s.io/build-image/distroless-iptables:v0.2.3` ([#117126](https://github.com/kubernetes/kubernetes/pull/117126), [@xmudrii](https://github.com/xmudrii)) [SIG Testing] ### Bug or Regression - Recreate DaemonSet pods completed with Succeeded phase ([#117073](https://github.com/kubernetes/kubernetes/pull/117073), [@mimowo](https://github.com/mimowo)) [SIG Apps and Testing] - The encryption response from KMS v2 plugins is now validated earlier at DEK generation time instead of waiting until an encryption is performed. ([#116877](https://github.com/kubernetes/kubernetes/pull/116877), [@enj](https://github.com/enj)) [SIG API Machinery and Auth] ## Dependencies ### Added _Nothing has changed._ ### Changed - github.com/rogpeppe/go-internal: [v1.9.0 → v1.10.0](https://github.com/rogpeppe/go-internal/compare/v1.9.0...v1.10.0) ### Removed _Nothing has changed._ # v1.27.0-rc.0 ## Downloads for v1.27.0-rc.0 ### Source Code filename | sha512 hash -------- | ----------- [kubernetes.tar.gz](https://dl.k8s.io/v1.27.0-rc.0/kubernetes.tar.gz) | 00c1377aacf2540f9dd92538e95e4d676bb77839edf59645dec6be96d4988d64b79f0a2f4a3e604a42a8c06710a88efc86d7c4bdab8d11269aadbbeaa8f02cc0 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.27.0-rc.0/kubernetes-src.tar.gz) | e39d9fe4d1426ad35db1593c75d9bcc1e64178fac46a4c759aeb24cf37e061e1e559ab2fe8d3c4f7f66e813a11aa730aefc06649ba5ff3e8a7ac5b4db79db278 ### Client Binaries filename | sha512 hash -------- | ----------- [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.27.0-rc.0/kubernetes-client-darwin-amd64.tar.gz) | 811e5f52ee5f000bbac5ef5f45a7266da96ec56054056e793fab1b39dfcea2c872b6c464d163f9ff445e928750ac7fe04539b38751646c06ec204c10968d3114 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.27.0-rc.0/kubernetes-client-darwin-arm64.tar.gz) | 4a1b6ee903132f23d153369bdf97b40eeb7d111c67f60fa6367909c0bb6dd82ec7ad017ced4cb381bdbe272db514f5e6f12576b8f4b5f384a4cbd544a39268f5 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.27.0-rc.0/kubernetes-client-linux-386.tar.gz) | a422191ad2118c8f1debd73834d5a963d1992441c4c0a917ebfd64818f59038d1971589b9a6f8ba7252949b5bfde62dbfab60da8783502733550c5f65cfae592 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.0-rc.0/kubernetes-client-linux-amd64.tar.gz) | aca2b4ff673b381da5a979266395b4eea61c48ba59b6ae81555da21394fd535bb4b408ba584586a1336c02bf0706a90ed53ee9b6d0712519612d60b1c1f7b59c [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.27.0-rc.0/kubernetes-client-linux-arm.tar.gz) | dd5d9d5adc928e114a9a35d1f845fa98b2b236a3937f056adce5c7280f91d7024f013329e3a9a17329f1247c42a1af7c011d37bfb7830937af2cf14babec3dc4 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.0-rc.0/kubernetes-client-linux-arm64.tar.gz) | 0dff54bcb39c82a7142dea53ccec384c12637b9b3f261d67338d7ab1508a32897baa29657ffeeaf9bf8e65d11feb25b83349749671c7fb2a38e1a44491f50716 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.0-rc.0/kubernetes-client-linux-ppc64le.tar.gz) | 28538ade0567bfd90f3b3975ba4e1e5c986fc2a0e7e02d4cd5f93b22a9f7ec9a0b82226938a0512a1b95149e5d801c16b46c9181aff33cf4b02ec958fefdcd73 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.0-rc.0/kubernetes-client-linux-s390x.tar.gz) | 39cbbed42e2e53955e0b76306046f7ad06a88a800902c76584fe9a8a2741349c1cd74ebf4e9d0c7711e5721dd2b028df3f882eeee7242596c747b39f7ddadf87 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.27.0-rc.0/kubernetes-client-windows-386.tar.gz) | 6f5947e9d4760bb00393d9bd8a2a5d389306cb51e0cc46012becaaa2ebbc4f6d0b63d239a7b99f24136ec6ce50ea51032659a68daedd79b8a494acc2cc966e09 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.0-rc.0/kubernetes-client-windows-amd64.tar.gz) | 5436de39bcb3edbc7db16e8628e161a0c1fea9cf501ec6969622f3980c2b009f62137f9a60bff77b33751d3188f24812b13bafe4b397e341cd3e8979459f9972 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.27.0-rc.0/kubernetes-client-windows-arm64.tar.gz) | 9390bc41779539e2475e992969a6f41eb68f82082d75abb793a8c9b832b86f4de42be6e313fb963b9e50ec9fd3265d9b8c8b223f7999d823a74062baaf536406 ### Server Binaries filename | sha512 hash -------- | ----------- [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.0-rc.0/kubernetes-server-linux-amd64.tar.gz) | 652e12dd953d3dbd0fc11c02d145cfe608c41639077a056cc8ecef8d427a73c937d5fc341b8d7e4adb9a68bb6babc3165315106fe554a7ea961f1e1a45cf9566 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.0-rc.0/kubernetes-server-linux-arm64.tar.gz) | c2d93520b9a7e6554207477527f408698d1b086c9ee6c6c5716f42e007367b15d1d137c723768d0d8b55cc4f1b32e5f2c57bb05b589c3a2ffe71af9c5626e303 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.0-rc.0/kubernetes-server-linux-ppc64le.tar.gz) | f388a0b93b722814e51c0de14dd332fbe4517549a717237fa3d2d75503d5f0db7d73db154e07bc958e40ca4bea11f88312b5c8b334b77d9494637b0da4046b2e [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.0-rc.0/kubernetes-server-linux-s390x.tar.gz) | 2ff2af56a4306b807e6792524489c4e2e04b2e9b661b767fe9efa34ede151ff6281b30d2a86332ea9ad69ef3031d96aa072bd099a893ad8db0be81c5c1215d98 ### Node Binaries filename | sha512 hash -------- | ----------- [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.0-rc.0/kubernetes-node-linux-amd64.tar.gz) | 2fbe286b501a1d84bad54c035fd5daecff6268b21adec003ad90b5c6c813964d00853e8014ea8d4fc9e748586801f3231ad5fa405f213c1fc15b5b0e1819eda8 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.0-rc.0/kubernetes-node-linux-arm64.tar.gz) | 1531ac12836e354269f2c0df254593d37f5860352a408c931594735212aa3504fd0ba21bee6ac8df0081d9502d307f093d6932536bd6a48124884e8925a2a76d [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.0-rc.0/kubernetes-node-linux-ppc64le.tar.gz) | 87e3599e471a33e922235d6c54eb76c477100c04b185d897896b2942396b609492e851f5537de46b08973732e7518ccdbf5fe88c015b1479bf899e6e8c2549b4 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.0-rc.0/kubernetes-node-linux-s390x.tar.gz) | 4c67645f9658809cf4357f94c0a8363a9ef4535ee07e7212565443f5e35a3037d87293a57e4b5fdc7cdca4ef946f8f0e1db09f4db87d747230da21df5f59bd8b [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.0-rc.0/kubernetes-node-windows-amd64.tar.gz) | e5bd3034027e38bfe9ffd81f85a3d4c5f5a3e07542d5d718aa4e89d4d4662530316ac7c3b17bbaeb514ecdcc553117688c58899d067b89ffc0e8f3a282e119c2 ### Container Images All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures ---- | ------------- [registry.k8s.io/conformance:v1.27.0-rc.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) [registry.k8s.io/kube-apiserver:v1.27.0-rc.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) [registry.k8s.io/kube-controller-manager:v1.27.0-rc.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) [registry.k8s.io/kube-proxy:v1.27.0-rc.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) [registry.k8s.io/kube-scheduler:v1.27.0-rc.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) ## Changelog since v1.27.0-beta.0 ## Changes by Kind ### API Change - Added a new alpha API: ClusterTrustBundle (`certificates.k8s.io/v1alpha1`). A ClusterTrustBundle may be used to distribute [X.509](https://www.itu.int/rec/T-REC-X.509) trust anchors to workloads within the cluster. ([#113218](https://github.com/kubernetes/kubernetes/pull/113218), [@ahmedtd](https://github.com/ahmedtd)) [SIG API Machinery, Auth and Testing] - Remove `kubernetes.io/grpc` standard appProtocol ([#116866](https://github.com/kubernetes/kubernetes/pull/116866), [@LiorLieberman](https://github.com/LiorLieberman)) [SIG API Machinery and Apps] ### Feature - Give terminal phase correctly to all pods that will not be restarted. In particular, assign Failed phase to pods which are deleted while pending. Also, assign a terminal phase (Succeeded or Failed, depending on the exit statuses of the pod containers) to pods which are deleted while running. This fixes the issue for jobs using pod failure policy (with JobPodFailurePolicy and PodDisruptionConditions feature gates enabled) that their pods could get stuck in the pending phase when deleted. ([#115331](https://github.com/kubernetes/kubernetes/pull/115331), [@mimowo](https://github.com/mimowo)) [SIG Cloud Provider, Node and Testing] ### Bug or Regression - Fixed two regressions introduced by the PodDisruptionConditions feature (on by default in 1.26): - pod eviction API calls returned spurious precondition errors and required a second evict API call to succeed - dry-run eviction API calls persisted a DisruptionTarget condition into the pod being evicted ([#116554](https://github.com/kubernetes/kubernetes/pull/116554), [@atiratree](https://github.com/atiratree)) [SIG API Machinery and Testing] - Fixes a regression in the pod binding subresource to honor the `metadata.uid` precondition. This allows kube-scheduler to ensure it is assigns node names to the same instances of pods it made scheduling decisions for. ([#116550](https://github.com/kubernetes/kubernetes/pull/116550), [@alculquicondor](https://github.com/alculquicondor)) [SIG API Machinery and Testing] - Fixes bug in beta aggregated discovery endpoint which caused CRD discovery information to be temporarily missing when an Aggregated APIService with the same GroupVersion is deleted (and vice versa). ([#116770](https://github.com/kubernetes/kubernetes/pull/116770), [@alexzielenski](https://github.com/alexzielenski)) [SIG API Machinery and Testing] ## Dependencies ### Added _Nothing has changed._ ### Changed _Nothing has changed._ ### Removed _Nothing has changed._ # v1.27.0-beta.0 ## Downloads for v1.27.0-beta.0 ### Source Code filename | sha512 hash -------- | ----------- [kubernetes.tar.gz](https://dl.k8s.io/v1.27.0-beta.0/kubernetes.tar.gz) | a648cbc81d762e1b37f673871906ebe7f3b871f0a3c527d0dcfb5d20a9f4eff519354155d6a2cec8deabc2f0e9db8bb4b6ac2215597a11caad396e9d31461944 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.27.0-beta.0/kubernetes-src.tar.gz) | 2cb02e63a58590dc65962f42a6be484b804595adbecb1bcbfaf94186004bb3f9e0000aa8be9e1fb270de89733ea3baa0853211673e8c2f76d6be436782bba5dd ### Client Binaries filename | sha512 hash -------- | ----------- [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.27.0-beta.0/kubernetes-client-darwin-amd64.tar.gz) | 957d1abe4282ae6bba75732b83f858b5c3a61de4148c947862bbc90f0ecf290a3cd94eb267da2127bb2ff28237a50c0b913c261c014e06580a766f69e4b45d5b [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.27.0-beta.0/kubernetes-client-darwin-arm64.tar.gz) | 5827723ec6bc6f0d96cd20046bd736a3045f168cbf78a9064645f0e94653f3e751bcca6d18836aa038cb726ab991a48b1451fcc00bd0e751eb0af30d7bf002aa [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.27.0-beta.0/kubernetes-client-linux-386.tar.gz) | bdacf9b42269238e97b6301a975c4accd7363a05a63a35305d0d74916c138c70985491ac9d13a152d0b10609f265aede4a910ebed61bdf1b8a37264773dffd3b [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.0-beta.0/kubernetes-client-linux-amd64.tar.gz) | e139daa8df28d13ad8625c819ba94e6e4dd7805c89dd2a0bba6ce478a2bc7d9b52a3fccc18de08c13dca1b98c693d50d37599e8a3b34b7a1f39401098dea2df5 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.27.0-beta.0/kubernetes-client-linux-arm.tar.gz) | 3f669851c6317d67bbcae591056ee9cfda6e9bca3eeac02cc41eae35db3448e745e123ab75da8b9dbb546172b07d625bf821da3b0a1b6420d41140eb7b96b474 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.0-beta.0/kubernetes-client-linux-arm64.tar.gz) | c8f394650db292a117e1db5a76775541087ab0da9b3d43041d50f3126ef47a0dcb65ebbe61d8be9bdb67adce1c43d5f7a695ff0b9909c8c9461d6937ebe9160f [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.0-beta.0/kubernetes-client-linux-ppc64le.tar.gz) | 83b51c787f57b698584c3c585a772470819260008808a2102a9e765ef1458d9bb536aeb3e2587d391c6efb06d56326f1c8b47f12ab98069d1605ef210ecd6e8c [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.0-beta.0/kubernetes-client-linux-s390x.tar.gz) | d79766f56263a78549d7e2bc8f93977d8730435beeb7fe9413686d09ac6a6edc8a868621023623656782272e518fa7955275ab0d4aecb8a71cb4ba544dd5f77d [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.27.0-beta.0/kubernetes-client-windows-386.tar.gz) | 399741ba92a59c0c3640f4d4d0c961b63bd24ba8a5ce036f4a82dcd040a0d2873e7e3237af10da1b2982af5ca6ae8edb2a4d023db3af87dfae6c90528a487de3 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.0-beta.0/kubernetes-client-windows-amd64.tar.gz) | 3eba7adbb6c7c386d04bdddcd6d66ca7f5799789680c7fbb9216a0520884264dc5fdb35a0417d03d77955097c6341a30e3e07d077266c2ed2f96d1765f344e39 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.27.0-beta.0/kubernetes-client-windows-arm64.tar.gz) | 158475196f75764dd115e187a5fb27894367a8a2ddad755e3d542e5f225fe9bad476f592c0b7fad2a3dded4638ccec2a1f717eec4d04c8e510334a3a410e0541 ### Server Binaries filename | sha512 hash -------- | ----------- [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.0-beta.0/kubernetes-server-linux-amd64.tar.gz) | fb9caa627e77d1bd39b11106dd95c9dd008c5d418234636a0beddd48e59c980d4924ed3006133e20d2ac0715a4353d14a90f7ebc5345804f24160a13efb7a2b5 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.0-beta.0/kubernetes-server-linux-arm64.tar.gz) | 34f61cfeba8adf7fd3dd83599e34ed36d5942a41904f0430a7b8a5078d306283a4dd7eec40716c8aa6f4ff87dea1faa588fff66a2c388aac8c7b461a64366c33 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.0-beta.0/kubernetes-server-linux-ppc64le.tar.gz) | c37a226fa7b6d35b32420c13e67482820f4b23cd9dc9c23820d8f3024bf969d2acc96dd31267a964a73e3a4a61a046c778ab3443598b111eccbf20a682b93f40 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.0-beta.0/kubernetes-server-linux-s390x.tar.gz) | b2f29641f5756bb77b048cd336997e89ae50236fb32a7b425c348fab1f077534facce6c90ad9650dd2db5b708bff1ddabb478e29fc69f32b59e5ded247665840 ### Node Binaries filename | sha512 hash -------- | ----------- [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.0-beta.0/kubernetes-node-linux-amd64.tar.gz) | 305ee41682bb222e040134e75aefeda6cad1f81f4af761c514bb5d66fe83d42dd993c0a118c178a9e8abd6d2ae3fdb7b70c0509f1134f032c2ef2ef2bc103d81 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.0-beta.0/kubernetes-node-linux-arm64.tar.gz) | 6bc84fb35f278742734ac0c6265d6f2d654a7d57d65e98d597ba4c438b7ea20033e0431515f120fbcbf2fb6e99d3f50d4b4ecfc88e3705d08fc949b7f42c3776 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.0-beta.0/kubernetes-node-linux-ppc64le.tar.gz) | 638ad423ddbc52179320fe497f775d50c210745044aca9cea00c674dc1e710e979b7fca564811ccae99b801582e075194b09a00548f789740e0e6c4791309bdc [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.0-beta.0/kubernetes-node-linux-s390x.tar.gz) | 4874d3e34145c19973aa130c3f2c4eb5b01991142eb9bbf7391378bb6f83179a163659c80b3e45526cf334f7c63868502381afce18205ab92c521f4c911e3179 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.0-beta.0/kubernetes-node-windows-amd64.tar.gz) | 5d3e9e88577e5be11d56e65d76cec6ab931811f106fd1683551d9b2514ec8edf21f39c6512adc3ce901862f015b28237fc1774b0ccfaf771f106237a2ed599c6 ### Container Images All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures ---- | ------------- [registry.k8s.io/conformance:v1.27.0-beta.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) [registry.k8s.io/kube-apiserver:v1.27.0-beta.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) [registry.k8s.io/kube-controller-manager:v1.27.0-beta.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) [registry.k8s.io/kube-proxy:v1.27.0-beta.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) [registry.k8s.io/kube-scheduler:v1.27.0-beta.0](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) ## Changelog since v1.27.0-alpha.3 ## Urgent Upgrade Notes ### (No, really, you MUST read this before you upgrade) - The `IPv6DualStack` feature gate for external cloud providers was removed. (The feature became GA in 1.23 and the gate was removed for all other components several releases ago.) If you were still manually enabling it you must stop now. ([#116255](https://github.com/kubernetes/kubernetes/pull/116255), [@danwinship](https://github.com/danwinship)) [SIG API Machinery, Cloud Provider and Network] ## Changes by Kind ### Deprecation - The alpha SecurityContextDeny admission plugin is deprecated and now requires enabling the alpha `SecurityContextDeny` feature gate to use. It will be removed in a future version. ([#115879](https://github.com/kubernetes/kubernetes/pull/115879), [@mtardy](https://github.com/mtardy)) [SIG Auth] ### API Change - API: resource.k8s.io/v1alpha1.PodScheduling was renamed to resource.k8s.io/v1alpha2.PodSchedulingContext. ([#116556](https://github.com/kubernetes/kubernetes/pull/116556), [@pohly](https://github.com/pohly)) [SIG API Machinery, Apps, Auth, CLI, Node, Scheduling and Testing] - APIServerTracing feature gate is now enabled by default. Tracing in the API Server is still disabled by default, and requires a config file to enable. ([#116144](https://github.com/kubernetes/kubernetes/pull/116144), [@dashpole](https://github.com/dashpole)) [SIG API Machinery and Testing] - Added CEL runtime cost calculation into ValidatingAdmissionPolicy, matching the evaluation cost restrictions that already apply to CustomResourceDefinition. If rule evaluation uses more compute than the limit, the API server aborts the evaluation and the admission check that was being performed is aborted; the `failurePolicy` for the ValidatingAdmissionPolicy determines the outcome. ([#115747](https://github.com/kubernetes/kubernetes/pull/115747), [@cici37](https://github.com/cici37)) [SIG API Machinery] - Added `messageExpression` to `ValidatingAdmissionPolicy`, to set custom failure message via CEL expression. ([#116397](https://github.com/kubernetes/kubernetes/pull/116397), [@jiahuif](https://github.com/jiahuif)) [SIG API Machinery] - Added a new IPAddress object kind - Added a new ClusterIP allocator. The new allocator removes previous Service CIDR block size limitations for IPv4, and limits IPv6 size to a /64 ([#115075](https://github.com/kubernetes/kubernetes/pull/115075), [@aojea](https://github.com/aojea)) [SIG API Machinery, Apps, Auth, CLI, Cluster Lifecycle, Network and Testing] - Added a new alpha API: ClusterTrustBundle (`certificates.k8s.io/v1alpha1`). A ClusterTrustBundle may be used to distribute [X.509](https://www.itu.int/rec/T-REC-X.509) trust anchors to workloads within the cluster. ([#113218](https://github.com/kubernetes/kubernetes/pull/113218), [@ahmedtd](https://github.com/ahmedtd)) [SIG API Machinery, Auth and Testing] - Added authorization check support to the CEL expressions of ValidatingAdmissionPolicy via a `authorizer` variable with expressions. The new variable provides a builder that allows expressions such `authorizer.group('').resource('pods').check('create').allowed()`. ([#116054](https://github.com/kubernetes/kubernetes/pull/116054), [@jpbetz](https://github.com/jpbetz)) [SIG API Machinery and Testing] - Added matchConditions field to ValidatingAdmissionPolicy, enabled support for CEL based custom match criteria. ([#116350](https://github.com/kubernetes/kubernetes/pull/116350), [@maxsmythe](https://github.com/maxsmythe)) [SIG API Machinery and Testing] - Added messageExpression field to ValidationRule. (#115969, @DangerOnTheRanger) ([#115969](https://github.com/kubernetes/kubernetes/pull/115969), [@DangerOnTheRanger](https://github.com/DangerOnTheRanger)) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Instrumentation, Node and Testing] - Added the `MatchConditions` field to `ValidatingWebhookConfiguration` and `MutatingWebhookConfiguration` for the v1beta and v1 apis. The `AdmissionWebhookMatchConditions` featuregate is now in Alpha ([#116261](https://github.com/kubernetes/kubernetes/pull/116261), [@ivelichkovich](https://github.com/ivelichkovich)) [SIG API Machinery and Testing] - Added validation to ensure that if `service.kubernetes.io/topology-aware-hints` and `service.kubernetes.io/topology-mode` annotations are both set, they are set to the same value. - Added deprecation warning if `service.kubernetes.io/topology-aware-hints` annotation is used. ([#116612](https://github.com/kubernetes/kubernetes/pull/116612), [@robscott](https://github.com/robscott)) [SIG Apps, Network and Testing] - Adds auditAnnotations to ValidatingAdmissionPolicy, enabling CEL to be used to add audit annotations to request audit events. Adds validationActions to ValidatingAdmissionPolicyBinding, enabling validation failures to be handled by any combination of the warn, audit and deny enforcement actions. ([#115973](https://github.com/kubernetes/kubernetes/pull/115973), [@jpbetz](https://github.com/jpbetz)) [SIG API Machinery and Testing] - Adds feature gate `NodeLogQuery` which provides cluster administrators with a streaming view of logs using kubectl without them having to implement a client side reader or logging into the node. ([#96120](https://github.com/kubernetes/kubernetes/pull/96120), [@LorbusChris](https://github.com/LorbusChris)) [SIG API Machinery, Apps, CLI, Node, Testing and Windows] - Api: validation of a PodSpec now rejects invalid ResourceClaim and ResourceClaimTemplate names. For a pod, the name generated for the ResourceClaim when using a template also must be valid. ([#116576](https://github.com/kubernetes/kubernetes/pull/116576), [@pohly](https://github.com/pohly)) [SIG Apps] - Bump default API QPS limits for Kubelet. ([#116121](https://github.com/kubernetes/kubernetes/pull/116121), [@wojtek-t](https://github.com/wojtek-t)) [SIG API Machinery and Node] - Enable the "StatefulSetStartOrdinal" feature gate in beta ([#115260](https://github.com/kubernetes/kubernetes/pull/115260), [@pwschuurman](https://github.com/pwschuurman)) [SIG API Machinery and Apps] - Extended the kubelet's PodResources API to include resources allocated in `ResourceClaims` via `DynamicResourceAllocation`. Additionally, added a new `Get()` method to query a specific pod for its resources. ([#115847](https://github.com/kubernetes/kubernetes/pull/115847), [@moshe010](https://github.com/moshe010)) [SIG Node] - Forbid to set matchLabelKeys when labelSelector isn’t set in topologySpreadConstraints ([#116535](https://github.com/kubernetes/kubernetes/pull/116535), [@denkensk](https://github.com/denkensk)) [SIG API Machinery, Apps and Scheduling] - GCE does not support LoadBalancer Services with ports with different protocols (TCP and UDP) ([#115966](https://github.com/kubernetes/kubernetes/pull/115966), [@aojea](https://github.com/aojea)) [SIG Apps and Cloud Provider] - GRPC probes are now a GA feature. GRPCContainerProbe feature gate was locked to default value and will be removed in v1.29. If you were setting this feature gate explicitly, please remove it now. ([#116233](https://github.com/kubernetes/kubernetes/pull/116233), [@SergeyKanzhelev](https://github.com/SergeyKanzhelev)) [SIG API Machinery, Apps and Node] - Graduate Kubelet Topology Manager to GA. ([#116093](https://github.com/kubernetes/kubernetes/pull/116093), [@swatisehgal](https://github.com/swatisehgal)) [SIG API Machinery, Node and Testing] - Graduate `KubeletTracing` to beta, which means that the feature gate is now enabled by default. ([#115750](https://github.com/kubernetes/kubernetes/pull/115750), [@saschagrunert](https://github.com/saschagrunert)) [SIG Instrumentation and Node] - Graduate the container resource metrics feature on HPA to beta. ([#116046](https://github.com/kubernetes/kubernetes/pull/116046), [@sanposhiho](https://github.com/sanposhiho)) [SIG Autoscaling] - Introduced a breaking change to the `resource.k8s.io` API in its `AllocationResult` struct. This change allows a kubelet plugin for the `DynamicResourceAllocation` feature to service allocations from multiple resource driver controllers. ([#116332](https://github.com/kubernetes/kubernetes/pull/116332), [@klueska](https://github.com/klueska)) [SIG API Machinery, Apps, CLI, Node, Scheduling and Testing] - Introduces new alpha functionality to the reflector, allowing user to enable API streaming. To activate this feature, users can set the `ENABLE_CLIENT_GO_WATCH_LIST_ALPHA` environmental variable. It is important to note that the server must support streaming for this feature to function properly. If streaming is not supported by the server, the reflector will revert to the previous method of obtaining data through LIST/WATCH semantics. ([#110772](https://github.com/kubernetes/kubernetes/pull/110772), [@p0lyn0mial](https://github.com/p0lyn0mial)) [SIG API Machinery] - Kubelet: change MemoryThrottlingFactor default value to 0.9 and formulas to calculate memory.high ([#115371](https://github.com/kubernetes/kubernetes/pull/115371), [@pacoxu](https://github.com/pacoxu)) [SIG API Machinery, Apps and Node] - Migrated the DaemonSet controller (within `kube-controller-manager) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging) ([#113622](https://github.com/kubernetes/kubernetes/pull/113622), [@249043822](https://github.com/249043822)) [SIG API Machinery, Apps, Instrumentation and Testing] - New `service.kubernetes.io/topology-mode` annotation has been introduced as a replacement for the `service.kubernetes.io/topology-aware-hints` annotation. - `service.kubernetes.io/topology-aware-hints` annotation has been deprecated. - kube-proxy now accepts any value that is not "disabled" for these annotations, enabling custom implementation-specific and/or future built-in heuristics to be used. ([#116522](https://github.com/kubernetes/kubernetes/pull/116522), [@robscott](https://github.com/robscott)) [SIG Apps, Network and Testing] - NodeResourceFit and NodeResourcesBalancedAllocation implement the PreScore extension point for a more performant calculation. ([#115655](https://github.com/kubernetes/kubernetes/pull/115655), [@tangwz](https://github.com/tangwz)) [SIG Scheduling] - Pods owned by a Job will now use the labels `batch.kubernetes.io/job-name` and `batch.kubernetes.io/controller-uid`. The legacy labels `job-name` and `controller-uid` are still added for compatibility. ([#114930](https://github.com/kubernetes/kubernetes/pull/114930), [@kannon92](https://github.com/kannon92)) [SIG Apps] - Promote CronJobTimeZone feature to GA ([#115904](https://github.com/kubernetes/kubernetes/pull/115904), [@soltysh](https://github.com/soltysh)) [SIG API Machinery and Apps] - Promoted `SelfSubjectReview` to Beta ([#116274](https://github.com/kubernetes/kubernetes/pull/116274), [@nabokihms](https://github.com/nabokihms)) [SIG API Machinery, Auth, CLI and Testing] - Relax API validation to allow pod node selector to be mutable for gated pods (additions only, no deletions or mutations). ([#116161](https://github.com/kubernetes/kubernetes/pull/116161), [@danielvegamyhre](https://github.com/danielvegamyhre)) [SIG Apps, Scheduling and Testing] - Remove deprecated `--enable-taint-manager` and `--pod-eviction-timeout` CLI flags ([#115840](https://github.com/kubernetes/kubernetes/pull/115840), [@atosatto](https://github.com/atosatto)) [SIG API Machinery, Apps, Node and Testing] - Resource.k8s.io/v1alpha1 was replaced with resource.k8s.io/v1alpha2. Before upgrading a cluster, all objects in resource.k8s.io/v1alpha1 (ResourceClaim, ResourceClaimTemplate, ResourceClass, PodScheduling) must be deleted. The changes will be internal, so YAML files which create pods and resource claims don't need changes except for the newer `apiVersion`. ([#116299](https://github.com/kubernetes/kubernetes/pull/116299), [@pohly](https://github.com/pohly)) [SIG API Machinery, Apps, CLI, Node, Scheduling and Testing] - SELinuxMountReadWriteOncePod graduated to Beta. ([#116425](https://github.com/kubernetes/kubernetes/pull/116425), [@jsafrane](https://github.com/jsafrane)) [SIG Storage and Testing] - StatefulSetAutoDeletePVC feature gate promoted to beta. ([#116501](https://github.com/kubernetes/kubernetes/pull/116501), [@mattcary](https://github.com/mattcary)) [SIG Apps, Auth and Testing] - The API server now re-uses data encryption keys while the kms v2 plugin's key ID is stable. Data encryption keys are still randomly generated on server start but an atomic counter is used to prevent nonce collisions. ([#116155](https://github.com/kubernetes/kubernetes/pull/116155), [@enj](https://github.com/enj)) [SIG API Machinery, Auth and Testing] - The API server's encryption at rest configuration now allows the use of wildcards in the list of resources. For example, '*.*' can be used to encrypt all resources, including all current and future custom resources. ([#115149](https://github.com/kubernetes/kubernetes/pull/115149), [@nilekhc](https://github.com/nilekhc)) [SIG API Machinery, Auth and Testing] - Update KMSv2 to beta ([#115123](https://github.com/kubernetes/kubernetes/pull/115123), [@aramase](https://github.com/aramase)) [SIG API Machinery, Auth and Testing] - Updated: Redefine AppProtocol field description and add new standard values ([#115433](https://github.com/kubernetes/kubernetes/pull/115433), [@LiorLieberman](https://github.com/LiorLieberman)) [SIG API Machinery, Apps and Network] - ValidatingAdmissionPolicy now provides a status field that contains results of type checking the validation expression. The type checking is fully informational, and the behavior of the policy is unchanged. ([#115668](https://github.com/kubernetes/kubernetes/pull/115668), [@jiahuif](https://github.com/jiahuif)) [SIG API Machinery, Auth, Cloud Provider and Testing] - We have removed support for the v1alpha1 kubeletplugin API of DynamicResourceManagement. All plugins must update to v1alpha2 in order to function properly going forward. ([#116558](https://github.com/kubernetes/kubernetes/pull/116558), [@klueska](https://github.com/klueska)) [SIG API Machinery, Apps, CLI, Node, Scheduling and Testing] ### Feature - #### Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.: ([#113428](https://github.com/kubernetes/kubernetes/pull/113428), [@mengjiao-liu](https://github.com/mengjiao-liu)) [SIG API Machinery, Apps, Instrumentation and Testing] - Add e2e test to node expand volume with secret ([#115451](https://github.com/kubernetes/kubernetes/pull/115451), [@zhucan](https://github.com/zhucan)) [SIG Storage and Testing] - Added NewVolumeManagerReconstruction feature gate and enable it by default to enable updated discovery of mounted volumes during kubelet startup. Please watch for kubelet getting stuck at startup and / or not unmounting volumes from deleted Pods and report any issues in this area. ([#115268](https://github.com/kubernetes/kubernetes/pull/115268), [@jsafrane](https://github.com/jsafrane)) [SIG Node and Storage] - Added metrics for volume reconstruction during kubelet startup. ([#115965](https://github.com/kubernetes/kubernetes/pull/115965), [@jsafrane](https://github.com/jsafrane)) [SIG Node and Storage] - Added the ability to host webhooks in the cloud controller manager. ([#108838](https://github.com/kubernetes/kubernetes/pull/108838), [@nckturner](https://github.com/nckturner)) [SIG API Machinery, Cloud Provider and Testing] - Adding e2e tests for kubectl --subresource for beta graduation ([#116590](https://github.com/kubernetes/kubernetes/pull/116590), [@MadhavJivrajani](https://github.com/MadhavJivrajani)) [SIG CLI and Testing] - Adds --output plaintext-openapiv2 argument to kubectl explain to use old openapiv2 `explain` implementation. ([#115480](https://github.com/kubernetes/kubernetes/pull/115480), [@alexzielenski](https://github.com/alexzielenski)) [SIG Architecture, Auth, CLI, Cloud Provider and Node] - By enabling the `UserNamespacesStatelessPodsSupport` feature gate in kubelet, you can now run a stateless pod in a separate user namespace ([#116377](https://github.com/kubernetes/kubernetes/pull/116377), [@giuseppe](https://github.com/giuseppe)) [SIG Apps, Node and Storage] - By enabling the alpha `CloudNodeIPs` feature gate in kubelet and the cloud provider, you can now specify a dual-stack `--node-ip` value (when using an external cloud provider that supports that functionality). ([#116305](https://github.com/kubernetes/kubernetes/pull/116305), [@danwinship](https://github.com/danwinship)) [SIG API Machinery, Cloud Provider, Network and Node] - Change kubectl --subresource flag to beta ([#116595](https://github.com/kubernetes/kubernetes/pull/116595), [@MadhavJivrajani](https://github.com/MadhavJivrajani)) [SIG CLI] - Changed metrics for aggregated discovery to publish new time series (alpha). ([#115630](https://github.com/kubernetes/kubernetes/pull/115630), [@Jefftree](https://github.com/Jefftree)) [SIG API Machinery and Testing] - Graduate CRI Events driven Pod LifeCycle Event Generator (Evented PLEG) to Beta ([#115967](https://github.com/kubernetes/kubernetes/pull/115967), [@harche](https://github.com/harche)) [SIG Node] - Graduated `matchLabelKeys` in `podTopologySpread` to Beta ([#116291](https://github.com/kubernetes/kubernetes/pull/116291), [@denkensk](https://github.com/denkensk)) [SIG Scheduling] - Graduates the CSINodeExpandSecret feature to Beta. This feature facilitates passing secrets to CSI driver as part of Node Expansion CSI operation. ([#115621](https://github.com/kubernetes/kubernetes/pull/115621), [@humblec](https://github.com/humblec)) [SIG Storage] - HPA controller exposes the following metrics from the kube-controller-manager. - `metric_computation_duration_seconds`: Number of metric computations. - `metric_computation_total`: The time(seconds) that the HPA controller takes to calculate one metric. ([#116326](https://github.com/kubernetes/kubernetes/pull/116326), [@sanposhiho](https://github.com/sanposhiho)) [SIG Apps, Autoscaling and Instrumentation] - HPA controller starts to expose metrics from the kube-controller-manager. - `reconciliations_total`: Number of reconciliation of HPA controller. - `reconciliation_duration_seconds`: The time(seconds) that the HPA controller takes to reconcile once. ([#116010](https://github.com/kubernetes/kubernetes/pull/116010), [@sanposhiho](https://github.com/sanposhiho)) [SIG Apps, Autoscaling and Instrumentation] - Kube-scheduler: Optimized implementation of null labelSelector in topology spreading. ([#116607](https://github.com/kubernetes/kubernetes/pull/116607), [@alculquicondor](https://github.com/alculquicondor)) [SIG Scheduling] - Kubelet allows pods to use the `net.ipv4.ip_local_reserved_ports` sysctl by default and the minimal kernel version is 3.16; Pod Security admission allows this sysctl in v1.27+ versions of the baseline and restricted policies. ([#115374](https://github.com/kubernetes/kubernetes/pull/115374), [@pacoxu](https://github.com/pacoxu)) [SIG Auth, Network and Node] - Kubernetes is now built with go 1.20.2 ([#116404](https://github.com/kubernetes/kubernetes/pull/116404), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing] - Locks CSIMigrationvSphere feature gate. ([#116610](https://github.com/kubernetes/kubernetes/pull/116610), [@xing-yang](https://github.com/xing-yang)) [SIG Storage] - Make `apiextensions-apiserver` binary linking static (also affects the deb and rpm packages). ([#114226](https://github.com/kubernetes/kubernetes/pull/114226), [@saschagrunert](https://github.com/saschagrunert)) [SIG API Machinery and Release] - Make `kube-aggregator` binary linking static (also affects the deb and rpm packages). ([#114227](https://github.com/kubernetes/kubernetes/pull/114227), [@saschagrunert](https://github.com/saschagrunert)) [SIG API Machinery and Release] - Migrated controller helper functions to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#115049](https://github.com/kubernetes/kubernetes/pull/115049), [@fatsheep9146](https://github.com/fatsheep9146)) [SIG Apps] - Migrated the ClusterRole aggregation controller (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#113910](https://github.com/kubernetes/kubernetes/pull/113910), [@mengjiao-liu](https://github.com/mengjiao-liu)) [SIG API Machinery, Apps and Instrumentation] - Migrated the Deployment controller (within `kube-controller-manager) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging) ([#113525](https://github.com/kubernetes/kubernetes/pull/113525), [@249043822](https://github.com/249043822)) [SIG API Machinery, Apps, Instrumentation and Testing] - Migrated the StatefulSet controller (within `kube-controller-manager) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging) ([#113840](https://github.com/kubernetes/kubernetes/pull/113840), [@249043822](https://github.com/249043822)) [SIG API Machinery, Apps, Instrumentation and Testing] - Migrated the bootstrap signer controller and the token cleaner controller (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#113464](https://github.com/kubernetes/kubernetes/pull/113464), [@mengjiao-liu](https://github.com/mengjiao-liu)) [SIG API Machinery, Apps and Instrumentation] - Migrated the defaultbinder scheduler plugin to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#116571](https://github.com/kubernetes/kubernetes/pull/116571), [@mengjiao-liu](https://github.com/mengjiao-liu)) [SIG Instrumentation and Scheduling] - Migrated the main kube-controller-manager binary to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#116529](https://github.com/kubernetes/kubernetes/pull/116529), [@pohly](https://github.com/pohly)) [SIG API Machinery, Apps, Auth and Node] - Migrated the replicaset controller (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#114871](https://github.com/kubernetes/kubernetes/pull/114871), [@Namanl2001](https://github.com/Namanl2001)) [SIG API Machinery, Apps, Instrumentation and Testing] - Migrated the service-account controller (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#114918](https://github.com/kubernetes/kubernetes/pull/114918), [@Namanl2001](https://github.com/Namanl2001)) [SIG API Machinery, Apps, Auth, Instrumentation and Testing] - Migrated the volume attach/detach controller (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). Migrated the PersistentVolumeClaim protection controller (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). Migrated the PersistentVolume protection controller (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#113584](https://github.com/kubernetes/kubernetes/pull/113584), [@yangjunmyfm192085](https://github.com/yangjunmyfm192085)) [SIG API Machinery, Apps, Instrumentation, Node, Scheduling, Storage and Testing] - Migrated the “TTL after finished” controller (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#113916](https://github.com/kubernetes/kubernetes/pull/113916), [@songxiao-wang87](https://github.com/songxiao-wang87)) [SIG API Machinery, Apps, Instrumentation and Testing] - New "plugin_evaluation_total" is added to the scheduler. This metric counts how many times the specific plugin affects the scheduling result. The metric doesn't get incremented when the plugin has nothing to do with an incoming Pod. ([#115082](https://github.com/kubernetes/kubernetes/pull/115082), [@sanposhiho](https://github.com/sanposhiho)) [SIG Instrumentation and Scheduling] - Promote `whoami` kubectl command. ([#116510](https://github.com/kubernetes/kubernetes/pull/116510), [@nabokihms](https://github.com/nabokihms)) [SIG Auth and CLI] - Promote aggregated discovery endpoint to beta and it will be enabled by default ([#116108](https://github.com/kubernetes/kubernetes/pull/116108), [@Jefftree](https://github.com/Jefftree)) [SIG API Machinery] - Promoted `OpenAPIV3` to GA ([#116235](https://github.com/kubernetes/kubernetes/pull/116235), [@Jefftree](https://github.com/Jefftree)) [SIG API Machinery] - StorageVersionGC (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#113986](https://github.com/kubernetes/kubernetes/pull/113986), [@songxiao-wang87](https://github.com/songxiao-wang87)) [SIG API Machinery, Apps and Testing] - Switched kubectl explain to use OpenAPIV3 information published by the server. OpenAPIV2 backend can still be used with the `--output plaintext-openapiv2` argument ([#116390](https://github.com/kubernetes/kubernetes/pull/116390), [@alexzielenski](https://github.com/alexzielenski)) [SIG API Machinery, CLI and Testing] - The job controller back-off logic is now decoupled from workqueue. In case of parallelism > 1, if there are multiple new failures in a reconciliation cycle, all the failures are taken into account to compute the back-off. Previously, the back-off kicked in for all types of failures; with this change, only pod failures are taken into account. If the back-off limits exceeds, the job is marked as failed immediately; before this change, the job is marked as failed in the next back-off. ([#114768](https://github.com/kubernetes/kubernetes/pull/114768), [@sathyanarays](https://github.com/sathyanarays)) [SIG Apps and Testing] - The scheduler's metric "plugin_execution_duration_seconds" now records PreEnqueue plugins execution seconds. ([#116201](https://github.com/kubernetes/kubernetes/pull/116201), [@sanposhiho](https://github.com/sanposhiho)) [SIG Scheduling] - Unlocked the `CSIMigrationvSphere` feature gate. The change allow users to continue using the in-tree vSphere driver,pending a vSphere CSI driver release that has with GA support for Windows, XFS, and raw block access. ([#116342](https://github.com/kubernetes/kubernetes/pull/116342), [@msau42](https://github.com/msau42)) [SIG Storage] - Update kube-apiserver SLO/SLI latency metrics to exclude priority & fairness queue wait times ([#116420](https://github.com/kubernetes/kubernetes/pull/116420), [@andrewsykim](https://github.com/andrewsykim)) [SIG API Machinery] - Updated distroless iptables to use released image `registry.k8s.io/build-image/distroless-iptables:v0.2.2` - Updated setcap to use released image `registry.k8s.io/build-image/setcap:bullseye-v1.4.2` ([#116509](https://github.com/kubernetes/kubernetes/pull/116509), [@cpanato](https://github.com/cpanato)) [SIG Testing] - Upgrades functionality of `kubectl kustomize` as described at https://github.com/kubernetes-sigs/kustomize/releases/tag/kustomize%2Fv5.0.0 and https://github.com/kubernetes-sigs/kustomize/releases/tag/kustomize%2Fv5.0.1. This is a new major release of kustomize, so there are a few backwards-incompatible changes, most of which are rare use cases, bug fixes with side effects, or things that have been deprecated for multiple releases already: - https://github.com/kubernetes-sigs/kustomize/pull/4911: Drop support for a very old, legacy style of patches. patches used to be allowed to be used as an alias for patchesStrategicMerge in kustomize v3. You now have to use patchesStrategicMerge explicitly, or update to the new syntax supported by patches. See examples in the PR description of https://github.com/kubernetes-sigs/kustomize/pull/4911. - https://github.com/kubernetes-sigs/kustomize/issues/4731: Remove a potential build-time side-effect in ConfigMapGenerator and SecretGenerator, which loaded values from the local environment under some circumstances, breaking kustomize build's side-effect-free promise. While this behavior was never intended, we deprecated it and are announcing it as a breaking change since it existed for a long time. See also the Eschewed Features documentation. - https://github.com/kubernetes-sigs/kustomize/pull/4985: If you previously included .git in an AWS or Azure URL, we will no longer automatically remove that suffix. You may need to add an extra / to replace the .git for the URL to properly resolve. - https://github.com/kubernetes-sigs/kustomize/pull/4954: Drop support for using gh: as a host (e.g. gh:kubernetes-sigs/kustomize). We were unable to find any usage of or basis for this and believe it may have been targeting a custom gitconfig shorthand syntax. ([#116598](https://github.com/kubernetes/kubernetes/pull/116598), [@natasha41575](https://github.com/natasha41575)) [SIG CLI] - When an unsupported PodDisruptionBudget configuration is found, an event and log will be emitted to inform users of the misconfiguration. ([#115861](https://github.com/kubernetes/kubernetes/pull/115861), [@JayKayy](https://github.com/JayKayy)) [SIG Apps] - [alpha: kubectl apply --prune --applyset] Enables certain custom resources (CRs) to be used as ApplySet parent objects. To enable this for a given CR, apply the label `applyset.k8s.io/is-parent-type: true` to the CustomResourceDefinition (CRD) that defines it . ([#116353](https://github.com/kubernetes/kubernetes/pull/116353), [@KnVerey](https://github.com/KnVerey)) [SIG CLI] ### Documentation - The change affects the following CLI command: kubectl create rolebinding -h ([#107124](https://github.com/kubernetes/kubernetes/pull/107124), [@ptux](https://github.com/ptux)) [SIG CLI] ### Failing Test - Setting the Kubelet config option ``--resolv-conf=Host`` on Windows will now result in Kubelet applying the Pod DNS Policies as intended. ([#110566](https://github.com/kubernetes/kubernetes/pull/110566), [@claudiubelu](https://github.com/claudiubelu)) [SIG Network, Node, Testing and Windows] ### Bug or Regression - Expands the partial fix for https://github.com/kubernetes/kubernetes/issues/111539 which was already started in https://github.com/kubernetes/kubernetes/pull/109706 Specifically, we will now reduce the amount of syncs for ETP=local services even further in the CCM and avoid re-configuring LBs to an even greater extent. ([#111658](https://github.com/kubernetes/kubernetes/pull/111658), [@alexanderConstantinescu](https://github.com/alexanderConstantinescu)) [SIG Cloud Provider and Network] - Fix the problem Pod terminating stuck because of trying to umount not actual mounted dir. ([#115769](https://github.com/kubernetes/kubernetes/pull/115769), [@mochizuki875](https://github.com/mochizuki875)) [SIG Node and Storage] - Fixed a rare race condition in kube-apiserver that could lead to missing events when a watch API request was created at the same time kube-apiserver was re-initializing its internal watch. ([#116172](https://github.com/kubernetes/kubernetes/pull/116172), [@wojtek-t](https://github.com/wojtek-t)) [SIG API Machinery] - Fixed data race in `kube-scheduler` when preemption races with a Pod update. ([#116395](https://github.com/kubernetes/kubernetes/pull/116395), [@alculquicondor](https://github.com/alculquicondor)) [SIG Scheduling] - Fixed incorrect watch events when a watch is initialized simultanously with a reinitializing watchcache. ([#116436](https://github.com/kubernetes/kubernetes/pull/116436), [@wojtek-t](https://github.com/wojtek-t)) [SIG API Machinery] - Fixed performance regression in scheduler caused by frequent metric lookup on critical code path. ([#116428](https://github.com/kubernetes/kubernetes/pull/116428), [@mborsz](https://github.com/mborsz)) [SIG Scheduling] - Fixes #115825. Kube-proxy will now include the `healthz` state in its response to the LB HC as to avoid indicating to the LB that it should use the node in question when Kube-proxy is not healthy. ([#111661](https://github.com/kubernetes/kubernetes/pull/111661), [@alexanderConstantinescu](https://github.com/alexanderConstantinescu)) [SIG Network] - Force deleted pods may fail to terminate until the kubelet is restarted when the container runtime returns an error during termination. We have strengthened testing for runtime failures and now perform a more rigorous reconciliation to ensure static pods (especially those that use fixed UIDs) are restarted. As a side effect of these changes static pods will be restarted with lower latency than before (2s vs 4s, on average) and rapid updates to pod configuration should take effect sooner. A new metric `kubelet_known_pods` has been added at ALPHA stability to report the number of pods a Kubelet is tracking in a number of internal states. Operators may use the metrics to track an excess of pods in the orphaned state that may not be completing. ([#113145](https://github.com/kubernetes/kubernetes/pull/113145), [@smarterclayton](https://github.com/smarterclayton)) [SIG API Machinery, Auth, Cloud Provider, Node and Testing] - From now on, the HPA controller will return an error for the container resource metrics when the feature gate "HPAContainerMetrics" is disabled. As a result, HPA with a container resource metric performs no scale-down and performs only scale-up based on other metrics. ([#116043](https://github.com/kubernetes/kubernetes/pull/116043), [@sanposhiho](https://github.com/sanposhiho)) [SIG API Machinery, Apps and Autoscaling] - Ingress with ingressClass annotation and IngressClassName both set can be created now. ([#115447](https://github.com/kubernetes/kubernetes/pull/115447), [@kidddddddddddddddddddddd](https://github.com/kidddddddddddddddddddddd)) [SIG Network] - Kube-controller-manager: fix a bug that the "kubeconfig" field of "kubecontrollermanager.config.k8s.io" configuration is not populated correctly ([#116219](https://github.com/kubernetes/kubernetes/pull/116219), [@SataQiu](https://github.com/SataQiu)) [SIG API Machinery and Cloud Provider] - Kubelet: fix recording issue when pulling image did finish ([#114904](https://github.com/kubernetes/kubernetes/pull/114904), [@TommyStarK](https://github.com/TommyStarK)) [SIG Node] - PVCs will automatically be recreated if they are missing for a pending Pod. ([#113270](https://github.com/kubernetes/kubernetes/pull/113270), [@rrangith](https://github.com/rrangith)) [SIG Apps and Testing] - PersistentVolume API objects which set NodeAffinities using beta Kubernetes labels for OS, architecture, zone, region, and instance type may now be modified to use the stable Kubernetes labels. ([#115391](https://github.com/kubernetes/kubernetes/pull/115391), [@haoruan](https://github.com/haoruan)) [SIG Apps and Storage] - Potentially breaking change - Updating the polling interval for Windows stats collection from 1 second to 10 seconds ([#116546](https://github.com/kubernetes/kubernetes/pull/116546), [@marosset](https://github.com/marosset)) [SIG Node and Windows] - Update the Event series starting count when emitting isomorphic events from 1 to 2. ([#112334](https://github.com/kubernetes/kubernetes/pull/112334), [@dgrisonnet](https://github.com/dgrisonnet)) [SIG API Machinery and Testing] - When GCing pods, kube-controller-manager will delete Evicted pods first. ([#116167](https://github.com/kubernetes/kubernetes/pull/116167), [@borgerli](https://github.com/borgerli)) [SIG Apps] - Windows CPU usage node stats are now correctly calculated for nodes with multiple Processor Groups. ([#110864](https://github.com/kubernetes/kubernetes/pull/110864), [@claudiubelu](https://github.com/claudiubelu)) [SIG Node, Testing and Windows] - In-Place Update of Pod Resources (alpha feature) - resizing of pod may race with other pod updates. ([#116826](https://github.com/kubernetes/kubernetes/issues/116826) [SIG Node]) ### Other (Cleanup or Flake) - Added basic Denial Of Service prevention for the the node-local kubelet `podresource` API ([#116459](https://github.com/kubernetes/kubernetes/pull/116459), [@ffromani](https://github.com/ffromani)) [SIG Node and Testing] - Introduce new metrics removing the redundant subsystem in kube-apiserver pod logs metrics and deprecate the original ones: - kube_apiserver_pod_logs_pods_logs_backend_tls_failure_total becomes kube_apiserver_pod_logs_backend_tls_failure_total - kube_apiserver_pod_logs_pods_logs_insecure_backend_total becomes kube_apiserver_pod_logs_insecure_backend_total ([#114497](https://github.com/kubernetes/kubernetes/pull/114497), [@dgrisonnet](https://github.com/dgrisonnet)) [SIG API Machinery] - Kubelet: remove deprecated flag `--container-runtime` ([#114017](https://github.com/kubernetes/kubernetes/pull/114017), [@calvin0327](https://github.com/calvin0327)) [SIG Cloud Provider and Node] - Kubelet: the deprecated `--master-service-namespace` flag is removed in v1.27 ([#116015](https://github.com/kubernetes/kubernetes/pull/116015), [@SataQiu](https://github.com/SataQiu)) [SIG Node] - Linux/arm will not ship in Kubernetes 1.27 as we are running into issues with building artifacts using golang 1.20.2 (please see issue #116492) ([#115742](https://github.com/kubernetes/kubernetes/pull/115742), [@dims](https://github.com/dims)) [SIG Architecture, Release and Testing] - Migrate `pkg/controller/nodeipam/ipam/cloud_cidr_allocator.go, pkg/controller/nodeipam/ipam/multi_cidr_range_allocator.go pkg/controller/nodeipam/ipam/range_allocator.go pkg/controller/nodelifecycle/node_lifecycle_controller.go` to structured logging ([#112670](https://github.com/kubernetes/kubernetes/pull/112670), [@yangjunmyfm192085](https://github.com/yangjunmyfm192085)) [SIG API Machinery, Apps, Architecture, Cloud Provider, Instrumentation, Network and Testing] - Migrated the Kubernetes object garbage collector (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#113471](https://github.com/kubernetes/kubernetes/pull/113471), [@ncdc](https://github.com/ncdc)) [SIG API Machinery, Apps and Testing] - Migrated the ttlafterfinished controller (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#115332](https://github.com/kubernetes/kubernetes/pull/115332), [@obaranov1](https://github.com/obaranov1)) [SIG Apps] - Migrated the “sample-controller” controller to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#113879](https://github.com/kubernetes/kubernetes/pull/113879), [@pchan](https://github.com/pchan)) [SIG API Machinery and Instrumentation] - Remove Azure disk in-tree storage plugin ([#116301](https://github.com/kubernetes/kubernetes/pull/116301), [@andyzhangx](https://github.com/andyzhangx)) [SIG API Machinery, Cloud Provider, Node, Scheduling, Storage and Testing] - Remove the following deprecated metrics: - node_collector_evictions_number replaced by node_collector_evictions_total - scheduler_e2e_scheduling_duration_seconds replaced by scheduler_scheduling_attempt_duration_seconds ([#115209](https://github.com/kubernetes/kubernetes/pull/115209), [@dgrisonnet](https://github.com/dgrisonnet)) [SIG Apps and Scheduling] - Removed AWS kubelet credential provider. Please use the external kubelet credential provider binary named `ecr-credential-provider` instead. ([#116329](https://github.com/kubernetes/kubernetes/pull/116329), [@dims](https://github.com/dims)) [SIG Node, Storage and Testing] - Storage.k8s.io/v1beta1 API version of CSIStorageCapacity will no longer be served ([#116523](https://github.com/kubernetes/kubernetes/pull/116523), [@pacoxu](https://github.com/pacoxu)) [SIG API Machinery] - The `wait.Poll*` and `wait.ExponentialBackoff*` functions have been deprecated and will be removed in a future release. Callers should switch to using `wait.PollUntilContextCancel`, `wait.PollUntilContextTimeout`, or `wait.ExponentialBackoffWithContext` as appropriate. `PollWithContext(Cancel|Deadline)` will no longer return `ErrWaitTimeout` - use the `Interrupted(error) bool` helper to replace checks for `err == ErrWaitTimeout`, or compare specifically to context errors as needed. A future release will make the `ErrWaitTimeout` error private and callers must use `Interrupted()` instead. If you are returning `ErrWaitTimeout` from your own methods, switch to creating a location specific `cause err` and pass it to the new method `wait.ErrorInterrupted(cause) error` which will ensure `Interrupted()` returns true for your loop. The `wait.NewExponentialBackoffManager` and `wait.NewJitteringBackoffManager` functions have been marked as deprecated. Callers should switch to using the `Backoff{...}.DelayWithReset(clock, resetInterval)` method and must set the `Steps` field when using `Factor`. As a short term change, callers may use the `Timer()` method on the `BackoffManager` until the backoff managers are deprecated and removed. Please see the godoc of the deprecated functions for examples of how to replace usage of this function. ([#107826](https://github.com/kubernetes/kubernetes/pull/107826), [@smarterclayton](https://github.com/smarterclayton)) [SIG API Machinery, Auth, Cloud Provider, Storage and Testing] - Upgrade coredns to v1.10.1 ([#115603](https://github.com/kubernetes/kubernetes/pull/115603), [@pacoxu](https://github.com/pacoxu)) [SIG Cloud Provider and Cluster Lifecycle] - [KCCM - service controller]: enable connection draining for terminating pods upon node downscale by the cluster autoscaler. This is done by not reacting to the taint used by the cluster autoscaler to indicate that the node is going away soon, thus keeping the node referenced by the load balancer until the VM has been completely deleted. ([#115204](https://github.com/kubernetes/kubernetes/pull/115204), [@alexanderConstantinescu](https://github.com/alexanderConstantinescu)) [SIG API Machinery, Cloud Provider, Instrumentation and Network] ## Dependencies ### Added - sigs.k8s.io/kustomize/kustomize/v5: v5.0.1 ### Changed - github.com/aws/aws-sdk-go: [v1.44.147 → v1.35.24](https://github.com/aws/aws-sdk-go/compare/v1.44.147...v1.35.24) - github.com/coreos/go-systemd/v22: [v22.3.2 → v22.4.0](https://github.com/coreos/go-systemd/v22/compare/v22.3.2...v22.4.0) - github.com/go-errors/errors: [v1.0.1 → v1.4.2](https://github.com/go-errors/errors/compare/v1.0.1...v1.4.2) - github.com/golang/protobuf: [v1.5.2 → v1.5.3](https://github.com/golang/protobuf/compare/v1.5.2...v1.5.3) - github.com/onsi/ginkgo/v2: [v2.7.0 → v2.9.1](https://github.com/onsi/ginkgo/v2/compare/v2.7.0...v2.9.1) - github.com/onsi/gomega: [v1.26.0 → v1.27.4](https://github.com/onsi/gomega/compare/v1.26.0...v1.27.4) - golang.org/x/mod: v0.7.0 → v0.9.0 - golang.org/x/net: v0.7.0 → v0.8.0 - golang.org/x/sys: v0.5.0 → v0.6.0 - golang.org/x/term: v0.5.0 → v0.6.0 - golang.org/x/text: v0.7.0 → v0.8.0 - golang.org/x/tools: v0.4.0 → v0.7.0 - k8s.io/kube-openapi: 1cb3ae2 → 15aac26 - sigs.k8s.io/json: f223a00 → bc3834c - sigs.k8s.io/kustomize/api: v0.12.1 → v0.13.2 - sigs.k8s.io/kustomize/cmd/config: v0.10.9 → v0.11.1 - sigs.k8s.io/kustomize/kyaml: v0.13.9 → v0.14.1 ### Removed - github.com/PuerkitoBio/purell: [v1.1.1](https://github.com/PuerkitoBio/purell/tree/v1.1.1) - github.com/PuerkitoBio/urlesc: [de5bf2a](https://github.com/PuerkitoBio/urlesc/tree/de5bf2a) - github.com/mattn/go-runewidth: [v0.0.7](https://github.com/mattn/go-runewidth/tree/v0.0.7) - github.com/niemeyer/pretty: [a10e7ca](https://github.com/niemeyer/pretty/tree/a10e7ca) - github.com/olekukonko/tablewriter: [v0.0.4](https://github.com/olekukonko/tablewriter/tree/v0.0.4) - sigs.k8s.io/kustomize/kustomize/v4: v4.5.7 # v1.27.0-alpha.3 ## Downloads for v1.27.0-alpha.3 ### Source Code filename | sha512 hash -------- | ----------- [kubernetes.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes.tar.gz) | 86bdc8dfcb5ce47ef6f57917a9deed3dbd800669411e494f565bcb0fc6caf2982026d995205cac614e608be2cf240804668fc9f90579bef0872ee5e5ef33f4d8 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes-src.tar.gz) | c8395e5693aa148b0b326477b78d1067ff4368f34c755c003938fd88a777ae2303b102d2da240e762cf40cf171cc7a70746a4ddee4c6a35a16bd0eb9265877af ### Client Binaries filename | sha512 hash -------- | ----------- [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes-client-darwin-amd64.tar.gz) | 677ee66b49d137335a16ec3acc7fef33bcf1cee094edc0370487ac8060d728ed009c92b3438dbeb675c113320991ea9bf703579dbaf929883a82874222534760 [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes-client-darwin-arm64.tar.gz) | d1dbdc0d3f9ad5772aa33276a097f0848f445072c4a3cad1902c3fa015952aa0b78656d144a792beccbfe1bf7b78c0257b4507edcda00fb960fd2249b2cab5fd [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes-client-linux-386.tar.gz) | 8f85dfe2f157921b6b3250c744549c2ed4ccd491279b16eb0eb45402ad8f72e73a368eeaa46a4cc309fc5079c333b6f810d5904f87bb282b07b1d0abb0e22586 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes-client-linux-amd64.tar.gz) | a058c582c63da65fd9a4b95ea80e67367d0a98e5181c8733f6fe1023c356a85c87594c1ade300eb00f3ef3ef6a32f13c44eb8394f862f512217b4d437b6dbb63 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes-client-linux-arm.tar.gz) | 403e8083abc8ee509e636d0b58a013b35c7b36f10fb5e75cccbefbd6bc9a2760dfbd1ad40845b7dc0f5d78d6c93d4a6527e119a4e5f74f5dfa71281bda6a64bb [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes-client-linux-arm64.tar.gz) | a4defcf0bb8684cfe49352016abf1c720c36cc2cec8a599f987b164669ce67f581b23134a4194f4a2fb58ac489fd5b7c99ad5539efffa5a7ac5aaa2a31c79f65 [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes-client-linux-ppc64le.tar.gz) | fdf33f56239537311b1839f22de6db9a60f515bdcfadd5e470f5c459ab1a1d1fcdd67e27abdd4d6fcc5b356a8188794e7605d9633dd9364a4cccc01bce027357 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes-client-linux-s390x.tar.gz) | 8475beee121129cb7bf68763ba2cba816f9ad0daec055b564074d9364674b254dc5b368f5acf648ae32b00f0f925b7d8c0403c9ca15754fabf635c272e26a64a [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes-client-windows-386.tar.gz) | 7a03a2918722fb94e4a2cee827f6abf13184479274e9c246573cc5515bd9eb4cae46c62988bf9072ae0572086d4689986b78ee5349fcefa60a62e20f17c28d45 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes-client-windows-amd64.tar.gz) | 52e8d2da2ec5e3f51f79a2f946c49f57ab0c22321d721914a91656100a721f7278baf419d6aa629c9e8247be60177cc8cb077499b692159a6496506b56e8e17a [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes-client-windows-arm64.tar.gz) | efa2d76d57b6e3d9eb897a6e8d9812d67d4d57b9368754e22efeaa4a031d34e1ba4e8db0272b5f2ae9ee6d5615ab078201bc35fa4762f1e32c6b67c8fb7b6c8d ### Server Binaries filename | sha512 hash -------- | ----------- [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes-server-linux-amd64.tar.gz) | a1602f5be2abcbc67c763533c7352ac067fe9f2e778b3b2eeb72ee39c885e299edb53f01889420199111cd53465b463290097b8a4f8879c91b912ffbf1aa7dde [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes-server-linux-arm.tar.gz) | e129be174e855f48a5de6cf07731c9897edb360c9df1f842915323a8abf2bf704be1c275df04c93a878edbb51ea3316eeb37779ba3ebf3dc19887c97c4b5bc23 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes-server-linux-arm64.tar.gz) | f890256c2e5b4096ee944c095df2ff5fd38056ed09deabcbe7501f6b9b5665267e775d1db081f94735dba4e9423f0bebca4b275a2b38389119513bbe2d2b53c2 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes-server-linux-ppc64le.tar.gz) | 878ebfa012b184c93505f62e30ada071420bd199ef22d4690c28dff6cbc48de6048978d41d7a6f518d26564261c9da4b7fefce8f0da1e5d274ac871696bb7b93 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes-server-linux-s390x.tar.gz) | 217bcec3012f3aefa612178e9842779825cf8d50dc7e1d6a84239206f70976d939c1089d177851f3657a1f7f763ebe6a31b3ff3d99c1120105036cf37650900c ### Node Binaries filename | sha512 hash -------- | ----------- [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes-node-linux-amd64.tar.gz) | c15b9a152ceed976aff0b64c2d601985db4e58785b1a756fac6e4c83be27c70e96a277692448d75fccc69b82a5d2a872fa2ab3b5fe4b4145eacbab3d0fa03086 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes-node-linux-arm.tar.gz) | 6c0f7d3b97532aacd7aaba931bf4fb858cf49559bf51ea0e154efc4ee26de8d01f7f39f017462e136afe5faaa8d86ee49e954c3a9051751ce8dfa8312cd95f06 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes-node-linux-arm64.tar.gz) | fb2934282018539abb5ba0fde67225d5131d08c5798ed54dc93081739a803210c9aaf73412c56af7aa9999b497aac073be42ec3a7677255e5d4e39783330bd86 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes-node-linux-ppc64le.tar.gz) | 84078c93f7661c5aedf276964cc5866e1eba9b8cbc9e1c40b9e5bfe76ad115a632f4f591b177dc5fa926cb65a66c6847fb243b82ad1f880796d1d7cfe061b498 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes-node-linux-s390x.tar.gz) | dff5a35a352e5a17abad8eaa51b9447ca0a8a52df4b22e325ac7acf70efe3d0fe9bef958e8bb5a1e90d43c4586551d456e2dcf42cd01074605c07d69a52fc5b7 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.3/kubernetes-node-windows-amd64.tar.gz) | a2da87f8a7bd25d6efc8c3b7af79e268809e743ce36c187cdc0af3ff423d54f95a5bd03b50494aea8b0c394ca716e5c4225e7e2d49f40f08db6cd302d894eb3b ### Container Images All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures ---- | ------------- [registry.k8s.io/conformance:v1.27.0-alpha.3](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) [registry.k8s.io/kube-apiserver:v1.27.0-alpha.3](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) [registry.k8s.io/kube-controller-manager:v1.27.0-alpha.3](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) [registry.k8s.io/kube-proxy:v1.27.0-alpha.3](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) [registry.k8s.io/kube-scheduler:v1.27.0-alpha.3](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) ## Changelog since v1.27.0-alpha.2 ## Changes by Kind ### Deprecation - Added a [warning](https://k8s.io/blog/2020/09/03/warnings/) response when handling requests that set the deprecated `spec.externalID` field for a Node. ([#115944](https://github.com/kubernetes/kubernetes/pull/115944), [@SataQiu](https://github.com/SataQiu)) [SIG Node] ### API Change - Graduated seccomp profile defaulting to GA. Set the kubelet `--seccomp-default` flag or `seccompDefault` kubelet configuration field to `true` to make pods on that node default to using the `RuntimeDefault` seccomp profile. Enabling seccomp for your workload can have a negative performance impact depending on the kernel and container runtime version in use. Guidance for identifying and mitigating those issues is outlined in the Kubernetes [seccomp tutorial](https://k8s.io/docs/tutorials/security/seccomp). ([#115719](https://github.com/kubernetes/kubernetes/pull/115719), [@saschagrunert](https://github.com/saschagrunert)) [SIG API Machinery, Node, Storage and Testing] - Implements API for streaming for the watch-cache When sendInitialEvents ListOption is set together with watch=true, it begins the watch stream with synthetic init events followed by a synthetic "Bookmark" after which the server continues streaming events. ([#110960](https://github.com/kubernetes/kubernetes/pull/110960), [@p0lyn0mial](https://github.com/p0lyn0mial)) [SIG API Machinery] - Introduce API for streaming. Add SendInitialEvents field to the ListOptions. When the new option is set together with watch=true, it begins the watch stream with synthetic init events followed by a synthetic "Bookmark" after which the server continues streaming events. ([#115402](https://github.com/kubernetes/kubernetes/pull/115402), [@p0lyn0mial](https://github.com/p0lyn0mial)) [SIG API Machinery] - Kubelet: a "maxParallelImagePulls" field can now be specified in the kubelet configuration file to control how many image pulls the kubelet can perform in parallel. ([#115220](https://github.com/kubernetes/kubernetes/pull/115220), [@ruiwen-zhao](https://github.com/ruiwen-zhao)) [SIG API Machinery, Node and Scalability] - PodSchedulingReadiness is graduated to beta. ([#115815](https://github.com/kubernetes/kubernetes/pull/115815), [@Huang-Wei](https://github.com/Huang-Wei)) [SIG API Machinery, Apps, Scheduling and Testing] - In-place resize feature for Kubernetes Pods - Changed the Pod API so that the `resources` defined for containers are mutable for `cpu` and `memory` resource types. - Added `resizePolicy` for containers in a pod to allow users control over how their containers are resized. - Added `allocatedResources` field to container status in pod status that describes the node resources allocated to a pod. - Added `resources` field to container status that reports actual resources applied to running containers. - Added `resize` field to pod status that describes the state of a requested pod resize. For details, see KEPs below. ([#102884](https://github.com/kubernetes/kubernetes/pull/102884), [@vinaykul](https://github.com/vinaykul)) [SIG API Machinery, Apps, Instrumentation, Node, Scheduling and Testing] - The PodDisruptionBudget `spec.unhealthyPodEvictionPolicy` field has graduated to beta and is enabled by default. On servers with the feature enabled, this field may be set to `AlwaysAllow` to always allow unhealthy pods covered by the PodDisruptionBudget to be evicted. ([#115363](https://github.com/kubernetes/kubernetes/pull/115363), [@ravisantoshgudimetla](https://github.com/ravisantoshgudimetla)) [SIG Apps, Auth and Node] - The `DownwardAPIHugePages` kubelet feature graduated to stable / GA. ([#115721](https://github.com/kubernetes/kubernetes/pull/115721), [@saschagrunert](https://github.com/saschagrunert)) [SIG Apps and Node] - Volumes: `resource.claims` gets cleared for PVC specs during create or update of a pod spec with inline PVC template or of a PVC because it has no effect. ([#115928](https://github.com/kubernetes/kubernetes/pull/115928), [@pohly](https://github.com/pohly)) [SIG API Machinery, Apps and Storage] ### Feature - API validation relaxed allowing Indexed Jobs to be scaled up/down by changing parallelism and completions in tandem, such that parallelism == completions. ([#115236](https://github.com/kubernetes/kubernetes/pull/115236), [@danielvegamyhre](https://github.com/danielvegamyhre)) [SIG Apps and Testing] - Add kubelet Topology Manager metric to measure topology manager admission latency. ([#115590](https://github.com/kubernetes/kubernetes/pull/115590), [@swatisehgal](https://github.com/swatisehgal)) [SIG Node and Testing] - Added "netadmin" debugging profiles for kubectl debug. ([#115712](https://github.com/kubernetes/kubernetes/pull/115712), [@wedaly](https://github.com/wedaly)) [SIG CLI] - Added apiserver_envelope_encryption_invalid_key_id_from_status_total to measure number of times an invalid keyID is returned by the Status RPC call. ([#115846](https://github.com/kubernetes/kubernetes/pull/115846), [@ritazh](https://github.com/ritazh)) [SIG API Machinery and Auth] - Apiserver_storage_transformation_operations_total metric has been updated to include labels transformer_prefix and status. ([#115394](https://github.com/kubernetes/kubernetes/pull/115394), [@ritazh](https://github.com/ritazh)) [SIG API Machinery, Auth, Instrumentation and Testing] - Client-go: metadatainformer and dynamicinformer `SharedInformerFactory`s now supports waiting for goroutines during shutdown ([#114434](https://github.com/kubernetes/kubernetes/pull/114434), [@howardjohn](https://github.com/howardjohn)) [SIG API Machinery] - Graduate the `ReadWriteOncePod` feature gate to beta ([#114494](https://github.com/kubernetes/kubernetes/pull/114494), [@chrishenzie](https://github.com/chrishenzie)) [SIG Scheduling, Storage and Testing] - Kubeadm: show a warning message when detecting that the sandbox image of the container runtime is inconsistent with that used by kubeadm ([#115610](https://github.com/kubernetes/kubernetes/pull/115610), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle] - Kubernetes is now built with go 1.20.1 ([#115828](https://github.com/kubernetes/kubernetes/pull/115828), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing] - Performance improvements in klog ([#115277](https://github.com/kubernetes/kubernetes/pull/115277), [@pohly](https://github.com/pohly)) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node, Storage and Testing] - Pod template `schedulingGates` are now mutable for Jobs that are suspended and have never been started ([#115940](https://github.com/kubernetes/kubernetes/pull/115940), [@ahg-g](https://github.com/ahg-g)) [SIG Apps] - Pods which have an invalid negative `spec.terminationGracePeriodSeconds` value will be treated as having terminationGracePeriodSeconds of 1 ([#115606](https://github.com/kubernetes/kubernetes/pull/115606), [@wzshiming](https://github.com/wzshiming)) [SIG Apps, Node and Testing] - The Pod API field `.spec.schedulingGates[*].name` now requires qualified names (like `example.com/mygate`), matching validation for names of `.spec.readinessGates[*].name`. Any uses of the alpha scheduling gate feature prior to 1.27 that do not match that validation must be renamed or deleted before upgrading to 1.27. ([#115821](https://github.com/kubernetes/kubernetes/pull/115821), [@lianghao208](https://github.com/lianghao208)) [SIG Apps and Scheduling] - The `JobMutableNodeSchedulingDirectives` feature gate has graduated to GA. ([#116116](https://github.com/kubernetes/kubernetes/pull/116116), [@ahg-g](https://github.com/ahg-g)) [SIG Apps, Scheduling and Testing] - Two changes to the /debug/api_priority_and_fairness/dump_priority_levels endpoint of API Priority and Fairness: add total number of dispatched, timed-out, rejected and cancelled requests; output now sorted by PriorityLevelName. ([#112393](https://github.com/kubernetes/kubernetes/pull/112393), [@borgerli](https://github.com/borgerli)) [SIG API Machinery] - Updated distroless iptables to use released image `registry.k8s.io/distroless-iptables:v0.2.1` ([#115905](https://github.com/kubernetes/kubernetes/pull/115905), [@cpanato](https://github.com/cpanato)) [SIG Testing] - [E2E] Pods spawned by E2E tests can now pull images from the private registry using the new --e2e-docker-config-file flag ([#114625](https://github.com/kubernetes/kubernetes/pull/114625), [@Divya063](https://github.com/Divya063)) [SIG Node and Testing] ### Documentation - Document the reason field in CRI API to ensure it equals OOMKilled for the containers terminated by OOM killer ([#112977](https://github.com/kubernetes/kubernetes/pull/112977), [@mimowo](https://github.com/mimowo)) [SIG Node] ### Failing Test - Fixed panic in vSphere e2e tests. ([#115863](https://github.com/kubernetes/kubernetes/pull/115863), [@jsafrane](https://github.com/jsafrane)) [SIG Storage and Testing] ### Bug or Regression - Cacher: If RV is unset, the watch is now served from the underlying storage as documented. ([#115096](https://github.com/kubernetes/kubernetes/pull/115096), [@MadhavJivrajani](https://github.com/MadhavJivrajani)) [SIG API Machinery] - Client-go: fix the wait time for trying to acquire the leader lease ([#114872](https://github.com/kubernetes/kubernetes/pull/114872), [@Iceber](https://github.com/Iceber)) [SIG API Machinery] - File content check for IPV4 is not enabled by default, and the check of IPV4 or IPV6 is done for `kubeadm init` or `kubeadm join` only in case the user intends to create a cluster to support that kind of IP address family ([#115420](https://github.com/kubernetes/kubernetes/pull/115420), [@chendave](https://github.com/chendave)) [SIG Cluster Lifecycle and Network] - Fix log line in scheduler that inaccurately implies that volume binding has finalized ([#116018](https://github.com/kubernetes/kubernetes/pull/116018), [@TommyStarK](https://github.com/TommyStarK)) [SIG Scheduling and Storage] - Fix missing delete events on informer re-lists to ensure all delete events are correctly emitted and using the latest known object state, so that all event handlers and stores always reflect the actual apiserver state as best as possible ([#115620](https://github.com/kubernetes/kubernetes/pull/115620), [@odinuge](https://github.com/odinuge)) [SIG API Machinery] - Fixed a bug where Kubernetes would apply a default StorageClass to a PersistentVolumeClaim, even when the deprecated annotation `volume.beta.kubernetes.io/storage-class` was set. ([#116089](https://github.com/kubernetes/kubernetes/pull/116089), [@cvvz](https://github.com/cvvz)) [SIG Apps and Storage] - Fixed an EndpointSlice Controller hashing bug that could cause EndpointSlices to incorrectly handle Pods with duplicate IP addresses. For example this could happen when a new Pod reused an IP that was also assigned to a Pod in a completed state. ([#115907](https://github.com/kubernetes/kubernetes/pull/115907), [@qinqon](https://github.com/qinqon)) [SIG Apps and Network] - Fixing issue with Winkernel Proxier - ClusterIP Loadbalancers are missing if the ExternalTrafficPolicy is set to Local and the available endpoints are all remoteEndpoints. ([#115919](https://github.com/kubernetes/kubernetes/pull/115919), [@princepereira](https://github.com/princepereira)) [SIG Network and Windows] - Golang.org/x/net updates to v0.7.0 to fix CVE-2022-41723 ([#115786](https://github.com/kubernetes/kubernetes/pull/115786), [@liggitt](https://github.com/liggitt)) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node and Storage] - Kubeadm: fix a bug where the uploaded kubelet configuration in `kube-system/kubelet-config` ConfigMap does not respect user patch ([#115575](https://github.com/kubernetes/kubernetes/pull/115575), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle] - Kubeadm: modify '--config' flag from required to optional for 'kubeadm kubeconfig user' command ([#116074](https://github.com/kubernetes/kubernetes/pull/116074), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle] - Yes, discovery document will correctly return the resources for aggregated apiservers that do not implement aggregated disovery ([#115770](https://github.com/kubernetes/kubernetes/pull/115770), [@Jefftree](https://github.com/Jefftree)) [SIG API Machinery] ### Other (Cleanup or Flake) - Improved FormatMap: Improves performance by about 4x, or nearly 2x in the worst case ([#112661](https://github.com/kubernetes/kubernetes/pull/112661), [@aimuz](https://github.com/aimuz)) [SIG Node] - Upgrade go-jose to v2.6.0 ([#115893](https://github.com/kubernetes/kubernetes/pull/115893), [@mgoltzsche](https://github.com/mgoltzsche)) [SIG API Machinery, Auth, Cluster Lifecycle and Testing] - `apiserver_admission_webhook_admission_duration_seconds` buckets have been expanded, 25s is now the largest bucket size to match the webhook default timeout. ([#115802](https://github.com/kubernetes/kubernetes/pull/115802), [@logicalhan](https://github.com/logicalhan)) [SIG API Machinery and Instrumentation] ## Dependencies ### Added _Nothing has changed._ ### Changed - github.com/coredns/corefile-migration: [v1.0.18 → v1.0.20](https://github.com/coredns/corefile-migration/compare/v1.0.18...v1.0.20) - github.com/golang-jwt/jwt/v4: [v4.2.0 → v4.4.2](https://github.com/golang-jwt/jwt/v4/compare/v4.2.0...v4.4.2) - go.etcd.io/etcd/api/v3: v3.5.5 → v3.5.7 - go.etcd.io/etcd/client/pkg/v3: v3.5.5 → v3.5.7 - go.etcd.io/etcd/client/v2: v2.305.5 → v2.305.7 - go.etcd.io/etcd/client/v3: v3.5.5 → v3.5.7 - go.etcd.io/etcd/pkg/v3: v3.5.5 → v3.5.7 - go.etcd.io/etcd/raft/v3: v3.5.5 → v3.5.7 - go.etcd.io/etcd/server/v3: v3.5.5 → v3.5.7 - go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp: v0.35.0 → v0.35.1 - golang.org/x/net: v0.5.0 → v0.7.0 - golang.org/x/sys: v0.4.0 → v0.5.0 - golang.org/x/term: v0.4.0 → v0.5.0 - golang.org/x/text: v0.6.0 → v0.7.0 - gopkg.in/square/go-jose.v2: v2.2.2 → v2.6.0 - k8s.io/klog/v2: v2.80.1 → v2.90.1 ### Removed - github.com/form3tech-oss/jwt-go: [v3.2.3+incompatible](https://github.com/form3tech-oss/jwt-go/tree/v3.2.3) # v1.27.0-alpha.2 ## Downloads for v1.27.0-alpha.2 ### Source Code filename | sha512 hash -------- | ----------- [kubernetes.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes.tar.gz) | 5420d881db6412c1c1e55044aea61f310ef42d7809d4d90113b2a80ae0d1446f3e7988a8205100c476a313182a0c8b2d1605ad3000eee3b45fec4034d17f2ac2 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes-src.tar.gz) | 3b9693bd03ed7f5aee3257a167e431b9de4c576f8843f1441f81cbcdfc6be607c84ca703bd2e7ca4bb5f3b9dee9fbb8645cdf49c1921d796e1a3f027c8f23162 ### Client Binaries filename | sha512 hash -------- | ----------- [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes-client-darwin-amd64.tar.gz) | ce9875156a7c80452dc3303177b0a137cfc6ae398b66a32b1436768ab77771b000287dba0702510da239c056a697e624416f6126a6205c3c65e78ff6d7d4635b [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes-client-darwin-arm64.tar.gz) | bc0791295f926f285f18163bef7faf893162918d75e8de0aa46704d2ac665bbff641a7332d5a3d112d93dd5e14087f8e7333e39b4cc44ab71330e059b0abe4bb [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes-client-linux-386.tar.gz) | 76abdb1dbb8886c554628ba634449c42f0c61eb47e168d1cb7bb1eabb0354b37474738879955805165e52a4cbbe39c57cabe63a2b8dbbaed00e14a0da5a7419f [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes-client-linux-amd64.tar.gz) | 5ced29d3f8411f34ba9dbd115aab7e45d542b34f7feebba6bfe8dcd394abd9fe127daa6c36460c2d8b35ca6386729a3e644b23b5631fe4d81ae3ae0cf1297e67 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes-client-linux-arm.tar.gz) | fff2ce7f24f9fa6c5240d69d81a0c363742b17963265ce744e3366d05d149bce005cfe96fb1dd20b7c5faceed481225da0715dee8b2743ee3ff21391c742a1a0 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes-client-linux-arm64.tar.gz) | 54da883e07f1a6e6bb9ca29ca4b5bedb2d24485cd07c8ba03da90b063a07e01271d0ad3b58d20fc3370a40486134b7b6144ad2d18049d7e3a38600ad14d84f8f [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes-client-linux-ppc64le.tar.gz) | 50be6728b20612ea3d422e3346150c4cece1cd42356446cf8fd2f9164a40ac997188d840536dc45deab5acf12143233d36b76d9ef12165bb0884024f1725f28c [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes-client-linux-s390x.tar.gz) | fa76f8655266fb9b64c450185c175f6854ac1d569140f6d62c383d829111091b79a2f266929ca10f642a989e9ada066988845666621fe13c75cbcfa971f5aa0d [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes-client-windows-386.tar.gz) | 67c36d790cd5de91e0241cee3800fcdb49db3f3a9e91e087937686367149d7b06c489e62919ef6fdcb8ec29974ead4a64bab0eb3278f404188cf8fffe89baba9 [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes-client-windows-amd64.tar.gz) | 49e73490b58576237627cc8015f84eda36aa3af02b8e80b251b07d294fe161e90815ee4149f3b8605fad7c43b278f7b0f631ae3d51fa344ad326abcd480d781d [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes-client-windows-arm64.tar.gz) | 27ac0573663d5e45585b205c84cb0e5a7f16282654e30445ad4115a148c20b548d0c3520e45327a4443768c83b68c96e8754cdfb34f17fa0ebf875e4eec2eb48 ### Server Binaries filename | sha512 hash -------- | ----------- [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes-server-linux-amd64.tar.gz) | 0a2fa2de60af23f722a27479ee0721551561a6bf947ec66c9548b0d14410745de2db3f69c6536768768ffeec4f6afe3af3bd336aeccef67391c4cdaca4a427a6 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes-server-linux-arm.tar.gz) | c5ab1da7a7e19acebdba7107c27954e522d33c245ea04556347b601e2cc0f40595b9ca5159661b134a090d5505a76d967da3161b3a409b2a4d9c0f36e1b4d7b1 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes-server-linux-arm64.tar.gz) | 3a5cad9ae0f4a1086a238e2fb44f59733361d9dea206390c73825daf25dc8b333fce166f5c5e6c0e1ca3be80b303afb1b6b6c8e9dc13666446c2a70b5b7bc1cb [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes-server-linux-ppc64le.tar.gz) | 411be709cde53aa27bca30e7d5ab7523f4dea192c85a1aa810985b23a41cdd00c6969e9b9614a193618be94d346900c2f8e9211c95927a12398142268db4ce5e [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes-server-linux-s390x.tar.gz) | 0ac65f78a5cad3506649ca40912f328b2af747ae6367f0ca16ba741af20aeecd73c27ea216921ab7a28da8a61b58f0760e9211c65953e70304fec4b940a39440 ### Node Binaries filename | sha512 hash -------- | ----------- [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes-node-linux-amd64.tar.gz) | 3a0559b2305136a15cd43104ce728f7651b4fcde13db69f565d66e117ad7f8f30a017d3ea6be92811e4ab880273033c089688675559912bfb6d2aa2c92d60225 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes-node-linux-arm.tar.gz) | ac7597cfab9eb93dd9c0f1cd088dd08d120991bc94a718fa89ddd9b8fa12a9f6b9987eaaee66b8aafbb055c836c289ca7ca415b57f61bd8f9159045025026100 [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes-node-linux-arm64.tar.gz) | 395d65f26b4f482cd1d8be49b846ce80f536ca825ae8ce25d10fe746d95e4297c31512247d22caefe632d2236a33616e2650ed385811ced24c3e6338a5eda36d [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes-node-linux-ppc64le.tar.gz) | 7f92e5fdfba981ac80b71fdc00e84b4eb661604861f5602e5fa489f13a10ac699e6e5795cc3654ff95e1b5f7fd51df7773bd5ae511ace9b861a87b6fb1465cc7 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes-node-linux-s390x.tar.gz) | 75ec78e900a4df4819c899893fe98fe32b6fa8ae000318dcfed8972d356cc1c5e0a3875885681375c080b0770c377164c03c100a6c45b7d025363e174a00af00 [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.2/kubernetes-node-windows-amd64.tar.gz) | d7630730d547414bdb2b245e1b444a5949cecead751d6c243db72e8f20782ba85e1c06dfab499c13e1b529a74a5d4acef4a9b1a6ca571faf41d3253b1bf74773 ### Container Images All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures ---- | ------------- [registry.k8s.io/conformance:v1.27.0-alpha.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) [registry.k8s.io/kube-apiserver:v1.27.0-alpha.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) [registry.k8s.io/kube-controller-manager:v1.27.0-alpha.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) [registry.k8s.io/kube-proxy:v1.27.0-alpha.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) [registry.k8s.io/kube-scheduler:v1.27.0-alpha.2](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) ## Changelog since v1.27.0-alpha.1 ## Changes by Kind ### API Change - A fix in the resource.k8s.io/v1alpha1/ResourceClaim API avoids harmless (?) ".status.reservedFor: element 0: associative list without keys has an element that's a map type" errors in the apiserver. Validation now rejects the incorrect reuse of the same UID in different entries. ([#115354](https://github.com/kubernetes/kubernetes/pull/115354), [@pohly](https://github.com/pohly)) [SIG API Machinery] - CacheSize field in EncryptionConfiguration is not supported for KMSv2 provider ([#113121](https://github.com/kubernetes/kubernetes/pull/113121), [@aramase](https://github.com/aramase)) [SIG API Machinery, Auth and Testing] - K8s.io/client-go/tools/record.EventBroadcaster: after Shutdown() is called, the broadcaster now gives up immediately after a failure to write an event to a sink. Previously it tried multiple times for 12 seconds in a goroutine. ([#115514](https://github.com/kubernetes/kubernetes/pull/115514), [@pohly](https://github.com/pohly)) [SIG API Machinery] - K8s.io/component-base/logs now also supports adding command line flags to a flag.FlagSet. ([#114731](https://github.com/kubernetes/kubernetes/pull/114731), [@pohly](https://github.com/pohly)) [SIG Architecture] - Update API reference for Requests, specifying they must not exceed limits ([#115434](https://github.com/kubernetes/kubernetes/pull/115434), [@ehashman](https://github.com/ehashman)) [SIG Architecture, Docs and Node] - `/metrics/slis` is made available for control plane components allowing you to scrape health check metrics. ([#114997](https://github.com/kubernetes/kubernetes/pull/114997), [@Richabanker](https://github.com/Richabanker)) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scheduling, Storage and Testing] ### Feature - A new client side metric `rest_client_request_retries_total` has been added that tracks the number of retries sent to the server, partitioned by status code, verb, and host ([#108396](https://github.com/kubernetes/kubernetes/pull/108396), [@tkashem](https://github.com/tkashem)) [SIG API Machinery, Architecture and Instrumentation] - A new feature has been enabled to improve the performance of the iptables mode of kube-proxy in large clusters. You do not need to take any action, however: 1. If you experience problems with Services not syncing to iptables correctly, you can disable the feature by passing `--feature-gates=MinimizeIPTablesRestore=false` to kube-proxy (and file a bug if this fixes it). (This might also be detected by seeing the value of kube-proxy's `sync_proxy_rules_iptables_partial_restore_failures_total` metric rising.) 2. If you were previously overriding the kube-proxy configuration for performance reasons, this may no longer be necessary. See https://kubernetes.io/docs/reference/networking/virtual-ips/#optimizing-iptables-mode-performance. ([#115138](https://github.com/kubernetes/kubernetes/pull/115138), [@danwinship](https://github.com/danwinship)) [SIG Network] - Add kubelet Topology Manager metrics to track admission requests processed by it and occured admission errors. ([#115137](https://github.com/kubernetes/kubernetes/pull/115137), [@swatisehgal](https://github.com/swatisehgal)) [SIG Node and Testing] - Add logging-format option to CCMs based on k8s.io/cloud-provider ([#108984](https://github.com/kubernetes/kubernetes/pull/108984), [@LittleFox94](https://github.com/LittleFox94)) [SIG Cloud Provider and Instrumentation] - Add new -f flag into debug command to be used passing pod or node files instead explicit names. ([#111453](https://github.com/kubernetes/kubernetes/pull/111453), [@ardaguclu](https://github.com/ardaguclu)) [SIG CLI and Testing] - Added "general", "baseline", and "restricted" debugging profiles for kubectl debug. ([#114280](https://github.com/kubernetes/kubernetes/pull/114280), [@sding3](https://github.com/sding3)) [SIG CLI] - Added apiserver_envelope_encryption_kms_operations_latency_seconds metric to measure the KMSv2 grpc calls latency. ([#115649](https://github.com/kubernetes/kubernetes/pull/115649), [@aramase](https://github.com/aramase)) [SIG API Machinery, Auth and Testing] - Adds scheduler preemption support for pods using `ReadWriteOncePod` PVCs ([#114051](https://github.com/kubernetes/kubernetes/pull/114051), [@chrishenzie](https://github.com/chrishenzie)) [SIG Scheduling, Storage and Testing] - Adds the applyconfiguration generator to the code-generator script that generates server-side apply configuration and client APIs ([#114987](https://github.com/kubernetes/kubernetes/pull/114987), [@astefanutti](https://github.com/astefanutti)) [SIG API Machinery] - Dynamic Resource Allocation framework can be used for network devices ([#114364](https://github.com/kubernetes/kubernetes/pull/114364), [@bart0sh](https://github.com/bart0sh)) [SIG Node] - Fixed bug which caused the status of Indexed Jobs to only be updated when there are newly completed indexes. The completed indexes are now updated if the .status.completedIndexes has values outside of the [0, .spec.completions> range ([#115349](https://github.com/kubernetes/kubernetes/pull/115349), [@danielvegamyhre](https://github.com/danielvegamyhre)) [SIG Apps] - GRPC probes now set a linger option of 1s to improve the TIME-WAIT state. ([#115321](https://github.com/kubernetes/kubernetes/pull/115321), [@rphillips](https://github.com/rphillips)) [SIG Network and Node] - Kubelet config file will be backed up to `/etc/kubernetes/tmp/` folder with `kubeadm-kubelet-config` append with a random suffix as the filename ([#114695](https://github.com/kubernetes/kubernetes/pull/114695), [@chendave](https://github.com/chendave)) [SIG Cluster Lifecycle] - Kubelet no longer creates certain legacy iptables rules by default. It is possible that this will cause problems with some third-party components that improperly depended on those rules. If this affects you, you can run kubelet with `--feature-gates=IPTablesOwnershipCleanup=false`, but you should also file a bug against the third-party component. ([#114472](https://github.com/kubernetes/kubernetes/pull/114472), [@danwinship](https://github.com/danwinship)) [SIG Network] - Kubernetes is now built with go 1.20 ([#114502](https://github.com/kubernetes/kubernetes/pull/114502), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing] - Migrated the ResourceQuota controller (within `kube-controller-manager`) to use [contextual logging](https://k8s.io/docs/concepts/cluster-administration/system-logs/#contextual-logging). ([#113315](https://github.com/kubernetes/kubernetes/pull/113315), [@ncdc](https://github.com/ncdc)) [SIG API Machinery, Apps and Testing] - New feature gate, ServiceNodePortStaticSubrange, to enable the new strategy in the NodePort Service port allocators, so the node port range is subdivided and dynamic allocated NodePort port for Services are allocated preferentially from the upper range. ([#114418](https://github.com/kubernetes/kubernetes/pull/114418), [@xuzhenglun](https://github.com/xuzhenglun)) [SIG Network] - Scheduler doesn't run plugin's Score method when its PreScore method returned a Skip status. In other words, your PreScore/Score plugin can return a Skip status in PreScore if the plugin does nothing in Score for that Pod. ([#115652](https://github.com/kubernetes/kubernetes/pull/115652), [@kidddddddddddddddddddddd](https://github.com/kidddddddddddddddddddddd)) [SIG Scheduling] - The go version defined in `.go-version` is now fetched when invoking test, build, and code generation targets if the current go version does not match it. Set $FORCE_HOST_GO=y while testing or building to skip this behavior, or set $GO_VERSION to override the selected go version. ([#115377](https://github.com/kubernetes/kubernetes/pull/115377), [@liggitt](https://github.com/liggitt)) [SIG Testing] - The mount-utils mounter now provides an option to limit the number of concurrent format operations. ([#115379](https://github.com/kubernetes/kubernetes/pull/115379), [@artemvmin](https://github.com/artemvmin)) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node and Storage] ### Bug or Regression - Apply configurations can be generated for types with non-builtin map fields ([#114920](https://github.com/kubernetes/kubernetes/pull/114920), [@astefanutti](https://github.com/astefanutti)) [SIG API Machinery] - Enforce nodeName cannot be set along with non-empty schedulingGates ([#115569](https://github.com/kubernetes/kubernetes/pull/115569), [@Huang-Wei](https://github.com/Huang-Wei)) [SIG Apps and Scheduling] - Etcd: Update to v3.5.7 ([#115310](https://github.com/kubernetes/kubernetes/pull/115310), [@mzaian](https://github.com/mzaian)) [SIG API Machinery, Cloud Provider, Cluster Lifecycle and Testing] - Fix a bug that caused to panic the apiserver when trying to allocate a Service with a dynamic ClusterIP and it has been configured with Service CIDRs with a /28 mask for IPv4 and a /124 mask for IPv6 ([#115322](https://github.com/kubernetes/kubernetes/pull/115322), [@aojea](https://github.com/aojea)) [SIG Testing] - Fix an issue where a CSI migrated volume may be prematurely detached when the CSI driver is not running on the node. If CSI migration is enabled on the node, even the csi-driver is not up and ready, we will still add this volume to DSW. ([#115464](https://github.com/kubernetes/kubernetes/pull/115464), [@sunnylovestiramisu](https://github.com/sunnylovestiramisu)) [SIG Apps and Storage] - Fix nil pointer error in nodevolumelimits csi logging ([#115179](https://github.com/kubernetes/kubernetes/pull/115179), [@sunnylovestiramisu](https://github.com/sunnylovestiramisu)) [SIG Scheduling] - Fix the regression that introduced 34s timeout for DELETECOLLECTION calls ([#115341](https://github.com/kubernetes/kubernetes/pull/115341), [@tkashem](https://github.com/tkashem)) [SIG API Machinery] - Fixing issue with Winkernel Proxier - IPV6 load balancer policies are missing when service is configured with ipFamilyPolicy: RequireDualStack ([#115503](https://github.com/kubernetes/kubernetes/pull/115503), [@princepereira](https://github.com/princepereira)) [SIG Network and Windows] - Fixing issue with Winkernel Proxier - IPV6 load balancer policies are missing when service is configured with ipFamilyPolicy: RequireDualStack ([#115577](https://github.com/kubernetes/kubernetes/pull/115577), [@princepereira](https://github.com/princepereira)) [SIG Network and Windows] - Flag `workerCount` has been added to cloud node controller which defines how many workers will be synchronizing nodes. ([#113104](https://github.com/kubernetes/kubernetes/pull/113104), [@pawbana](https://github.com/pawbana)) [SIG API Machinery, Cloud Provider and Scalability] - Kube-apiserver: errors decoding objects in etcd are now recorded in an `apiserver_storage_decode_errors_total` counter metric ([#114376](https://github.com/kubernetes/kubernetes/pull/114376), [@baomingwang](https://github.com/baomingwang)) [SIG API Machinery and Instrumentation] - Kube-apiserver: regular expressions specified with the `--cors-allowed-origins` option are now validated to match the entire `hostname` inside the `Origin` header of the request and must contain '^' or the '//' prefix to anchor to the start, and '$' or the port separator ':' to anchor to the end. ([#112809](https://github.com/kubernetes/kubernetes/pull/112809), [@tkashem](https://github.com/tkashem)) [SIG API Machinery] - Kubeadm: fix an etcd learner-mode bug by preparing an etcd static pod manifest before promoting ([#115038](https://github.com/kubernetes/kubernetes/pull/115038), [@tobiasgiese](https://github.com/tobiasgiese)) [SIG Cluster Lifecycle] - Kubelet: fix a bug of stoping rendering configmap when enabling fsquota monitoring ([#112624](https://github.com/kubernetes/kubernetes/pull/112624), [@pacoxu](https://github.com/pacoxu)) [SIG Node and Storage] - Set device stage path whenever available for expansion during mount ([#115346](https://github.com/kubernetes/kubernetes/pull/115346), [@gnufied](https://github.com/gnufied)) [SIG Storage and Testing] - The Kubernetes API server now correctly detects and closes existing TLS connections when its client certificate file for kubelet authentication has been rotated. ([#115315](https://github.com/kubernetes/kubernetes/pull/115315), [@enj](https://github.com/enj)) [SIG API Machinery, Auth, Node and Testing] ### Other (Cleanup or Flake) - Changes docs for --contention-profiling flag to reflect it performs block profiling ([#114490](https://github.com/kubernetes/kubernetes/pull/114490), [@MadhavJivrajani](https://github.com/MadhavJivrajani)) [SIG API Machinery, Cloud Provider, Docs, Node and Scheduling] - E2e framework: added `--report-complete-ginkgo` and `--report-complete-junit` parameters. They work like `ginkgo --json-report /ginkgo/report.json --junit-report /ginkgo/report.xml`. ([#115678](https://github.com/kubernetes/kubernetes/pull/115678), [@pohly](https://github.com/pohly)) [SIG Testing] - Promote pod resource limit/request metrics to stable. ([#115454](https://github.com/kubernetes/kubernetes/pull/115454), [@dgrisonnet](https://github.com/dgrisonnet)) [SIG Instrumentation and Scheduling] - The `ControllerManagerLeaderMigration ` feature, GA since 1.24, is unconditionally enabled and the feature gate option has been removed. ([#113534](https://github.com/kubernetes/kubernetes/pull/113534), [@pacoxu](https://github.com/pacoxu)) [SIG API Machinery and Cloud Provider] ## Dependencies ### Added _Nothing has changed._ ### Changed - github.com/onsi/gomega: [v1.24.2 → v1.26.0](https://github.com/onsi/gomega/compare/v1.24.2...v1.26.0) - go.uber.org/goleak: v1.2.0 → v1.2.1 - golang.org/x/net: v0.4.0 → v0.5.0 - golang.org/x/sys: v0.3.0 → v0.4.0 - golang.org/x/term: v0.3.0 → v0.4.0 - golang.org/x/text: v0.5.0 → v0.6.0 - k8s.io/kube-openapi: 3758b55 → 1cb3ae2 - k8s.io/utils: 1a15be2 → a36077c ### Removed _Nothing has changed._ # v1.27.0-alpha.1 ## Downloads for v1.27.0-alpha.1 ### Source Code filename | sha512 hash -------- | ----------- [kubernetes.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes.tar.gz) | 36ddbb7f1cdf386cd6857d891029b7244dd13aa346a78ba2fa2146b866751802989fc5c5c8a8675f80b72b12816ae94b2b00fdeb01421ef15aad8ae87c06c512 [kubernetes-src.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes-src.tar.gz) | d5be5bdf5734c89338b2fd52942b70f8d57d831659235a0ea91c6fb10d74637c2a2aeed602bdafb4da01071764f754ddec9d37abf8aeb8eaa11636e405e93b04 ### Client Binaries filename | sha512 hash -------- | ----------- [kubernetes-client-darwin-amd64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes-client-darwin-amd64.tar.gz) | d9b5f7ec09b64a6e0d270c077fdcd4835b6e16f39373fbf1a2e2526f80aa4df25f41a7e1dab1c33c4ec3f3c430b1cd42b08944d4c39d8ac78c4b23db83d6411b [kubernetes-client-darwin-arm64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes-client-darwin-arm64.tar.gz) | e1a0f33d0610dbac940db0cad930bf4530f90a9c1702e534bb8c5524077af82d6da2f63befd3ba331ddfe84710ece101b9dd93f308ef727605646cc0d0847292 [kubernetes-client-linux-386.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes-client-linux-386.tar.gz) | 1e49e5f94a3bd14c6c5681ea8ad003948f7824566dc4d8ff299aed0a3e650a2b0674ade00fc951131fe34c210b8b7832c1a1fd5c290ee8c9e42bac89efdc8f26 [kubernetes-client-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes-client-linux-amd64.tar.gz) | ddb000a6a1604a5cb95bbe296366eebd9e0b9b4be250b0d22302c697294840c216641f287dc7212b49f9f121549172590a1f1e285b3d87cee32bcbd95a7d19b1 [kubernetes-client-linux-arm.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes-client-linux-arm.tar.gz) | 40c242c7bae26da4948fe97daf256dc48d99edc91a535ff9f4516e214ff50cfbbbc985be8e4361a97bfd54dc8b406e6f4a68b1054396b01a157d54a6bd82c7e3 [kubernetes-client-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes-client-linux-arm64.tar.gz) | 54954248f4aa1d2977fe92a552cf7e0298c94adac8bf0720c697cfce654a1fbdf01cd56219e0dd064bab7b07a4cd125c257b61cd4d69af03ec550aec31cb26ac [kubernetes-client-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes-client-linux-ppc64le.tar.gz) | d58709ea83b2a82ea44ae402a574ddc97177494bac90b9c6de30caf3fbcc79697addd0bb8245a62bbaf2fd35483415293659703cbed573bbe11a86a57814f8a3 [kubernetes-client-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes-client-linux-s390x.tar.gz) | 08ab82af97bb2ccf0b90d5de23960ce2634a42eaafcc50725ce345e3df61e082acc2e733c3aae159463645cbad28a8536371850899403eaeb7a040ed221ba861 [kubernetes-client-windows-386.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes-client-windows-386.tar.gz) | 047fdddf4c1095240eb1296a1d79e9dbb90325d13e9ed94b3532ecfa843dbd089b4b5aa3d6a51e4265c1ecdb2f08b73cd05f309b8bf4c585b213fa605a0bc74d [kubernetes-client-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes-client-windows-amd64.tar.gz) | 1bd72cfd091f452b4d32496a1f1b0c78231ae6cf9696ca029a340761aaeba08e27003d9fcb5942a63b2feddb2562b287b0d8c49fb0d92f2e50b65a947591b105 [kubernetes-client-windows-arm64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes-client-windows-arm64.tar.gz) | 07daf9e21a7741908e358f0a9ae30164229344c1653c435fb1c6029932431e5bb985842ebcc97330a6ca5aff57584488a17fe01b234de6286f6753faaa4d31f3 ### Server Binaries filename | sha512 hash -------- | ----------- [kubernetes-server-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes-server-linux-amd64.tar.gz) | d426504774023346a994ca709ae48daf47baf0e2f680a2237cc1b8b5b2ba7d4a41be9d0f80af4f1c097348b01a3e1a968b3bbea5cb5937d17e0ef23db6a36b27 [kubernetes-server-linux-arm.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes-server-linux-arm.tar.gz) | d4b4eaa2012b9ce5f3fa111cd1a61ef98aec238905f134636470bab2661d4c14464dfc8223bd74456a8848e3c7fe879d14e1eba6c4f2376f9042893077fcb068 [kubernetes-server-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes-server-linux-arm64.tar.gz) | bc9b1388a93c65f6b603271e8a1a1d622698fd54ce26d3743e888efd5f61cd538229161006a2ab5afcbda8782f87646c9da02a5e7cc93ff52e48616a7d56cb33 [kubernetes-server-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes-server-linux-ppc64le.tar.gz) | 69972b9c95b50b7566660b44547d974133e6f2f18737a93c2f4a6ca5833400473c0917c0c44b1a27336990214c1f188c75e39701d91c567b8ff7d8ccd3496243 [kubernetes-server-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes-server-linux-s390x.tar.gz) | 1867d483df955260b9cf13bfe43451c2f906c1854399a396e0d9e2fac33fc4343d3c79039891c1b943cf4707312cedcffb4d0cedf7a69271020f88c50d230d5d ### Node Binaries filename | sha512 hash -------- | ----------- [kubernetes-node-linux-amd64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes-node-linux-amd64.tar.gz) | 132c74737e8fd525c8a4bcedbf7ad783917a42a485e740586933dceef76177c6043682654f76f88670933fea0a3066adfcfdd08a9b6e07b3a6ad23e903c1b4a5 [kubernetes-node-linux-arm.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes-node-linux-arm.tar.gz) | 7d7716457ff136344e8338ad52e819346de13495d79a5c615a8bfc9d45a00fa659d60630aa2025d8bc4570080d77d1e298e47a8934d47c0c3fb5b70b043ef2dd [kubernetes-node-linux-arm64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes-node-linux-arm64.tar.gz) | df5b9d5ae99d8001b25448bf6148b9cf3a47ce8b4d1f7b1f52ab3dd8ad0ee6958045a382a561d3a617991a2255712f2e6ed0c8d6fb843edbdd0284054a238d74 [kubernetes-node-linux-ppc64le.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes-node-linux-ppc64le.tar.gz) | 9c0fc5952d8dcf07cb0225c252112fead50a4920d978f174a17a881d92c8b34ed162ec3a806bb451a990b81e8294f39755c93acbfd6147748c5dea9bb8ad38f2 [kubernetes-node-linux-s390x.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes-node-linux-s390x.tar.gz) | 0b203edad11a1350d3f3b6f16391450f2ca7bb19424a9eb398f69f8ef2195c821d3c72105248889cfb6efd317a3012d94c9564cb6b5794ba050fcc0902364eaa [kubernetes-node-windows-amd64.tar.gz](https://dl.k8s.io/v1.27.0-alpha.1/kubernetes-node-windows-amd64.tar.gz) | 68b2d73b7bf98445498e46188d39fde06acfee946ac9075f064f39bc7ea658cfcb82498ffe06bc2a2272ad5c105cac21d4929de47038e0ce95e07e3cd9d519ad ### Container Images All container images are available as manifest lists and support the described architectures. It is also possible to pull a specific architecture directly by adding the "-$ARCH" suffix to the container image name. name | architectures ---- | ------------- [registry.k8s.io/conformance:v1.27.0-alpha.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/conformance-s390x) [registry.k8s.io/kube-apiserver:v1.27.0-alpha.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-apiserver-s390x) [registry.k8s.io/kube-controller-manager:v1.27.0-alpha.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-controller-manager-s390x) [registry.k8s.io/kube-proxy:v1.27.0-alpha.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-proxy-s390x) [registry.k8s.io/kube-scheduler:v1.27.0-alpha.1](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler) | [amd64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-amd64), [arm](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm), [arm64](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-arm64), [ppc64le](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-ppc64le), [s390x](https://console.cloud.google.com/gcr/images/k8s-artifacts-prod/us/kube-scheduler-s390x) ## Changelog since v1.26.0 ## Changes by Kind ### Deprecation - Add warnings to the Services API. Kubernetes now warns for Services in the case of: - IPv4 addresses with leading zeros - IPv6 address in non-canonical format (RFC 5952) ([#114505](https://github.com/kubernetes/kubernetes/pull/114505), [@aojea](https://github.com/aojea)) [SIG Network] - Support for the alpha seccomp annotations `seccomp.security.alpha.kubernetes.io/pod` and `container.seccomp.security.alpha.kubernetes.io`, deprecated since v1.19, has been completely removed. The seccomp fields are no longer auto-populated when pods with seccomp annotations are created. Pods should use the corresponding pod or container `securityContext.seccompProfile` field instead. ([#114947](https://github.com/kubernetes/kubernetes/pull/114947), [@saschagrunert](https://github.com/saschagrunert)) ### API Change - A terminating pod on a node that is not caused by preemption won't prevent kube-scheduler from preempting pods on that node - Rename 'PreemptionByKubeScheduler' to 'PreemptionByScheduler' ([#114623](https://github.com/kubernetes/kubernetes/pull/114623), [@Huang-Wei](https://github.com/Huang-Wei)) [SIG Scheduling] - Added new option to the InterPodAffinity scheduler plugin to ignore existing pods` preferred inter-pod affinities if the incoming pod has no preferred inter-pod affinities. This option can be used as an optimization for higher scheduling throughput (at the cost of an occasional pod being scheduled non-optimally/violating existing pods' preferred inter-pod affinities). To enable this scheduler option, set the InterPodAffinity scheduler plugin arg "ignorePreferredTermsOfExistingPods: true". ([#114393](https://github.com/kubernetes/kubernetes/pull/114393), [@danielvegamyhre](https://github.com/danielvegamyhre)) [SIG API Machinery and Scheduling] - Added warnings about workload resources (Pods, ReplicaSets, Deployments, Jobs, CronJobs, or ReplicationControllers) whose names are not valid DNS labels. ([#114412](https://github.com/kubernetes/kubernetes/pull/114412), [@thockin](https://github.com/thockin)) [SIG API Machinery and Apps] - K8s.io/component-base/logs: usage of the pflag values in a normal Go flag set led to panics when printing the help message ([#114680](https://github.com/kubernetes/kubernetes/pull/114680), [@pohly](https://github.com/pohly)) [SIG Instrumentation] - Kube-proxy, kube-scheduler and kubelet have HTTP APIs for changing the logging verbosity at runtime. This now also works for JSON output. ([#114609](https://github.com/kubernetes/kubernetes/pull/114609), [@pohly](https://github.com/pohly)) [SIG API Machinery, Architecture, Cloud Provider, Instrumentation and Testing] - Kubeadm: explicitly set `priority` for static pods with `priorityClassName: system-node-critical` ([#114338](https://github.com/kubernetes/kubernetes/pull/114338), [@champtar](https://github.com/champtar)) [SIG Cluster Lifecycle] - Kubelet: migrate "--container-runtime-endpoint" and "--image-service-endpoint" to kubelet config ([#112136](https://github.com/kubernetes/kubernetes/pull/112136), [@pacoxu](https://github.com/pacoxu)) [SIG API Machinery, Node and Scalability] - Kubernetes components that perform leader election now only support using Leases for this. ([#114055](https://github.com/kubernetes/kubernetes/pull/114055), [@aimuz](https://github.com/aimuz)) [SIG API Machinery, Cloud Provider and Scheduling] - StatefulSet names must be DNS labels, rather than subdomains. Any StatefulSet which took advantage of subdomain validation (by having dots in the name) can't possibly have worked, because we eventually set `pod.spec.hostname` from the StatefulSetName, and that is validated as a DNS label. ([#114172](https://github.com/kubernetes/kubernetes/pull/114172), [@thockin](https://github.com/thockin)) [SIG Apps] - The following feature gates for volume expansion GA features have been removed and must no longer be referenced in `--feature-gates` flags: ExpandCSIVolumes, ExpandInUsePersistentVolumes, ExpandPersistentVolumes ([#113942](https://github.com/kubernetes/kubernetes/pull/113942), [@mengjiao-liu](https://github.com/mengjiao-liu)) [SIG API Machinery, Apps and Testing] - The list-type of the alpha resourceClaims field introduced to Pods in 1.26.0 was modified from "set" to "map", resolving an incompatibility with use of this schema in CustomResourceDefinitions and with server-side apply. ([#114585](https://github.com/kubernetes/kubernetes/pull/114585), [@JoelSpeed](https://github.com/JoelSpeed)) [SIG API Machinery] ### Feature - Graduated the `LegacyServiceAccountTokenTracking` feature gate to Beta. The usage of auto-generated secret-based service account token now produces warnings by default, and relevant Secrets are labeled with a last-used timestamp (label key `kubernetes.io/legacy-token-last-used`). ([#114523](https://github.com/kubernetes/kubernetes/pull/114523), [@zshihang](https://github.com/zshihang)) [SIG API Machinery and Auth] - Kube-proxy accepts the ContextualLogging, LoggingAlphaOptions, LoggingBetaOptions feature gates. ([#115233](https://github.com/kubernetes/kubernetes/pull/115233), [@pohly](https://github.com/pohly)) [SIG Instrumentation and Network] - Kube-up now includes CoreDNS version v1.9.3 ([#114279](https://github.com/kubernetes/kubernetes/pull/114279), [@pacoxu](https://github.com/pacoxu)) [SIG Cloud Provider and Cluster Lifecycle] - Kubeadm: add the experimental (alpha) feature gate EtcdLearnerMode that allows etcd members to be joined as learner and only then promoted as voting members ([#113318](https://github.com/kubernetes/kubernetes/pull/113318), [@pacoxu](https://github.com/pacoxu)) [SIG Cluster Lifecycle] - Kubectl will display SeccompProfile for pods, containers and ephemeral containers, if values were set. ([#113284](https://github.com/kubernetes/kubernetes/pull/113284), [@williamyeh](https://github.com/williamyeh)) [SIG CLI and Security] - Kubectl: add e2e test for default container annotation ([#115046](https://github.com/kubernetes/kubernetes/pull/115046), [@pacoxu](https://github.com/pacoxu)) [SIG Architecture, CLI and Testing] - Kubelet TCP and HTTP probes are more effective using networking resources: conntrack entries, sockets, ... This is achieved by reducing the TIME-WAIT state of the connection to 1 second, instead of the defaults 60 seconds. This allows kubelet to free the socket, and free conntrack entry and ephemeral port associated. ([#115143](https://github.com/kubernetes/kubernetes/pull/115143), [@aojea](https://github.com/aojea)) [SIG Network and Node] - Kubernetes is now built with Go 1.19.5 ([#115010](https://github.com/kubernetes/kubernetes/pull/115010), [@cpanato](https://github.com/cpanato)) [SIG Release and Testing] - Make `kubectl-convert` binary linking static (also affects the deb and rpm packages). ([#114228](https://github.com/kubernetes/kubernetes/pull/114228), [@saschagrunert](https://github.com/saschagrunert)) [SIG Release] - New metrics `cidrset_cidrs_max_total` and `multicidrset_cidrs_max_total` expose the max number of CIDRs that can be allocated. ([#112260](https://github.com/kubernetes/kubernetes/pull/112260), [@aryan9600](https://github.com/aryan9600)) [SIG Apps, Instrumentation and Network] - Profiling can now be served on a unix-domain socket by using the `--profiling-path` option (when profiling is enabled) for security purposes. ([#114191](https://github.com/kubernetes/kubernetes/pull/114191), [@apelisse](https://github.com/apelisse)) [SIG API Machinery] - Scheduler doesn't run plugin's Filter method when its PreFilter method returned a Skip status. In other words, your PreFilter/Filter plugin can return a Skip status in PreFilter if the plugin does nothing in Filter for that Pod. Scheduler skips NodeAffinity Filter plugin when NodeAffinity Filter plugin has nothing to do with a Pod. It may affect some metrics values related to the NodeAffinity Filter plugin. ([#114125](https://github.com/kubernetes/kubernetes/pull/114125), [@sanposhiho](https://github.com/sanposhiho)) [SIG Scheduling, Storage and Testing] - Scheduler skips InterPodAffinity Filter plugin when InterPodAffinity Filter plugin has nothing to do with a Pod. It may affect some metrics values related to the InterPodAffinity Filter plugin. ([#114889](https://github.com/kubernetes/kubernetes/pull/114889), [@sanposhiho](https://github.com/sanposhiho)) [SIG Scheduling and Testing] - Scheduler volumebinding: leverage PreFilterResult to reduce down to only eligible node(s) for pod with bound claim(s) to local PersistentVolume(s) ([#109877](https://github.com/kubernetes/kubernetes/pull/109877), [@yibozhuang](https://github.com/yibozhuang)) [SIG Scheduling, Storage and Testing] - The MinDomainsInPodTopologySpread feature gate is enabled by default as a Beta feature in 1.27. ([#114445](https://github.com/kubernetes/kubernetes/pull/114445), [@mengjiao-liu](https://github.com/mengjiao-liu)) [SIG Scheduling] - The `AdvancedAuditing` feature gate was locked to _true_ in v1.27, and will be removed completely in v1.28 ([#115163](https://github.com/kubernetes/kubernetes/pull/115163), [@SataQiu](https://github.com/SataQiu)) [SIG API Machinery] - Updated cAdvisor to v0.47.0 ([#114883](https://github.com/kubernetes/kubernetes/pull/114883), [@bobbypage](https://github.com/bobbypage)) [SIG Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node and Storage] - Use HorizontalPodAutoscaler v2 for kubectl ([#114886](https://github.com/kubernetes/kubernetes/pull/114886), [@a7i](https://github.com/a7i)) [SIG CLI] - Verify that the key matches the cert ([#113581](https://github.com/kubernetes/kubernetes/pull/113581), [@aimuz](https://github.com/aimuz)) [SIG Apps] - When any scheduler plugin returns an `unschedulableAndUnresolvable` status in `PostFilter`, the scheduling cycle terminates immediately for that Pod. ([#114699](https://github.com/kubernetes/kubernetes/pull/114699), [@kerthcet](https://github.com/kerthcet)) [SIG Scheduling and Testing] ### Documentation - Error message for Pods with requests exceeding limits will have a limit value printed. ([#112925](https://github.com/kubernetes/kubernetes/pull/112925), [@SergeyKanzhelev](https://github.com/SergeyKanzhelev)) [SIG Apps and Node] ### Failing Test - Deflake a preemption test that may patch Nodes incorrectly. ([#114350](https://github.com/kubernetes/kubernetes/pull/114350), [@Huang-Wei](https://github.com/Huang-Wei)) [SIG Scheduling and Testing] ### Bug or Regression - Adding (dry run) and (server dry run) suffixes to kubectl scale command when dry-run is passed ([#114252](https://github.com/kubernetes/kubernetes/pull/114252), [@ardaguclu](https://github.com/ardaguclu)) [SIG CLI and Testing] - Change the error message to "cannot exec into multiple objects at a time" when file passed to kubectl exec contains multiple resources ([#114249](https://github.com/kubernetes/kubernetes/pull/114249), [@ardaguclu](https://github.com/ardaguclu)) [SIG CLI and Testing] - Changing the error message of kubectl rollout restart when subsequent kubectl rollout restart commands are executed within a second ([#113040](https://github.com/kubernetes/kubernetes/pull/113040), [@ardaguclu](https://github.com/ardaguclu)) [SIG CLI] - Client-go: fixes potential data races retrying requests using a custom io.Reader body; with this fix, only requests with no body or with string / []byte / runtime.Object bodies can be retried ([#113933](https://github.com/kubernetes/kubernetes/pull/113933), [@liggitt](https://github.com/liggitt)) [SIG API Machinery] - Do not add DisruptionTarget condition by PodGC for pods which are in terminal phase ([#115056](https://github.com/kubernetes/kubernetes/pull/115056), [@mimowo](https://github.com/mimowo)) [SIG Apps and Testing] - Do not include preemptor pod metadata in the event message ([#114923](https://github.com/kubernetes/kubernetes/pull/114923), [@mimowo](https://github.com/mimowo)) [SIG Scheduling] - Do not include preemptor pod metadata in the message of DisruptionTarget condition ([#114914](https://github.com/kubernetes/kubernetes/pull/114914), [@mimowo](https://github.com/mimowo)) [SIG Scheduling] - Do not include scheduler name in the preemption event message ([#114980](https://github.com/kubernetes/kubernetes/pull/114980), [@mimowo](https://github.com/mimowo)) [SIG Scheduling] - Don't create endpoints for Service of type ExternalName. ([#114814](https://github.com/kubernetes/kubernetes/pull/114814), [@panslava](https://github.com/panslava)) [SIG Apps, Network and Testing] - Fail CRI connection if service or image endpoint is throwing any error on kubelet startup. ([#115102](https://github.com/kubernetes/kubernetes/pull/115102), [@saschagrunert](https://github.com/saschagrunert)) [SIG Node] - Failed pods associated with a job with `parallelism = 1` are recreated by the job controller honoring exponential backoff delay again. However, for jobs with `parallelism > 1`, pods might be created without exponential backoff delay. ([#114516](https://github.com/kubernetes/kubernetes/pull/114516), [@nikhita](https://github.com/nikhita)) [SIG Apps and Testing] - Fix SELinux label for host path volumes created by host path provisioner ([#112021](https://github.com/kubernetes/kubernetes/pull/112021), [@mrunalp](https://github.com/mrunalp)) [SIG Node and Storage] - Fix a bug on the endpointslice mirroring controller that generated multiple slices in some cases for custom endpoints in non canonical format ([#114155](https://github.com/kubernetes/kubernetes/pull/114155), [@aojea](https://github.com/aojea)) [SIG Apps, Network and Testing] - Fix a bug where events/v1 Events with similar event type and reporting instance were not aggregated by client-go. ([#112365](https://github.com/kubernetes/kubernetes/pull/112365), [@dgrisonnet](https://github.com/dgrisonnet)) [SIG API Machinery and Instrumentation] - Fix a bug where when emitting similar Events consecutively, some were rejected by the apiserver. ([#114237](https://github.com/kubernetes/kubernetes/pull/114237), [@dgrisonnet](https://github.com/dgrisonnet)) [SIG API Machinery] - Fix a data race when emitting similar Events consecutively ([#114236](https://github.com/kubernetes/kubernetes/pull/114236), [@dgrisonnet](https://github.com/dgrisonnet)) [SIG API Machinery] - Fix a regression that the scheduler always goes through all Filter plugins. ([#114518](https://github.com/kubernetes/kubernetes/pull/114518), [@Huang-Wei](https://github.com/Huang-Wei)) [SIG Scheduling] - Fix bug in CRD Validation Rules (beta) and ValidatingAdmissionPolicy (alpha) where all admission requests could result in `internal error: runtime error: index out of range [3] with length 3 evaluating rule: ` under certain circumstances. ([#114857](https://github.com/kubernetes/kubernetes/pull/114857), [@jpbetz](https://github.com/jpbetz)) [SIG API Machinery, Auth and Cloud Provider] - Fix clearing of rate-limiter for the queue of checks for cleaning stale pod disruption conditions. The bug could result in the PDB synchronization updates firing too often or the pod disruption cleanups taking too long to happen. ([#114770](https://github.com/kubernetes/kubernetes/pull/114770), [@mimowo](https://github.com/mimowo)) [SIG Apps] - Fix: Route controller should update routes with NodeIP changed ([#108095](https://github.com/kubernetes/kubernetes/pull/108095), [@lzhecheng](https://github.com/lzhecheng)) [SIG Cloud Provider and Network] - Fixed CSI PersistentVolumes to allow Secrets names longer than 63 characters. ([#114776](https://github.com/kubernetes/kubernetes/pull/114776), [@jsafrane](https://github.com/jsafrane)) [SIG Apps] - Fixed DaemonSet to update the status even if it fails to create a pod. ([#113787](https://github.com/kubernetes/kubernetes/pull/113787), [@gjkim42](https://github.com/gjkim42)) [SIG Apps and Testing] - Fixed StatefulSetAutoDeletePVC feature when OwnerReferencesPermissionEnforcement admission plugin is enabled. ([#114116](https://github.com/kubernetes/kubernetes/pull/114116), [@jsafrane](https://github.com/jsafrane)) [SIG Apps, Auth and Storage] - Fixed bug in reflector that couldn't recover from "Too large resource version" errors with API servers before 1.17.0 ([#115093](https://github.com/kubernetes/kubernetes/pull/115093), [@xuzhenglun](https://github.com/xuzhenglun)) [SIG API Machinery] - Fixed file permission issues that happened during update of Secret/ConfigMap/projected volume when fsGroup is used. The problem caused a race condition where application gets intermittent permission denied error when reading files that were just updated, before the correct permissions were applied. ([#114464](https://github.com/kubernetes/kubernetes/pull/114464), [@tsaarni](https://github.com/tsaarni)) [SIG Storage] - Fixes panic validating custom resource definition schemas that set `multipleOf` to 0 ([#114869](https://github.com/kubernetes/kubernetes/pull/114869), [@liggitt](https://github.com/liggitt)) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node and Storage] - Fixes stuck apiserver if an aggregated apiservice returned 304 Not Modified for aggregated discovery information ([#114459](https://github.com/kubernetes/kubernetes/pull/114459), [@alexzielenski](https://github.com/alexzielenski)) [SIG API Machinery] - Fixing issue in Winkernel Proxier - Unexpected active TCP connection drops while horizontally scaling the endpoints for a LoadBalancer Service with Internal Traffic Policy: Local ([#113742](https://github.com/kubernetes/kubernetes/pull/113742), [@princepereira](https://github.com/princepereira)) [SIG Network and Windows] - Fixing issue on Windows when calculating cpu limits on nodes with more than 64 logical processors ([#114231](https://github.com/kubernetes/kubernetes/pull/114231), [@mweibel](https://github.com/mweibel)) [SIG Node and Windows] - Fixing issue with Winkernel Proxier - No ingress load balancer rules with endpoints to support load balancing when all the endpoints are terminating. ([#113776](https://github.com/kubernetes/kubernetes/pull/113776), [@princepereira](https://github.com/princepereira)) [SIG Network, Testing and Windows] - Hide .metadata.managedFields when describing CRs ([#114584](https://github.com/kubernetes/kubernetes/pull/114584), [@soltysh](https://github.com/soltysh)) [SIG CLI] - IPVS: Any ipvs scheduler can now be configured. If a un-usable scheduler is configured `kube-proxy` will re-start and the logs must be checked (same as before but different log printouts). ([#114878](https://github.com/kubernetes/kubernetes/pull/114878), [@uablrek](https://github.com/uablrek)) [SIG Network] - If a user attempts to add an ephemeral container to a static pod, they will get a visible validation error. ([#114086](https://github.com/kubernetes/kubernetes/pull/114086), [@xmcqueen](https://github.com/xmcqueen)) [SIG Apps and Node] - Kube-apiserver: removed N^2 behavior loading webhook configurations. ([#114794](https://github.com/kubernetes/kubernetes/pull/114794), [@lavalamp](https://github.com/lavalamp)) [SIG API Machinery, Architecture, CLI, Cloud Provider and Node] - Kube-controller-manager will not run nodeipam controller when allocator type is CloudAllocator and the cloud provider is not enabled. ([#114596](https://github.com/kubernetes/kubernetes/pull/114596), [@andrewsykim](https://github.com/andrewsykim)) [SIG Cloud Provider] - Kube-proxy with proxy-mode=ipvs can be used with statically linked kernels. The reseved IPv4 range TEST-NET-2 in rfc5737 MUST NOT be used for ClusterIP or loadBalancerIP since address 198.51.100.0 is used for probing. ([#114669](https://github.com/kubernetes/kubernetes/pull/114669), [@uablrek](https://github.com/uablrek)) [SIG Network] - Kubeadm: fix the bug that kubeadm always do CRI detection even if it is not required by a phase subcommand ([#114455](https://github.com/kubernetes/kubernetes/pull/114455), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle] - Kubeadm: improve retries when updating node information, in case kube-apiserver is temporarily unavailable ([#114176](https://github.com/kubernetes/kubernetes/pull/114176), [@QuantumEnergyE](https://github.com/QuantumEnergyE)) [SIG Cluster Lifecycle] - Kubeadm: respect user provided kubeconfig during discovery process ([#113998](https://github.com/kubernetes/kubernetes/pull/113998), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle] - Kubectl port-forward now exits with exit code 1 when remote connection is lost ([#114460](https://github.com/kubernetes/kubernetes/pull/114460), [@brianpursley](https://github.com/brianpursley)) [SIG API Machinery] - Kubectl: use label selector for filtering out resources when pruning for kubectl diff ([#114863](https://github.com/kubernetes/kubernetes/pull/114863), [@danlenar](https://github.com/danlenar)) [SIG CLI and Testing] - LabelSelectors specified in topologySpreadConstraints are now validated to ensure that pod is scheduled as expected. Existing pods with invalid LabelSelectors can be updated, but new pods are required to specify valid LabelSelectors. ([#111802](https://github.com/kubernetes/kubernetes/pull/111802), [@maaoBit](https://github.com/maaoBit)) [SIG Apps] - Optimizing loadbalancer creation with the help of attribute Internal Traffic Policy: Local ([#114407](https://github.com/kubernetes/kubernetes/pull/114407), [@princepereira](https://github.com/princepereira)) [SIG Network] - Relax API validation for usage key encipherment and kubelet uses requested usages accordingly ([#111660](https://github.com/kubernetes/kubernetes/pull/111660), [@pacoxu](https://github.com/pacoxu)) [SIG API Machinery, Apps, Auth and Node] - Shared informers now correctly propagate whether they are synced or not. Individual informer handlers may now check if they are synced or not (new HasSynced method). Library support is added to assist controllers in tracking whether their own work is completed for items in the initial list (AsyncTracker). ([#113985](https://github.com/kubernetes/kubernetes/pull/113985), [@lavalamp](https://github.com/lavalamp)) [SIG API Machinery, Apps, Auth, Network, Node and Testing] - Statefulset status will be consistent on API errors ([#113834](https://github.com/kubernetes/kubernetes/pull/113834), [@atiratree](https://github.com/atiratree)) [SIG Apps] - Total test spec is now available by `ProgressReporter`, it will be reported before test suite got executed. ([#114417](https://github.com/kubernetes/kubernetes/pull/114417), [@chendave](https://github.com/chendave)) [SIG Architecture, Auth, CLI, Cloud Provider, Instrumentation, Node and Testing] - TryUnmount should respect `mounter.withSafeNotMountedBehavior` ([#114736](https://github.com/kubernetes/kubernetes/pull/114736), [@andyzhangx](https://github.com/andyzhangx)) [SIG Storage] - When describing deployments, `OldReplicaSets` now always shows all replicasets controlled the deployment, not just those that still have replicas available. ([#113083](https://github.com/kubernetes/kubernetes/pull/113083), [@llorllale](https://github.com/llorllale)) [SIG CLI] ### Other (Cleanup or Flake) - Callers of wait.ExponentialBackoffWithContext must pass a ConditionWithContextFunc to be consistent with the signature and avoid creating a duplicate context. If your condition does not need a context you can use the `ConditionFunc.WithContext()` helper to ignore the context, or use ExponentialBackoff directly. ([#115113](https://github.com/kubernetes/kubernetes/pull/115113), [@smarterclayton](https://github.com/smarterclayton)) [SIG API Machinery, Storage and Testing] - Fix incorrect log information ([#110723](https://github.com/kubernetes/kubernetes/pull/110723), [@yangjunmyfm192085](https://github.com/yangjunmyfm192085)) [SIG Network] - Improved misleading message, in case of no metrics received for the HPA controlled pods. ([#114740](https://github.com/kubernetes/kubernetes/pull/114740), [@kushagra98](https://github.com/kushagra98)) [SIG Apps and Autoscaling] - Kubeadm: remove the deprecated v1beta2 API. kubeadm 1.26's "config migrate" command can be used to migrate a v1beta2 configuration file to v1beta3. ([#114540](https://github.com/kubernetes/kubernetes/pull/114540), [@pacoxu](https://github.com/pacoxu)) [SIG Cluster Lifecycle] - Remove unused rule for `nodes/spec` from `ClusterRole system:kubelet-api-admin` ([#113267](https://github.com/kubernetes/kubernetes/pull/113267), [@hoskeri](https://github.com/hoskeri)) [SIG Auth and Cloud Provider] - Renamed API server identity Lease labels to use the key `apiserver.kubernetes.io/identity` ([#114586](https://github.com/kubernetes/kubernetes/pull/114586), [@andrewsykim](https://github.com/andrewsykim)) [SIG API Machinery, Apps, Cloud Provider and Testing] - The CSIMigrationAzureFile feature gate (for the feature which graduated to GA in v1.26) is now unconditionally enabled and will be removed in v1.28. ([#114953](https://github.com/kubernetes/kubernetes/pull/114953), [@enj](https://github.com/enj)) [SIG Storage] - The WaitFor and WaitForWithContext functions in the wait package have been marked private. Callers should use the equivalent Poll* method with a zero duration interval. ([#115116](https://github.com/kubernetes/kubernetes/pull/115116), [@smarterclayton](https://github.com/smarterclayton)) [SIG API Machinery] - The feature gates `CSIInlineVolume`, `CSIMigration`, `DaemonSetUpdateSurge`, `EphemeralContainers`, `IdentifyPodOS`, `LocalStorageCapacityIsolation`, `NetworkPolicyEndPort` and `StatefulSetMinReadySeconds` that graduated to GA in v1.25 and were unconditionally enabled have been removed in v1.27 ([#114410](https://github.com/kubernetes/kubernetes/pull/114410), [@SataQiu](https://github.com/SataQiu)) [SIG Node] - This flag `master-service-namespace` will be removed in v1.27. ([#114446](https://github.com/kubernetes/kubernetes/pull/114446), [@lengrongfu](https://github.com/lengrongfu)) [SIG API Machinery] - Wait.ContextForChannel() now implements the context.Context interface and does not return a cancellation function. ([#115140](https://github.com/kubernetes/kubernetes/pull/115140), [@smarterclayton](https://github.com/smarterclayton)) [SIG API Machinery and Cloud Provider] ## Dependencies ### Added - github.com/a8m/tree: [10a5fd5](https://github.com/a8m/tree/tree/10a5fd5) - github.com/dougm/pretty: [2ee9d74](https://github.com/dougm/pretty/tree/2ee9d74) - github.com/rasky/go-xdr: [4930550](https://github.com/rasky/go-xdr/tree/4930550) - github.com/vmware/vmw-guestinfo: [25eff15](https://github.com/vmware/vmw-guestinfo/tree/25eff15) ### Changed - github.com/Microsoft/hcsshim: [v0.8.22 → v0.8.25](https://github.com/Microsoft/hcsshim/compare/v0.8.22...v0.8.25) - github.com/aws/aws-sdk-go: [v1.44.116 → v1.44.147](https://github.com/aws/aws-sdk-go/compare/v1.44.116...v1.44.147) - github.com/coredns/corefile-migration: [v1.0.17 → v1.0.18](https://github.com/coredns/corefile-migration/compare/v1.0.17...v1.0.18) - github.com/creack/pty: [v1.1.11 → v1.1.18](https://github.com/creack/pty/compare/v1.1.11...v1.1.18) - github.com/docker/docker: [v20.10.18+incompatible → v20.10.21+incompatible](https://github.com/docker/docker/compare/v20.10.18...v20.10.21) - github.com/go-openapi/jsonpointer: [v0.19.5 → v0.19.6](https://github.com/go-openapi/jsonpointer/compare/v0.19.5...v0.19.6) - github.com/go-openapi/jsonreference: [v0.20.0 → v0.20.1](https://github.com/go-openapi/jsonreference/compare/v0.20.0...v0.20.1) - github.com/go-openapi/swag: [v0.19.14 → v0.22.3](https://github.com/go-openapi/swag/compare/v0.19.14...v0.22.3) - github.com/google/cadvisor: [v0.46.0 → v0.47.1](https://github.com/google/cadvisor/compare/v0.46.0...v0.47.1) - github.com/google/cel-go: [v0.12.5 → v0.12.6](https://github.com/google/cel-go/compare/v0.12.5...v0.12.6) - github.com/google/uuid: [v1.1.2 → v1.3.0](https://github.com/google/uuid/compare/v1.1.2...v1.3.0) - github.com/kr/pretty: [v0.2.1 → v0.3.0](https://github.com/kr/pretty/compare/v0.2.1...v0.3.0) - github.com/mailru/easyjson: [v0.7.6 → v0.7.7](https://github.com/mailru/easyjson/compare/v0.7.6...v0.7.7) - github.com/moby/ipvs: [v1.0.1 → v1.1.0](https://github.com/moby/ipvs/compare/v1.0.1...v1.1.0) - github.com/moby/term: [39b0c02 → 1aeaba8](https://github.com/moby/term/compare/39b0c02...1aeaba8) - github.com/onsi/ginkgo/v2: [v2.4.0 → v2.7.0](https://github.com/onsi/ginkgo/v2/compare/v2.4.0...v2.7.0) - github.com/onsi/gomega: [v1.23.0 → v1.24.2](https://github.com/onsi/gomega/compare/v1.23.0...v1.24.2) - github.com/opencontainers/runtime-spec: [1c3f411 → 494a5a6](https://github.com/opencontainers/runtime-spec/compare/1c3f411...494a5a6) - github.com/rogpeppe/go-internal: [v1.3.0 → v1.9.0](https://github.com/rogpeppe/go-internal/compare/v1.3.0...v1.9.0) - github.com/sirupsen/logrus: [v1.8.1 → v1.9.0](https://github.com/sirupsen/logrus/compare/v1.8.1...v1.9.0) - github.com/stretchr/objx: [v0.4.0 → v0.5.0](https://github.com/stretchr/objx/compare/v0.4.0...v0.5.0) - github.com/stretchr/testify: [v1.8.0 → v1.8.1](https://github.com/stretchr/testify/compare/v1.8.0...v1.8.1) - github.com/tmc/grpc-websocket-proxy: [e5319fd → 673ab2c](https://github.com/tmc/grpc-websocket-proxy/compare/e5319fd...673ab2c) - github.com/vishvananda/netns: [db3c7e5 → v0.0.2](https://github.com/vishvananda/netns/compare/db3c7e5...v0.0.2) - github.com/vmware/govmomi: [v0.20.3 → v0.30.0](https://github.com/vmware/govmomi/compare/v0.20.3...v0.30.0) - golang.org/x/mod: v0.6.0 → v0.7.0 - golang.org/x/net: 1e63c2f → v0.4.0 - golang.org/x/sync: 886fb93 → v0.1.0 - golang.org/x/tools: v0.2.0 → v0.4.0 - golang.org/x/xerrors: 5ec99f8 → 04be3eb - google.golang.org/grpc: v1.49.0 → v1.51.0 - gopkg.in/check.v1: 8fa4692 → 10cb982 - k8s.io/kube-openapi: 172d655 → 3758b55 - sigs.k8s.io/apiserver-network-proxy/konnectivity-client: v0.0.33 → v0.1.1 ### Removed - github.com/elazarl/goproxy: [947c36d](https://github.com/elazarl/goproxy/tree/947c36d) - github.com/mindprince/gonvml: [9ebdce4](https://github.com/mindprince/gonvml/tree/9ebdce4)