NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI · #1148 most downloaded on PyPI
Kubernetes Asynchronous Python Client
Last release 3 months ago
04 Jun 2026
Release timing varies
gaps range from 8 days to 5 months
Nearly every release is documented
notes for 57 of 59 stable releases
3 versions withdrawn
withdrawn after publishing
8 years old
74 releases · first in 2018
The annotation batch.kubernetes.io/job-tracking is now deprecated. The control plane will ignore it and stop adding it for new Jobs in v1.27.' (kubern…
Kubernetes API Version: v1.26.9
The list-type of the alpha resourceClaims field introduced to Pods in 1.26.0 was modified from "set" to "map", resolving an incompatibility with use of this schema in CustomResourceDefinitions and with server-side apply. (kubernetes/kubernetes#114617, @JoelSpeed) [SIG API Machinery]
'A new preEnqueue extension point was added to scheduler's component config
v1beta2/v1beta3/v1.'
(kubernetes/kubernetes#113275, @Huang-Wei)
'Added a ResourceClaim API (in the resource.k8s.io/v1alpha1 API group and
behind the DynamicResourceAllocation feature gate).
The new API is now more flexible than the existing Device Plugins feature of Kubernetes because it
allows Pods to request (claim) special kinds of resources, which can be available at node level, cluster
level, or following any other model you implement.' (kubernetes/kubernetes#111023, @pohly)
'Container preStop and postStart lifecycle handlers using httpGet now
honor the specified scheme and headers fields. This enables setting custom
headers and changing the scheme to HTTPS, consistent with container
startup/readiness/liveness probe capabilities. Lifecycle handlers configured
with scheme: HTTPS that encounter errors indicating the endpoint is actually
using HTTP fall back to making the request over HTTP for compatibility with
previous releases. When this happens, a LifecycleHTTPFallback event is recorded
in the namespace of the pod and a kubelet_lifecycle_handler_http_fallbacks_total
metric in the kubelet is incremented. Cluster administrators can opt out of the
expanded lifecycle handler capabilities by setting
--feature-gates=ConsistentHTTPGetHandlers=false in kubelet.'
(kubernetes/kubernetes#86139, @jasimmons)
'Graduated JobTrackingWithFinalizers to stable.
Jobs created before the feature was enabled are still tracked without finalizers.
Jobs tracked with finalizers have the annotation batch.kubernetes.io/job-tracking.
If the annotation is present and the user attempts to remove it, the control plane adds it back.
The annotation batch.kubernetes.io/job-tracking is now deprecated.
The control plane will ignore it and stop adding it for new Jobs in v1.27.' (kubernetes/kubernetes#113510, @alculquicondor)
'Kubelet added the following Pod failure conditions:
DisruptionTarget (graceful node shutdown, node pressure eviction)' (kubernetes/kubernetes#112360, @mimowo)'Priority and Fairness has introduced a new feature called borrowing that allows an API priority level
to borrow a number of seats from other priority level(s). As a cluster operator, you can enable borrowing
for a certain priority level configuration object via the two newly introduced fields lendablePercent, and
borrowingLimitPercent located under the .spec.limited field of the designated priority level.
This change added the following metrics:
apiserver_flowcontrol_nominal_limit_seats: Nominal number of execution seats configured for each priority levelapiserver_flowcontrol_lower_limit_seats: Configured lower bound on number of execution seats available to each priority levelapiserver_flowcontrol_upper_limit_seats: Configured upper bound on number of execution seats available to each priority levelapiserver_flowcontrol_demand_seats: Observations, at the end of every nanosecond, of (the number of seats each priority level could use) / (nominal number of seats for that level)apiserver_flowcontrol_demand_seats_high_watermark: High watermark, over last adjustment period, of demand_seatsapiserver_flowcontrol_demand_seats_average: Time-weighted average, over last adjustment period, of demand_seatsapiserver_flowcontrol_demand_seats_stdev: Time-weighted standard deviation, over last adjustment period, of demand_seatsapiserver_flowcontrol_demand_seats_smoothed: Smoothed seat demandsapiserver_flowcontrol_target_seats: Seat allocation targetsapiserver_flowcontrol_seat_fair_frac: Fair fraction of server's concurrency to allocate to each priority level that can use itapiserver_flowcontrol_current_limit_seats: current derived number of execution seats available to each priority level
The possibility of borrowing means that the old metric apiserver_flowcontrol_request_concurrency_limit can no longer mean both the configured concurrency limit and the enforced concurrency limit. Henceforth it means the configured concurrency limit.' (kubernetes/kubernetes#113485, @MikeSpreitzer)'NodeInclusionPolicy in podTopologySpread plugin is now enabled by default.'
(kubernetes/kubernetes#113500, @kerthcet)
'PodDisruptionBudget now adds an alpha spec.unhealthyPodEvictionPolicy field.
When the PDBUnhealthyPodEvictionPolicy feature-gate is enabled in kube-apiserver,
setting this field to "AlwaysAllow" allows pods to be evicted if they do not
have a ready condition, regardless of whether the PodDisruptionBudget is currently
healthy.'
(kubernetes/kubernetes#113375, @atiratree)
'metav1.LabelSelectors specified in API objects are now validated to ensure
they do not contain invalid label values that will error at time of use. Existing
invalid objects can be updated, but new objects are required to contain valid
label selectors.'
(kubernetes/kubernetes#113699, @liggitt)
Add percentageOfNodesToScore as a scheduler profile level parameter to API version v1. When a profile percentageOfNodesToScore is set, it will override global percentageOfNodesToScore. (kubernetes/kubernetes#112521, @yuanchen8911)
Add auth API to get self subject attributes (new selfsubjectreviews API is added).
The corresponding command for kubctl is provided - kubectl auth whoami. (kubernetes/kubernetes#111333, @nabokihms) [SIG API Machinery, Auth, CLI and Testing]
Added kubernetes_feature_enabled metric series to track whether each active feature gate is enabled. (kubernetes/kubernetes#112690, @logicalhan)
Added a --topology-manager-policy-options flag to the kubelet to support fine tuning the topology manager policies. The first policy option, prefer-closest-numa-nodes, allows these policies to favor sets of NUMA nodes with shorter distance between nodes when making admission decisions. (kubernetes/kubernetes#112914, @PiotrProkop)
Added a feature that allows a StatefulSet to start numbering replicas from an arbitrary non-negative ordinal, using the .spec.ordinals.start field. (kubernetes/kubernetes#112744, @pwschuurman)
Added a kube-proxy flag (--iptables-localhost-nodeports, default true) to allow disabling NodePort services on loopback addresses. Note: this only applies to iptables mode and ipv4. (kubernetes/kubernetes#108250, @cyclinder)
Added a new namespace alpha field to DataSourceRef field in PersistentVolumeClaim API. (kubernetes/kubernetes#113186, @ttakahashi21)
Aggregated discovery will be alpha and can be toggled with the AggregatedDiscoveryEndpoint feature flag. (kubernetes/kubernetes#113171, @Jefftree)
Clarified the CFS quota as 100ms in the code comments and set the minimum cpuCFSQuotaPeriod to 1ms to match Linux kernel expectations. (kubernetes/kubernetes#112123, @paskal)
Component-base: make the validation logic about LeaderElectionConfiguration consistent between component-base and client-go (kubernetes/kubernetes#111758, @SataQiu) [SIG API Machinery and Scheduling]
Deprecated the apiserver_request_slo_duration_seconds metric for v1.27 in favor of apiserver_request_sli_duration_seconds for naming consistency purposes with other SLI-specific metrics and to avoid any confusion between SLOs and SLIs. (kubernetes/kubernetes#112679, @dgrisonnet)
Enable the "Retriable and non-retriable pod failures for jobs" feature into beta. (kubernetes/kubernetes#113360, @mimowo)
Enabled kube-controller-manager to support '--concurrent-horizontal-pod-autoscaler-syncs' flag to set the number of horizontal pod autoscaler controller workers. (kubernetes/kubernetes#108501, @zroubalik)
Fixed spurious field is immutable errors validating updates to Event API objects via the events.k8s.io/v1 API. (kubernetes/kubernetes#112183, @liggitt)
Graduated ServiceInternalTrafficPolicy feature to GA. (kubernetes/kubernetes#113496, @avoltz)
In 'kube-proxy`: The "userspace" proxy mode (deprecated for over a year) is no longer supported on either Linux or Windows. Users should use "iptables" or "ipvs" on Linux, or "kernelspace" on Windows. (kubernetes/kubernetes#112133, @knabben)
Introduce v1beta3 for Priority and Fairness with the following changes to the API spec:
prioritylevelconfigurations. (kubernetes/kubernetes#112306, @tkashem)Introduced v1alpha1 API for validating admission policies, enabling extensible admission control via CEL expressions (KEP 3488: CEL for Admission Control). To use, enable the ValidatingAdmissionPolicy feature gate and the admissionregistration.k8s.io/v1alpha1 API via --runtime-config. (kubernetes/kubernetes#113314, @cici37)
KMS: added validation for duplicate kms config name when auto reload is enabled. If you enabled automatic reload of encryption configuration with API server flag --encryption-provider-config-automatic-reload, ensure all the KMS provider names (v1 and v2) in the encryption configuration are unique. (kubernetes/kubernetes#113697, @aramase)
Kubelet external Credential Provider feature is moved to GA. Credential Provider Plugin and Credential Provider Config APIs updated from v1beta1 to v1 with no API changes. (kubernetes/kubernetes#111616, @ndixita)
Legacy klog flags are no longer available. Only -v and -vmodule are still supported. (kubernetes/kubernetes#112120, @pohly) [SIG Architecture, CLI, Instrumentation, Node and Testing]
Moved MixedProtocolLBService from beta to GA. (kubernetes/kubernetes#112895, @janosi)
New Pod API field .spec.schedulingGates is introduced to enable users to control when to mark a Pod as scheduling ready. (kubernetes/kubernetes#113274, @Huang-Wei)
Protobuf serialization of metav1.MicroTime timestamps (used in Lease and Event API objects) has been corrected to truncate to microsecond precision, to match the documented behavior and JSON/YAML serialization. Any existing persisted data is truncated to microsecond when read from etcd. (kubernetes/kubernetes#111936, @haoruan)
Removed feature gates ServiceLoadBalancerClass and ServiceLBNodePortControl. These feature gates were enabled (and locked) since v1.24. (kubernetes/kubernetes#112577, @andrewsykim)
Reverted regression that prevented client-go latency metrics to be reported with a template URL to avoid label cardinality. (kubernetes/kubernetes#111752, @aanm)
The EndpointSliceTerminatingCondition feature gate was graduated to GA. The gate is now locked and will be removed in v1.28. (kubernetes/kubernetes#113351, @andrewsykim)
DynamicKubeletConfig feature gate has been removed from the API server.
Dynamic kubelet reconfiguration now can't be used even when older nodes are still
attempting to rely on it. This is aligned with the Kubernetes version skew policy.
(kubernetes/kubernetes#112643, @SergeyKanzhelev)
kubectl wait command with jsonpath flag will wait for target path until timeout.
(kubernetes/kubernetes#109525, @jonyhy96)
Add a ResourceClaim API (in the resource.k8s.io/v1alpha1 API group and
behind the DynamicResourceAllocation feature gate).
The new API is more flexible than the existing Device Plugins feature of Kubernetes because it
allows Pods to request (claim) special kinds of resources, which can be available at node level, cluster
level, or following any other model you implement. (kubernetes/kubernetes#111023, @pohly) [SIG API Machinery, Apps, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node, Release, Scheduling, Storage and Testing]
PodDisruptionBudget adds an alpha spec.unhealthyPodEvictionPolicy field. When the PDBUnhealthyPodEvictionPolicy feature-gate is enabled in kube-apiserver, setting this field to "AlwaysAllow" allows pods to be evicted if they do not have a ready condition, regardless of whether the PodDisruptionBudget is currently healthy. (kubernetes/kubernetes#113375, @atiratree) [SIG API Machinery, Apps, Auth and Testing]
A new preEnqueue extension point is added to scheduler's component config v1beta2/v1beta3/v1. (kubernetes/kubernetes#113275, @Huang-Wei) [SIG API Machinery, Apps, Instrumentation, Scheduling and Testing]
Add a new namespace alpha field to dataSourceRef field in PersistentVolumeClaim API. (kubernetes/kubernetes#113186, @ttakahashi21) [SIG API Machinery, Apps, Storage and Testing]
Add a kube-proxy flag (--iptables-localhost-nodeports, default true) to allow disabling NodePort services on loopback addresses. Note: this only applies to iptables mode and ipv4. (kubernetes/kubernetes#108250, @cyclinder) [SIG API Machinery, Cloud Provider, Network, Node, Scalability, Storage and Testing]
Added a --topology-manager-policy-options flag to the kubelet to support fine tuning the topology manager policies. The first policy option, prefer-closest-numa-nodes, allows these policies to favor sets of NUMA nodes with shorter distance between nodes when making admission decisions. (kubernetes/kubernetes#112914, @PiotrProkop) [SIG API Machinery and Node]
Added a feature that allows a StatefulSet to start numbering replicas from an arbitrary non-negative ordinal, using the .spec.ordinals.start field. (kubernetes/kubernetes#112744, @pwschuurman) [SIG API Machinery and Apps]
Deprecate the apiserver_request_slo_duration_seconds metric for v1.27 in favor of apiserver_request_sli_duration_seconds for naming consistency purposes with other SLI-specific metrics and to avoid any confusion between SLOs and SLIs. (kubernetes/kubernetes#112679, @dgrisonnet) [SIG API Machinery and Instrumentation]
Enable the "Retriable and non-retriable pod failures for jobs" feature into beta (kubernetes/kubernetes#113360, @mimowo) [SIG Apps, Auth, Node, Scheduling and Testing]
Graduate JobTrackingWithFinalizers to stable. Jobs created before the feature was enabled are still tracked without finalizers. Users can choose to migrate jobs to tracking with finalizers by adding the annotation batch.kubernetes.io/job-tracking. If the annotation was already present and the user attempts to remove it, the control plane adds the annotation back. (kubernetes/kubernetes#113510, @alculquicondor) [SIG API Machinery, Apps and Testing]
Graduate ServiceInternalTrafficPolicy feature to GA (kubernetes/kubernetes#113496, @avoltz) [SIG Apps and Network]
If you enabled automatic reload of encryption configuration with API server flag --encryption-provider-config-automatic-reload, ensure all the KMS provider names (v1 and v2) in the encryption configuration are unique. (kubernetes/kubernetes#113697, @aramase) [SIG API Machinery and Auth]
Introduce v1alpha1 API for validating admission policies, enabling extensible admission control via CEL expressions (KEP 3488: CEL for Admission Control). To use, enable the ValidatingAdmissionPolicy feature gate and the admissionregistration.k8s.io/v1alpha1 API via --runtime-config. (kubernetes/kubernetes#113314, @cici37) [SIG API Machinery, Auth, Cloud Provider and Testing]
Kubelet adds the following pod failure conditions:
Metav1.LabelSelectors specified in API objects are now validated to ensure they do not contain invalid label values that will error at time of use. Existing invalid objects can be updated, but new objects are required to contain valid label selectors. (kubernetes/kubernetes#113699, @liggitt) [SIG API Machinery, Apps, Auth, Network and Storage]
Moving MixedProtocolLBService from beta to GA (kubernetes/kubernetes#112895, @janosi) [SIG Apps, Network and Testing]
New Pod API field .spec.schedulingGates is introduced to enable users to control when to mark a Pod as scheduling ready. (kubernetes/kubernetes#113274, @Huang-Wei) [SIG Apps, Scheduling and Testing]
NodeInclusionPolicy in podTopologySpread plugin is enabled by default. (kubernetes/kubernetes#113500, @kerthcet) [SIG API Machinery, Apps, Scheduling and Testing]
Priority and Fairness has introduced a new feature called borrowing that allows an API priority level
to borrow a number of seats from other priority level(s). As a cluster operator, you can enable borrowing
for a certain priority level configuration object via the two newly introduced fields lendablePercent, and
borrowingLimitPercent located under the .spec.limited field of the designated priority level.
This PR adds the following metrics.
apiserver_flowcontrol_nominal_limit_seats: Nominal number of execution seats configured for each priority levelapiserver_flowcontrol_lower_limit_seats: Configured lower bound on number of execution seats available to each priority levelapiserver_flowcontrol_upper_limit_seats: Configured upper bound on number of execution seats available to each priority levelapiserver_flowcontrol_demand_seats: Observations, at the end of every nanosecond, of (the number of seats each priority level could use) / (nominal number of seats for that level)apiserver_flowcontrol_demand_seats_high_watermark: High watermark, over last adjustment period, of demand_seatsapiserver_flowcontrol_demand_seats_average: Time-weighted average, over last adjustment period, of demand_seatsapiserver_flowcontrol_demand_seats_stdev: Time-weighted standard deviation, over last adjustment period, of demand_seatsapiserver_flowcontrol_demand_seats_smoothed: Smoothed seat demandsapiserver_flowcontrol_target_seats: Seat allocation targetsapiserver_flowcontrol_seat_fair_frac: Fair fraction of server's concurrency to allocate to each priority level that can use itapiserver_flowcontrol_current_limit_seats: current derived number of execution seats available to each priority levelThe possibility of borrowing means that the old metric apiserver_flowcontrol_request_concurrency_limit can no longer mean both the configured concurrency limit and the enforced concurrency limit. Henceforth it means the configured concurrency limit. (kubernetes/kubernetes#113485, @MikeSpreitzer) [SIG API Machinery and Testing]
The EndpointSliceTerminatingCondition feature gate has graduated to GA. The gate is now locked and will be removed in v1.28. (kubernetes/kubernetes#113351, @andrewsykim) [SIG API Machinery, Apps, Network and Testing]
Yes, aggregated discovery will be alpha and can be toggled with the AggregatedDiscoveryEndpoint feature flag (kubernetes/kubernetes#113171, @Jefftree) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Cloud Provider, Cluster Lifecycle, Network, Node, Release, Scalability, Scheduling, Storage and Testing]
Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:
<!-- This section can be blank if this pull request does not require a release note.
When adding links which point to resources within git repositories, like KEPs or supporting documentation, please reference a specific commit and avoid linking directly to the master branch. This ensures that links reference a specific point in time, rather than a document that may change over time.
See here for guidance on getting permanent links to files: https://help.github.com/en/articles/getting-permanent-links-to-files
Please use the following format for linking documentation:
--> (kubernetes/kubernetes#86139, @jasimmons) [SIG API Machinery, Apps, Architecture, Auth, Autoscaling, CLI, Contributor Experience, Instrumentation, Network, Node, Release, Scheduling, Storage and Testing]
Add percentageOfNodesToScore as a scheduler profile level parameter to API version v1. If a profile percentageOfNodesToScore is set, it will override global percentageOfNodesToScore. (kubernetes/kubernetes#112521, @yuanchen8911) [SIG API Machinery, Scheduling and Testing]
Kube-controller-manager supports '--concurrent-horizontal-pod-autoscaler-syncs' flag to set the number of horizontal pod autoscaler controller workers. (kubernetes/kubernetes#108501, @zroubalik) [SIG API Machinery, Apps and Autoscaling]
Kube-proxy: The "userspace" proxy mode (deprecated for over a year) is no longer supported on either Linux or Windows. Users should use "iptables" or "ipvs" on Linux, or "kernelspace" on Windows. (kubernetes/kubernetes#112133, @knabben) [SIG API Machinery, Network, Scalability, Testing and Windows]
Kubectl wait command with jsonpath flag will wait for target path appear until timeout. (kubernetes/kubernetes#109525, @jonyhy96) [SIG CLI and Testing]
Kubelet external Credential Provider feature is moved to GA. Credential Provider Plugin and Credential Provider Config APIs updated from v1beta1 to v1 with no API changes. (kubernetes/kubernetes#111616, @ndixita) [SIG API Machinery, Node, Scheduling and Testing]
The DynamicKubeletConfig feature gate has been removed from the API server. Dynamic kubelet reconfiguration now cannot be used even when older nodes are still attempting to rely on it. This is aligned with the Kubernetes version skew policy. (kubernetes/kubernetes#112643, @SergeyKanzhelev) [SIG API Machinery, Apps, Auth, Node and Testing]
Add kubernetes_feature_enabled metric series to track whether each active feature gate is enabled. (kubernetes/kubernetes#112690, @logicalhan) [SIG API Machinery, Architecture, Cluster Lifecycle, Instrumentation, Network, Node and Scheduling]
Introduce v1beta3 for Priority and Fairness with the following changes to the API spec:
Legacy klog flags are no longer available. Only -v and -vmodule are still supported. (kubernetes/kubernetes#112120, @pohly) [SIG Architecture, CLI, Instrumentation, Node and Testing]
The feature gates ServiceLoadBalancerClass and ServiceLBNodePortControl have been removed. These feature gates were enabled (and locked) since v1.24. (kubernetes/kubernetes#112577, @andrewsykim) [SIG Apps]
Add auth API to get self subject attributes (new selfsubjectreviews API is added).
The corresponding command for kubctl is provided - kubectl auth whoami. (kubernetes/kubernetes#111333, @nabokihms) [SIG API Machinery, Auth, CLI and Testing]
Clarified the CFS quota as 100ms in the code comments and set the minimum cpuCFSQuotaPeriod to 1ms to match Linux kernel expectations. (kubernetes/kubernetes#112123, @paskal) [SIG API Machinery and Node]
Component-base: make the validation logic about LeaderElectionConfiguration consistent between component-base and client-go (kubernetes/kubernetes#111758, @SataQiu) [SIG API Machinery and Scheduling]
Fixes spurious field is immutable errors validating updates to Event API objects via the events.k8s.io/v1 API (kubernetes/kubernetes#112183, @liggitt) [SIG Apps]
Protobuf serialization of metav1.MicroTime timestamps (used in Lease and Event API objects) has been corrected to truncate to microsecond precision, to match the documented behavior and JSON/YAML serialization. Any existing persisted data is truncated to microsecond when read from etcd. (kubernetes/kubernetes#111936, @haoruan) [SIG API Machinery]
Revert regression that prevented client-go latency metrics to be reported with a template URL to avoid label cardinality. (kubernetes/kubernetes#111752, @aanm) [SIG API Machinery]
[kubelet] Change default cpuCFSQuotaPeriod value with enabled cpuCFSQuotaPeriod flag from 100ms to 100µs to match the Linux CFS and k8s defaults. cpuCFSQuotaPeriod of 100ms now requires customCPUCFSQuotaPeriod flag to be set to work. (kubernetes/kubernetes#111520, @paskal) [SIG API Machinery and Node]
One column per quarter.
Added a deprecated warning for node beta label usage in PV/SC/RC and CSI Storage Capacity. (kubernetes/kubernetes#108554, @pacoxu)
Kubernetes API Version: v1.25.11
NodeInclusionPolicy to TopologySpreadConstraints in PodSpec. (kubernetes/kubernetes#108492, @kerthcet)CheckpointRestore to enable support to checkpoint containers. If enabled it is possible to checkpoint a container using the newly kubelet API (/checkpoint/{podNamespace}/{podName}/{containerName}). (kubernetes/kubernetes#104907, @adrianreber) [SIG Node and Testing]restricted level no longer requires pods that set .spec.os.name="windows" to also set Linux-specific securityContext fields. If a 1.25+ cluster has unsupported out-of-skew nodes prior to v1.23 and wants to ensure namespaces enforcing the restricted policy continue to require Linux-specific securityContext fields on all pods, ensure a version of the restricted prior to v1.25 is selected by labeling the namespace (for example, pod-security.kubernetes.io/enforce-version: v1.24) (kubernetes/kubernetes#105919, @ravisantoshgudimetla)atomic to granular. (kubernetes/kubernetes#110495, @alexzielenski)MultiCIDRRangeAllocator=true, determines whether the MultiCIDRRangeAllocator controller can be used, while the kube-controller-manager flag below will pick the active controller.
Enabled the MultiCIDRRangeAllocator by setting --cidr-allocator-type=MultiCIDRRangeAllocator flag in kube-controller-manager. (kubernetes/kubernetes#109090, @sarveshr7)DisruptionTarget pod condition type. Its reason field indicates the reason for pod termination:
kubescheduler.config.k8s.io/v1 is available now.
Plugin SelectorSpread is removed in v1. (kubernetes/kubernetes#110534, @kerthcet)umount command is validated when the mounter is created. Additionally, mount point checks will be skipped in CleanupMountPoint/CleanupMountWithForce if the mounter's Unmount having the changed behavior of not returning error when target is not a mount point. (kubernetes/kubernetes#109676, @cartermckinnon) [SIG Storage]nil forever, but will be updated retroactively once any StorageClass is set or created as default. (kubernetes/kubernetes#111467, @RomanBednar)--feature-gates=StatefulSetMinReadySeconds=true are not needed on kube-apiserver and kube-controller-manager binaries and they'll be removed soon following policy at https://kubernetes.io/docs/reference/using-api/deprecation-policy/#deprecation (kubernetes/kubernetes#110896, @ravisantoshgudimetla) [SIG API Machinery, Apps and Testing]--feature-gates=DaemonSetUpdateSurge=true are not needed on kube-apiserver and kube-controller-manager binaries and they'll be removed soon following policy at https://kubernetes.io/docs/reference/using-api/deprecation-policy/#deprecation . (kubernetes/kubernetes#111194, @ravisantoshgudimetla)k8s.io/component-base was moved to k8s.io/component-base/logs/api/v1. The configuration file format and command line flags are the same as before. (kubernetes/kubernetes#105797, @pohly)spec.podOS field is promoted to GA. The IdentifyPodOS feature gate unconditionally enabled, and will no longer be accepted as a --feature-gates parameter in 1.27. (kubernetes/kubernetes#111229, @ravisantoshgudimetla)CSIInlineVolume feature has moved from beta to GA. (kubernetes/kubernetes#111258, @dobsonj)PodSecurity admission plugin has graduated to GA and is enabled by default. The admission configuration version has been promoted to pod-security.admission.config.k8s.io/v1. (kubernetes/kubernetes#110459, @wangyysde)endPort field in Network Policy is now promoted to GA
Network Policy providers that support endPort field now can use it to specify a range of ports to apply a Network Policy.
Previously, each Network Policy could only target a single port.
Please be aware that endPort field MUST BE SUPPORTED by the Network Policy provider. In case your provider does not support endPort and this field is specified in a Network Policy, the Network Policy will be created covering only the port field (single port). (kubernetes/kubernetes#110868, @rikatz)metadata.clusterName field is completely removed. This should not have any user-visible impact. (kubernetes/kubernetes#109602, @lavalamp)minDomains field in Pod Topology Spread is graduated to beta (kubernetes/kubernetes#110388, @sanposhiho) [SIG API Machinery and Apps]enable-taint-manager for kube-controller-manager is deprecated and will be removed in 1.26. The feature that it supports, taint based eviction, is enabled by default and will continue to be implicitly enabled when the flag is removed. (kubernetes/kubernetes#111411, @alculquicondor)NodeExpandSecret for CSI driver client which enables the CSI drivers to make use of this secret while performing node expansion operation based on the user request. Previously there was no secret provided as part of the nodeexpansion call, thus CSI drivers did not make use of the same while expanding the volume at the node side. (kubernetes/kubernetes#105963, @zhucan)EphemeralContainers feature gate is always enabled and should be removed from --feature-gates flag on the kube-apiserver and the kubelet command lines. The EphemeralContainers feature gate is deprecated and scheduled for removal in a future release. (kubernetes/kubernetes#111402, @verb)restricted level no longer requires pods that set .spec.os.name="windows" to also set Linux-specific securityContext fields. If a 1.25+ cluster has unsupported out-of-skew nodes prior to v1.23 and wants to ensure namespaces enforcing the restricted policy continue to require Linux-specific securityContext fields on all pods, ensure a version of the restricted prior to v1.25 is selected by labeling the namespace (for example, pod-security.kubernetes.io/enforce-version: v1.24) (kubernetes/kubernetes#105919, @ravisantoshgudimetla) [SIG API Machinery, Apps, Auth, Testing and Windows]atomic to granular. (kubernetes/kubernetes#110495, @alexzielenski) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Instrumentation and Testing]DisruptionTarget pod condition type. Its reason field indicates the reason for pod termination:
kubescheduler.config.k8s.io/v1 is available now.
Plugin SelectorSpread is removed in v1. (kubernetes/kubernetes#110534, @kerthcet) [SIG API Machinery, Scheduling and Testing]nil forever, but will be updated retroactively once any StorageClass is set or created as default. (kubernetes/kubernetes#111467, @RomanBednar) [SIG Apps, Storage and Testing]--feature-gates=DaemonSetUpdateSurge=true are not needed on kube-apiserver and kube-controller-manager binaries and they'll be removed soon following policy at https://kubernetes.io/docs/reference/using-api/deprecation-policy/#deprecation (kubernetes/kubernetes#111194, @ravisantoshgudimetla) [SIG Apps]spec.podOS field is promoted to GA. The IdentifyPodOS feature gate unconditionally enabled, and will no longer be accepted as a --feature-gates parameter in 1.27. (kubernetes/kubernetes#111229, @ravisantoshgudimetla) [SIG API Machinery, Apps and Windows]enable-taint-manager for kube-controller-manager is deprecated and will be removed in 1.26.
The feature that it supports, taint based eviction, is enabled by default and will continue to be implicitly enabled when the flag is removed. (kubernetes/kubernetes#111411, @alculquicondor) [SIG API Machinery]EphemeralContainers feature gate is always enabled and should be removed from --feature-gates flag on the kube-apiserver and the kubelet command lines. The EphemeralContainers feature gate is deprecated and scheduled for removal in a future release. (kubernetes/kubernetes#111402, @verb) [SIG API Machinery, Apps, Node, Storage and Testing]CheckpointRestore to enable support to checkpoint containers. If enabled it is possible to checkpoint a container using the newly kubelet API (/checkpoint/{podNamespace}/{podName}/{containerName}). (kubernetes/kubernetes#104907, @adrianreber) [SIG Node and Testing]umount command is validated when the mounter is created. Additionally, mount point checks will be skipped in CleanupMountPoint/CleanupMountWithForce if the mounter's Unmount having the changed behavior of not returning error when target is not a mount point. (kubernetes/kubernetes#109676, @cartermckinnon) [SIG Storage]--feature-gates=StatefulSetMinReadySeconds=true are not needed on kube-apiserver and kube-controller-manager binaries and they'll be removed soon following policy at https://kubernetes.io/docs/reference/using-api/deprecation-policy/#deprecation (kubernetes/kubernetes#110896, @ravisantoshgudimetla) [SIG API Machinery, Apps and Testing]spec.podOS field is promoted to GA. The IdentifyPodOS feature gate unconditionally enabled, and will no longer be accepted as a --feature-gates parameter in 1.27. (kubernetes/kubernetes#111229, @ravisantoshgudimetla) [SIG API Machinery, Apps and Windows]minDomains field in Pod Topology Spread is graduated to beta (kubernetes/kubernetes#110388, @sanposhiho) [SIG API Machinery and Apps]pod-security.admission.config.k8s.io/v1. (kubernetes/kubernetes#110459, @wangyysde) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node, Storage and Testing]metadata.clusterName field is completely removed. This should not have any user-visible impact. (kubernetes/kubernetes#109602, @lavalamp) [SIG API Machinery, Apps, Auth and Testing]chore: remove deprecated packages asynctest, codecov (#251, @tomplus)
fix: config reader handles bool types (#218, @tomplus)
fixed watch.stream bug of not working with apis with follow kwarg (#216, @mcreng)
CSIStorageCapacity.storage.k8s.io: The v1beta1 version of this API is deprecated in favor of v1, and will be removed in v1.27. If a CSI driver support…
Kubernetes API Version: v1.24.2
--forward-healthcheck-vip, if specified as true, health check traffic whose destination is service VIP will be forwarded to kube-proxy's healthcheck service. --root-hnsendpoint-name specifies the name of the hns endpoint for the root network namespace. This option enables the pass-through load balancers like Google's GCLB to correctly health check the backend services. Without this change, the health check packets is dropped, and Windows node will be considered to be unhealthy by those load balancers. (kubernetes/kubernetes#99287, @anfernee)webhook_fail_open_count to monitor webhooks that fail to open. (kubernetes/kubernetes#107171, @ltagliamonte-dd)InterfaceNamePrefix and BridgeInterface as arguments to --detect-local-mode option and also introduces a new optional --pod-interface-name-prefix and --pod-bridge-interface flags to kube-proxy. (kubernetes/kubernetes#95400, @tssurya)oldSelf. (kubernetes/kubernetes#108073, @benluddy)JSONSchemaProps.XValidations. (kubernetes/kubernetes#107956, @benluddy)oldSelf on a part of the schema that does not support it. (kubernetes/kubernetes#108013, @benluddy)fieldValidation=Strict consistently require apiVersion and kind, matching non-strict requests (kubernetes/kubernetes#109019, @liggitt)DefaultPodTopologySpread is graduated to GA (kubernetes/kubernetes#108278, @kerthcet)NonPreemptingPriority is graduated to GA (kubernetes/kubernetes#107432, @denkensk)PodOverhead is graduated to GA (kubernetes/kubernetes#108441, @pacoxu)strategic merge patch-type API requests for the selector field. Prior to 1.21, these requests would merge matchLabels content and replace matchExpressions content. In 1.21, patch requests touching the selector field started replacing the entire selector. This is consistent with server-side apply and the v1 PodDisruptionBudget behavior, but should not have been changed for v1beta1. (kubernetes/kubernetes#108138, @liggitt)--audit-log-version and --audit-webhook-version now only support the default value of audit.k8s.io/v1. The v1alpha1 and v1beta1 audit log versions, deprecated since 1.13, have been removed. (kubernetes/kubernetes#108092, @carlory)metadata.selfLink field can no longer be populated by kube-apiserver; it was deprecated in 1.16 and has not been populated by default since 1.20+. (kubernetes/kubernetes#107527, @wojtek-t)PodAffinityNamespaceSelector is locked and will be removed in 1.26. (kubernetes/kubernetes#108136, @ahg-g)evictions_number to evictions_total and mark it as stable. The original evictions_number metrics name is marked as "Deprecated" and has been removed in kubernetes 1.23 . (kubernetes/kubernetes#106366, @cyclinder)SuspendJob is locked and will be removed in 1.26. (kubernetes/kubernetes#108129, @ahg-g)spec.expirationSeconds API field has graduated to GA. The CSRDuration feature gate for the field is now unconditionally enabled and will be removed in 1.26. (kubernetes/kubernetes#108782, @cfryanr)ServerSideFieldValidation feature has graduated to beta and is now enabled by default. Kubectl 1.24 and newer will use server-side validation instead of client-side validation when writing to API servers with the feature enabled. (kubernetes/kubernetes#108889, @kevindelgado)ServiceLBNodePortControl feature has graduated to GA. The feature gate will be removed in 1.26. (kubernetes/kubernetes#107027, @uablrek)DynamicKubeletConfig has been removed from the kubelet. (kubernetes/kubernetes#106932, @SergeyKanzhelev)timeZone field as part of the CronJob spec to support running cron jobs in a specific time zone. (kubernetes/kubernetes#108032, @deejross)topologySpreadConstraints includes minDomains field to limit the minimum number of topology domains. (kubernetes/kubernetes#107674, @sanposhiho)--pod-interface-name-prefix and --pod-bridge-interface flags to kube-proxy. (kubernetes/kubernetes#95400, @tssurya) [SIG API Machinery and Network]oldSelf. (kubernetes/kubernetes#108073, @benluddy) [SIG API Machinery and Testing]ServerSideFieldValidation feature has graduated to beta and is now enabled by default. Kubectl 1.24 and newer will use server-side validation instead of client-side validation when writing to API servers with the feature enabled. (kubernetes/kubernetes#108889, @kevindelgado) [SIG API Machinery, Architecture, CLI and Testing]timeZone field as part of the CronJob spec to support running cron jobs in a specific time zone. (kubernetes/kubernetes#108032, @deejross) [SIG API Machinery and Apps]--forward-healthcheck-vip, if specified as true, health check traffic whose destination is service VIP will be forwarded to kube-proxy's healthcheck service. --root-hnsendpoint-name specifies the name of the hns endpoint for the root network namespace.
This option enables the pass-through load balancers like Google's GCLB to correctly health check the backend services. Without this change, the health check packets is dropped, and Windows node will be considered to be unhealthy by those load balancers. (kubernetes/kubernetes#99287, @anfernee) [SIG API Machinery, Cloud Provider, Network, Testing and Windows]DefaultPodTopologySpread is graduated to GA (kubernetes/kubernetes#108278, @kerthcet) [SIG Scheduling]PodOverhead is graduated to GA (kubernetes/kubernetes#108441, @pacoxu) [SIG API Machinery, Apps, Node and Scheduling]selector field. Prior to 1.21, these requests would merge matchLabels content and replace matchExpressions content. In 1.21, patch requests touching the selector field started replacing the entire selector. This is consistent with server-side apply and the v1 PodDisruptionBudget behavior, but should not have been changed for v1beta1. (kubernetes/kubernetes#108138, @liggitt) [SIG Apps, Auth and Testing]spec.expirationSeconds API field has graduated to GA. The CSRDuration feature gate for the field is now unconditionally enabled and will be removed in 1.26. (kubernetes/kubernetes#108782, @cfryanr) [SIG API Machinery, Apps, Auth, Instrumentation and Testing]NonPreemptingPriority is graduated to GA (kubernetes/kubernetes#107432, @denkensk) [SIG Apps, Scheduling and Testing]metadata.selfLink field can no longer be populated by kube-apiserver; it was deprecated in 1.16 and has not been populated by default in 1.20+. (kubernetes/kubernetes#107527, @wojtek-t) [SIG API Machinery, Apps, Auth, Autoscaling, CLI, Cloud Provider, Network, Scheduling, Storage and Testing]webhook_fail_open_count to monitor webhooks that fail open (kubernetes/kubernetes#107171, @ltagliamonte-dd) [SIG API Machinery and Instrumentation]evictions_number to evictions_total and mark it as stable. The original evictions_number metrics name is marked as "Deprecated" and will be removed in kubernetes 1.23 (kubernetes/kubernetes#106366, @cyclinder) [SIG API Machinery, Apps, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Release, Scalability, Scheduling, Storage, Testing and Windows]ServiceLBNodePortControl feature graduates to GA. The feature gate will be removed in 1.26. (kubernetes/kubernetes#107027, @uablrek) [SIG Network and Testing]…impacted by the associated security advisory: CVE-2021-29923 golang standard library "net" - Improper Input Validation of octal literals in golang 1.1…
selector field. Prior to 1.21, these requests would merge matchLabels content and replace matchExpressions content. In 1.21, patch requests touching the selector field started replacing the entire selector. This is consistent with server-side apply and the v1 PodDisruptionBudget behavior, but should not have been changed for v1beta1. (kubernetes/kubernetes#108139, @liggitt) [SIG Auth and Testing]omitManagedFields has been added to both audit.Policy and audit.PolicyRule
so cluster operators can opt in to omit managed fields of the request and response bodies from
being written to the API audit log. (kubernetes/kubernetes#94986, @tkashem) [SIG API Machinery, Auth, Cloud Provider and Testing]StatefulSetAutoDeletePVC, which allows PVCs automatically created for StatefulSet pods to be automatically deleted. (kubernetes/kubernetes#99728, @mattcary)generateName, the server now returns an "AlreadyExists" error with a retry option. (kubernetes/kubernetes#104699, @vincepri)-vmodule was made a bit more descriptive (pattern=N,... instead of moduleSpec). (kubernetes/kubernetes#106090, @pohly) [SIG API Machinery, Architecture, CLI, Cluster Lifecycle, Instrumentation, Node and Scheduling]OS field in the PodSpec (kubernetes/kubernetes#104693, @ravisantoshgudimetla)v1beta3 API for scheduler. This version
increases the weight of user specifiable priorities. The weights of following priority plugins are increased
TaintTolerations to 3 - as leveraging node tainting to group nodes in the cluster is becoming a widely-adopted practiceNodeAffinity to 2InterPodAffinity to 2Won't have HealthzBindAddress, MetricsBindAddress fields (kubernetes/kubernetes#104251, @ravisantoshgudimetla)
rbac.authorization.k8s.io/v1alpha1 API version is removed; use the rbac.authorization.k8s.io/v1 API, available since v1.8. The scheduling.k8s.io/v1alpha1 API version is removed; use the scheduling.k8s.io/v1 API, available since v1.14. (kubernetes/kubernetes#104248, @liggitt)v1beta1 is removed. Update configuration files to v1beta2(xref: https://github.com/kubernetes/enhancements/issues/2901) or v1beta3 before upgrading to 1.23. (kubernetes/kubernetes#104782, @kerthcet)MultiPoint which will register a plugin for all valid extension points (kubernetes/kubernetes#105611, @damemi) [SIG Scheduling and Testing]ResolverConfig field from a string to *string. (kubernetes/kubernetes#104624, @Haleygo)fieldValidation=[Strict,Warn,Ignore]. (kubernetes/kubernetes#105916, @kevindelgado)IPv6DualStack feature to stable.
Controller Manager flags for the node IPAM controller have slightly changed:
--node-cidr-mask-size-ipv4 and --node-cidr-mask-size-ipv6 to set the per-node IP mask sizes, instead of the previous --node-cidr-mask-size flag.--node-cidr-mask-size flag is mutually exclusive with --node-cidr-mask-size-ipv4 and --node-cidr-mask-size-ipv6.--node-cidr-mask-size flag or one of the newer --node-cidr-mask-size-ipv4 or --node-cidr-mask-size-ipv6 flags to configure the per-node IP mask size, provided that the flag's IP family matches the cluster's IP family (--cluster-cidr). (kubernetes/kubernetes#104691, @khenidak)NodeLease feature gate that was graduated and locked to stable in 1.17 release. (kubernetes/kubernetes#105222, @cyclinder)--seccomp-profile-root/seccompProfileRoot config. (kubernetes/kubernetes#103941, @saschagrunert)minReadySeconds is promoted to beta. (kubernetes/kubernetes#104045, @ravisantoshgudimetla)--register-with-taints option is now available via the Kubelet config file field registerWithTaints (kubernetes/kubernetes#105437, @cmssczy) [SIG Node and Scalability]CSIDriver.Spec.StorageCapacity can now be modified. (kubernetes/kubernetes#101789, @pohly)CSIVolumeFSGroupPolicy feature has moved from beta to GA. (kubernetes/kubernetes#105940, @dobsonj)IngressClass.Spec.Parameters.Namespace field is now GA. (kubernetes/kubernetes#104636, @hbagdi)Service.spec.ipFamilyPolicy field is now required in order to create or update a Service as dual-stack. This is a breaking change from the beta behavior. Previously the server would try to infer the value of that field from either ipFamilies or clusterIPs, but that caused ambiguity on updates. Users who want a dual-stack Service MUST specify ipFamilyPolicy as either "PreferDualStack" or "RequireDualStack". (kubernetes/kubernetes#96684, @thockin)TTLAfterFinished feature gate is now GA and enabled by default. (kubernetes/kubernetes#105219, @sahilvv)kube-controller-manager supports --concurrent-ephemeralvolume-syncs flag to set the number of ephemeral volume controller workers. (kubernetes/kubernetes#102981, @SataQiu)policy-config-file, policy-configmap, policy-configmap-namespace and use-legacy-policy-config are also removed. Migrate to Component Config instead, see https://kubernetes.io/docs/reference/scheduling/config/ for details. (kubernetes/kubernetes#105424, @kerthcet)LogFormatRegistry in component-base must update their code to use the logr v1.0.0 API. The JSON log output now uses the format from go-logr/zapr (no v field for error messages, additional information for invalid calls) and has some fixes (correct source code location for warnings about invalid log calls). (kubernetes/kubernetes#104103, @pohly)x-kubernetes-validations extension in OpenAPIv3 schemas (alpha). This is gated by the alpha "CustomResourceValidationExpressions" feature gate. (kubernetes/kubernetes#106051, @jpbetz) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node, Storage and Testing]fieldValidation=[Strict,Warn,Ignore] query parameter. (#105916, @kevindelgado) [SIG API Machinery, Apps, Auth, Cloud Provider and Testing]omitManagedFields has been added to both audit.Policy and audit.PolicyRule
so cluster operators can opt in to omit managed fields of the request and response bodies from
being written to the API audit log. (#94986, @tkashem) [SIG API Machinery, Auth, Cloud Provider and Testing]-vmodule was made a bit more descriptive (pattern=N,... instead of moduleSpec). (#106090, @pohly) [SIG API Machinery, Architecture, CLI, Cluster Lifecycle, Instrumentation, Node and Scheduling]MultiPoint which will register a plugin for all valid extension points (#105611, @damemi) [SIG Scheduling and Testing]--register-with-taints option is now available via the Kubelet config file field registerWithTaints (#105437, @cmssczy) [SIG Node and Scalability]x-kubernetes-validations extension in OpenAPIv3 schemas (alpha). This is gated by the alpha "CustomResourceValidationExpressions" feature gate. (#106051, @jpbetz) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node, Storage and Testing]increases the weight of user specifiable priorities. The weights of following priority plugins are increased
Won't have HealthzBindAddress, MetricsBindAddress fields (#104251, @ravisantoshgudimetla) [SIG Scheduling and Testing]
ResolverConfig field from a string to *string. (kubernetes/kubernetes#104624, @Haleygo) [SIG Cluster Lifecycle and Node]ResolverConfig field from a string to *string. (kubernetes/kubernetes#104624, @Haleygo) [SIG Cluster Lifecycle and Node]--seccomp-profile-root/seccompProfileRoot config (kubernetes/kubernetes#103941, @saschagrunert) [SIG Node]Service.spec.ipFamilyPolicy field is now required in order to create or update a Service as dual-stack. This is a breaking change from the beta behavior. Previously the server would try to infer the value of that field from either ipFamilies or clusterIPs, but that caused ambiguity on updates. Users who want a dual-stack Service MUST specify ipFamilyPolicy as either "PreferDualStack" or "RequireDualStack". (kubernetes/kubernetes#96684, @thockin) [SIG API Machinery, Apps, Network and Testing]v field for error messages, additional information for invalid calls) and has some fixes (correct source code location for warnings about invalid log calls). (kubernetes/kubernetes#104103, @pohly) [SIG API Machinery, Architecture, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation and Storage]rbac.authorization.k8s.io/v1alpha1 API version is removed; use the rbac.authorization.k8s.io/v1 API, available since v1.8. The scheduling.k8s.io/v1alpha1 API version is removed; use the scheduling.k8s.io/v1 API, available since v1.14. (kubernetes/kubernetes#104248, @liggitt) [SIG API Machinery, Auth, Network and Testing]feat: periodically refresh ServiceAccount tokens (#205, @JacobHenner)
fix regenerate library with correct crd spec (#198, @tomplus)
feat: configurable heartbeat value for WsApiClient (#195, @GlassOfWhiskey)
fix: add patch to update read buffer size (#192, @tomplus)
fix: passing Bearer token in authorization header (#190, @tomplus)
…plugins, which are marked as deprecated as of v1beta2. In v1beta1, the three plugins can still be used in v1beta1 but not at the same time with the sc…
socket_options to configuration for the rest clientKube-apiserver: Fixes handling of CRD schemas containing literal null values in enums (#104988, @liggitt) [SIG API Machinery, Apps and Network]
A new score extension for NodeResourcesFit plugin that merges the functionality of NodeResourcesLeastAllocated, NodeResourcesMostAllocated, RequestedToCapacityRatio plugins, which are marked as deprecated as of v1beta2. In v1beta1, the three plugins can still be used in v1beta1 but not at the same time with the score extension of NodeResourcesFit. (kubernetes/kubernetes#101822, @yuzhiquan)
A value of Auto is now a valid for the service.kubernetes.io/topology-aware-hints annotation. (kubernetes/kubernetes#100728, @robscott)
Add DataSourceRef alpha field to PVC spec, which allows contents other than PVCs and VolumeSnapshots to be data sources. (kubernetes/kubernetes#103276, @bswartz)
Add PersistentVolumeClaimDeletePoilcy to StatefulSet API. (kubernetes/kubernetes#99378, @mattcary)
Add a new Priority and Fairness rule that exempts all probes (/readyz, /healthz, /livez) to prevent restarting of healthy kube-apiserver instance by kubelet. (kubernetes/kubernetes#100678, @tkashem)
Add alpha support for HostProcess containers on Windows (kubernetes/kubernetes#99576, @marosset) [SIG API Machinery, Apps, Node, Testing and Windows]
Add distributed tracing to the kube-apiserver. It is can be enabled with the feature gate APIServerTracing (kubernetes/kubernetes#94942, @dashpole)
Add three metrics to the job controller to monitor if a job works in healthy condition.
IndexedJob has been promoted to Beta. (kubernetes/kubernetes#101292, @AliceZhang2016)
Added field .status.uncountedTerminatedPods to the Job resource. This field is used by the job controller to keep track of finished pods before adding them to the Job status counters. Pods created by the job controller get the finalizer batch.kubernetes.io/job-tracking
Jobs that are tracked using this mechanism get the annotation batch.kubernetes.io/job-tracking. This is a temporary measure. Two releases after this feature graduates to beta, the annotation won't be added to Jobs anymore. (kubernetes/kubernetes#98817, @alculquicondor)
Added new kubelet alpha feature SeccompDefault. This feature enables falling back to
the RuntimeDefault (former runtime/default) seccomp profile if nothing else is specified
in the pod/container SecurityContext or the pod annotation level. To use the feature, enable
the feature gate as well as set the kubelet configuration option SeccompDefault
(--seccomp-default) to true. (kubernetes/kubernetes#101943, @saschagrunert) [SIG Node]
Adds the ReadWriteOncePod access mode for PersistentVolumes and PersistentVolumeClaims. Restricts volume access to a single pod on a single node. (kubernetes/kubernetes#102028, @chrishenzie)
Alpha swap support can now be enabled on Kubernetes nodes with the NodeSwapEnabled feature flag. See KEP-2400 for details. (kubernetes/kubernetes#102823, @ehashman)
Because of the implementation logic of time.Format in golang, the displayed time zone is not consistent. (kubernetes/kubernetes#102366, @cndoit18)
Corrected the documentation for escaping dollar signs in a container's env, command and args property. (kubernetes/kubernetes#101916, @MartinKanters) [SIG Apps]
Enable MaxSurge for DaemonSet by default. (kubernetes/kubernetes#101742, @ravisantoshgudimetla)
Enforce the ReadWriteOncePod PVC access mode during scheduling (kubernetes/kubernetes#103082, @chrishenzie)
Ephemeral containers are now allowed to configure a securityContext that differs from that of the Pod. Cluster administrators should ensure that security policy controllers support EphemeralContainers before enabling this feature in clusters. (kubernetes/kubernetes#99023, @verb)
Exec plugin authors can override default handling of standard input via new interactiveMode kubeconfig field. (kubernetes/kubernetes#99310, @ankeesler)
If someone had the ProbeTerminationGracePeriod alpha feature enabled in 1.21, they should update/delete any workloads/pods with probe terminationGracePeriods < 1 before upgrading (kubernetes/kubernetes#103245, @wzshiming)
Improved parsing of label selectors (kubernetes/kubernetes#102188, @alculquicondor) [SIG API Machinery]
Introduce minReadySeconds api to the StatefulSets. (kubernetes/kubernetes#100842, @ravisantoshgudimetla)
Introducing Memory quality of service support with cgroups v2 (Alpha). The MemoryQoS feature is now in Alpha. This allows kubelet running with cgroups v2 to set memory QoS at container, pod and QoS level to protect and guarantee better memory quality. This feature can be enabled through feature gate Memory QoS. (kubernetes/kubernetes#102970, @borgerli)
Kube API server accepts Impersonate-Uid header to impersonate a user with a specific UID, in the same way that you can currently use Impersonate-User, Impersonate-Group and Impersonate-Extra. (kubernetes/kubernetes#99961, @margocrawf)
Kube-apiserver: --service-account-issuer can be specified multiple times now, to enable non-disruptive change of issuer. (kubernetes/kubernetes#101155, @zshihang) [SIG API Machinery, Auth, Node and Testing]
Kube-controller-manager: the --horizontal-pod-autoscaler-use-rest-clients flag and Heapster support in the horizontal pod autoscaler, deprecated since 1.12, is removed. (kubernetes/kubernetes#90368, @serathius)
Kube-scheduler: a plugin enabled in a v1beta2 configuration file takes precedence over the default configuration for that plugin. This simplifies enabling default plugins with custom configuration without needing to explicitly disable those default plugins. (kubernetes/kubernetes#99582, @chendave)
New node-high priority-level has been added to Suggested API Priority and (kubernetes/kubernetes#101151, @mborsz)
NodeSwapEnabled feature flag was renamed to NodeSwap
The flag was only available in the 1.22.0-beta.1 release, and the new flag should be used going forward. (kubernetes/kubernetes#103553, @ehashman) [SIG Node]
Omit comparison with boolean constant (kubernetes/kubernetes#101523, @chuntaochen) [SIG CLI and Cloud Provider]
Removed the feature flag for probe-level termination grace period from Kubelet. If a user wants to disable this feature on already created pods, they will have to delete and recreate the pods. (kubernetes/kubernetes#103168, @raisaat) [SIG Apps and Node]
Revert addition of Add PersistentVolumeClaimDeletePoilcy to StatefulSetAPI. (kubernetes/kubernetes#103747, @mattcary)
Scheduler could be configured to consider new resources beside CPU and memory, GPU for example, for the score plugin of NodeResourcesBalancedAllocation. (kubernetes/kubernetes#101946, @chendave) [SIG Scheduling]
Server Side Apply now treats all <Some>Selector fields as atomic (meaning the entire selector is managed by a single writer and updated together), since they contain interrelated and inseparable fields that do not merge in intuitive ways. (kubernetes/kubernetes#97989, @Danil-Grigorev) [SIG API Machinery]
Suspend Job feature graduated to beta. Added the action label to Job controller sync metrics job_sync_total and job_sync_duration_seconds. (kubernetes/kubernetes#102022, @adtac)
The API documentation for the DaemonSet's spec.updateStrategy.rollingUpdate.maxUnavailable field was corrected to state that the value is rounded up. (kubernetes/kubernetes#101296, @Miciah)
The CSIServiceAccountToken graduates to Ga and is unconditionally enabled. (kubernetes/kubernetes#103001, @zshihang)
The CertificateSigningRequest.certificates.k8s.io API supports an optional expirationSeconds field to allow the client to request a particular duration for the issued certificate. The default signer implementations provided by the Kubernetes controller manager will honor this field as long as it does not exceed the --cluster-signing-duration flag. (kubernetes/kubernetes#99494, @enj)
The EndpointSlicen Mirroring controller no longer mirrors the last-applied-configuration annotation created by kubectl to update EndpointSlices. (kubernetes/kubernetes#102731, @sharmarajdaksh)
The NetworkPolicyEndPort is graduated to beta and is enabled by default. (kubernetes/kubernetes#102834, @rikatz)
The PodDeletionCost feature has been promoted to beta, and enabled by default. (kubernetes/kubernetes#101080, @ahg-g)
The Server Side Apply treats certain structs as atomic. Meaning the entire selector field is managed by a single writer and updated together. (kubernetes/kubernetes#100684, @Jefftree)
The ServiceAppProtocol feature gate has been removed. It reached GA in Kubernetes (kubernetes/kubernetes#103190, @robscott)
The TerminationGracePeriodSeconds on pod specs and container probes should not be negative. Negative values of TerminationGracePeriodSeconds will be treated as the value 1s on the delete path. Immutable field validation will be relaxed in order to update negative values. In a future release, negative values will not be permitted. (kubernetes/kubernetes#98866, @wzshiming)
The kube-scheduler component config v1beta2 API available
Three scheduler plugins deprecated (NodeLabel, ServiceAffinity, NodePreferAvoidPods). (kubernetes/kubernetes#99597, @adtac)
The pod/eviction subresource now accepts policy/v1 eviction requests in addition to policy/v1beta1 eviction requests (kubernetes/kubernetes#100724, @liggitt)
The podAffinity, NamespaceSelector and the associated CrossNamespaceAffinity quota scope features graduate to Beta and they are now enabled by default. (kubernetes/kubernetes#101496, @ahg-g)
The pods/ephemeralcontainers API now returns and expects a Pod object instead of EphemeralContainers. This is incompatible with the previous alpha-level API. (kubernetes/kubernetes#101034, @verb) [SIG Apps, Auth, CLI and Testing]
The v1.Node and .status.images[].names are now optional. (kubernetes/kubernetes#102159, @roycaihw)
The deprecated flag --algorithm-provider has been removed from kube-scheduler. Use instead ComponentConfig to configure the set of enabled plugins. (kubernetes/kubernetes#102239, @Haleygo)
The options --ssh-user and --ssh-key are removed. They only functioned on GCE, and only in-tree. Use the apiserver network proxy instead. (kubernetes/kubernetes#102297, @deads2k)
Track Job completion through status and Pod finalizers, removing dependency on Pod tombstones. (kubernetes/kubernetes#98238, @alculquicondor) [SIG API Machinery, Apps, Auth and Testing]
Track ownership of scale subresource for all scalable resources i.e. Deployment, ReplicaSet, StatefulSet, ReplicationController, and Custom Resources. (kubernetes/kubernetes#98377, @nodo) [SIG API Machinery and Testing]
Revert addition of Add PersistentVolumeClaimDeletePoilcy to StatefulSet API. (kubernetes/kubernetes#103747, @mattcary) [SIG API Machinery and Apps]
Added field .status.uncountedTerminatedPods to the Job resource. This field is used by the job controller to keep track of finished pods before adding them to the Job status counters.
Pods created by the job controller get the finalizer batch.kubernetes.io/job-tracking
Jobs that are tracked using this mechanism get the annotation batch.kubernetes.io/job-tracking. This is a temporary measure. Two releases after this feature graduates to beta, the annotation won't be added to Jobs anymore. (kubernetes/kubernetes#98817, @alculquicondor) [SIG API Machinery, Apps, Auth and CLI]
Ephemeral containers are now allowed to configure a securityContext that differs from that of the Pod.
Cluster administrators should ensure that security policy controllers support EphemeralContainers before enabling this feature in clusters. (kubernetes/kubernetes#99023, @verb) [SIG API Machinery, Apps, Auth and Node]
If someone had the ProbeTerminationGracePeriod alpha feature enabled in 1.21, they should update/delete any workloads/pods with probe terminationGracePeriods < 1 before upgrading (kubernetes/kubernetes#103245, @wzshiming) [SIG Apps and Node]
Introducing Memory QoS support with cgroups v2 (Alpha) The MemoryQoS feature is now in Alpha. This allows kubelet running with cgroups v2 to set memory QoS at container, pod and QoS level to protect and guarantee better memory quality. This feature can be enabled through feature gate MemoryQoS. (kubernetes/kubernetes#102970, @borgerli) [SIG Node and Storage]
NodeSwapEnabled feature flag was renamed to NodeSwap
The flag was only available in the 1.22.0-beta.1 release, and the new flag should be used going forward. (kubernetes/kubernetes#103553, @ehashman) [SIG Node]
Removed the feature flag for probe-level termination grace period from Kubelet. If a user wants to disable this feature on already created pods, they will have to delete and recreate the pods. (kubernetes/kubernetes#103168, @raisaat) [SIG Apps and Node]
Track Job completion through status and Pod finalizers, removing dependency on Pod tombstones. (kubernetes/kubernetes#98238, @alculquicondor) [SIG API Machinery, Apps, Auth and Testing]
When using kubectl replace (or the equivalent API call) on a Service, the caller no longer needs to do a read-modify-write cycle to fetch the allocated values for .spec.clusterIP and .spec.ports[].nodePort. Instead the API server will automatically carry these forward from the original object when the new object does not specify them. (kubernetes/kubernetes#103532, @thockin) [SIG Apps and Network]
A new score extension for NodeResourcesFit plugin that merges the functionality of NodeResourcesLeastAllocated,NodeResourcesMostAllocated,RequestedToCapacityRatio plugins, which are marked as deprecated as of v1beta2. In v1beta1, the three plugins can still be used in v1beta1 but not at the same time with the score extension of NodeResourcesFit
Add DataSourceRef alpha field to PVC spec, which allows contents other than PVCs and VolumeSnapshots to be data sources. (kubernetes/kubernetes#103276, @bswartz) [SIG API Machinery, Apps and Storage]
Add PersistentVolumeClaimDeletePoilcy to StatefulSet API. (kubernetes/kubernetes#99378, @mattcary) [SIG API Machinery and Apps]
Add distributed tracing to the kube-apiserver. It is can be enabled with the feature gate: APIServerTracing=true (kubernetes/kubernetes#94942, @dashpole) [SIG API Machinery, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node, Storage and Testing]
Added new kubelet alpha feature SeccompDefault. This feature enables falling back to
the RuntimeDefault (former runtime/default) seccomp profile if nothing else is specified
in the pod/container SecurityContext or the pod annotation level. To use the feature, enable
the feature gate as well as set the kubelet configuration option SeccompDefault
(--seccomp-default) to true. (kubernetes/kubernetes#101943, @saschagrunert) [SIG Node]
Adds the ReadWriteOncePod access mode for PersistentVolumes and PersistentVolumeClaims. Restricts volume access to a single pod on a single node. (kubernetes/kubernetes#102028, @chrishenzie) [SIG Apps, CLI, Node, Scheduling and Storage]
Alpha swap support can now be enabled on Kubernetes nodes with the NodeSwapEnabled feature flag. See <website link> for details. (kubernetes/kubernetes#102823, @ehashman) [SIG Node]
CSIServiceAccountToken is GA. (kubernetes/kubernetes#103001, @zshihang) [SIG Auth and Storage]
Enforce the ReadWriteOncePod PVC access mode during scheduling (kubernetes/kubernetes#103082, @chrishenzie) [SIG Apps, CLI, Node, Scheduling and Storage]
Improved parsing of label selectors (kubernetes/kubernetes#102188, @alculquicondor) [SIG API Machinery]
Kube API server accepts Impersonate-Uid header to impersonate a user with a specific UID, in the same way that you can currently use Impersonate-User, Impersonate-Group and Impersonate-Extra (kubernetes/kubernetes#99961, @margocrawf) [SIG API Machinery, Auth and Testing]
Kube-scheduler: a plugin enabled in a v1beta2 configuration file takes precedence over the default configuration for that plugin; this simplifies enabling default plugins with custom configuration without needing to explicitly disable those default plugins. (kubernetes/kubernetes#99582, @chendave) [SIG Scheduling]
Scheduler could be configured to consider new resources beside CPU and memory, GPU for example, for the score plugin of NodeResourcesBalancedAllocation. (kubernetes/kubernetes#101946, @chendave) [SIG Scheduling]
Suspend Job feature graduated to beta Added the "action" label to Job controller sync metrics job_sync_total and job_sync_duration_seconds (kubernetes/kubernetes#102022, @adtac) [SIG Apps, Instrumentation and Testing]
TerminationGracePeriodSeconds on pod specs and container probes should not be negative.
Negative values of TerminationGracePeriodSeconds will be treated as the value 1s on the delete path.
Immutable field validation will be relaxed in order to update negative values.
In a future release, negative values will not be permitted. (kubernetes/kubernetes#98866, @wzshiming) [SIG API Machinery, Apps and Node]
The API documentation for the DaemonSet's spec.updateStrategy.rollingUpdate.maxUnavailable field was corrected to state that the value is rounded up. (kubernetes/kubernetes#101296, @Miciah) [SIG Apps and CLI]
The CertificateSigningRequest.certificates.k8s.io API supports an optional expirationSeconds field to allow the client to request a particular duration for the issued certificate. The default signer implementations provided by the Kubernetes controller manager will honor this field as long as it does not exceed the --cluster-signing-duration flag. (kubernetes/kubernetes#99494, @enj) [SIG API Machinery, Apps, Auth, CLI, Instrumentation, Node, Security and Testing]
The ServiceAppProtocol feature gate has been removed. It reached GA in Kubernetes 1.20. (kubernetes/kubernetes#103190, @robscott) [SIG Network]
Because of the implementation logic of time.Format in golang, the displayed time zone is not consistent (kubernetes/kubernetes#102366, @cndoit18) [SIG Apps, Auth, Autoscaling, CLI, Cluster Lifecycle, Instrumentation, Network, Node and Testing]
Endpoint slices mirroring controller no longer mirrors the last-applied-configuration annotation created by kubectl to updated endpoint slices (kubernetes/kubernetes#102731, @sharmarajdaksh) [SIG API Machinery, Apps, Cloud Provider, Network, Release, Scheduling, Storage and Testing]
Exec plugin authors can override default handling of standard input via new interactiveMode kubeconfig field (kubernetes/kubernetes#99310, @ankeesler) [SIG API Machinery, Auth, CLI and Testing]
Kube-scheduler component config v1beta2 API available Three scheduler plugins deprecated (NodeLabel, ServiceAffinity, NodePreferAvoidPods) (kubernetes/kubernetes#99597, @adtac) [SIG Scheduling]
Network Policy EndPort is graduated to beta and is enabled by default (kubernetes/kubernetes#102834, @rikatz) [SIG Network]
--ssh-user and --ssh-key options are removed. They only functioned on GCE, and only in-tree. Use the apiserver network proxy instead. (kubernetes/kubernetes#102297, @deads2k) [SIG API Machinery, Cloud Provider and Testing]
Enable MaxSurge for DS by default (kubernetes/kubernetes#101742, @ravisantoshgudimetla) [SIG Apps and Testing]
Introduce minReadySeconds api to the StatefulSets. (kubernetes/kubernetes#100842, @ravisantoshgudimetla) [SIG API Machinery, Apps and Testing]
Kube-controller-manger: the --horizontal-pod-autoscaler-use-rest-clients flag and Heapster support in the horizontal pod autoscaler, deprecated since 1.12, is removed. (kubernetes/kubernetes#90368, @serathius) [SIG API Machinery, Apps, Autoscaling, Cloud Provider and Instrumentation]
The deprecated flag --algorithm-provider has been removed from kube-scheduler. Use instead ComponentConfig to configure the set of enabled plugins (kubernetes/kubernetes#102239, @Haleygo) [SIG Cloud Provider and Scheduling]
Add alpha support for HostProcess containers on Windows (kubernetes/kubernetes#99576, @marosset) [SIG API Machinery, Apps, Node, Testing and Windows]
Add three metrics to job controller to monitor if Job works in a healthy condition. IndexedJob promoted to Beta (kubernetes/kubernetes#101292, @AliceZhang2016) [SIG Apps, Instrumentation and Testing]
Corrected the documentation for escaping dollar signs in a container's env, command and args property. (kubernetes/kubernetes#101916, @MartinKanters) [SIG Apps]
Omit comparison with boolean constant (kubernetes/kubernetes#101523, @GreenApple10) [SIG CLI and Cloud Provider]
Pod Affinity NamespaceSelector and the associated CrossNamespaceAffinity quota scope graduated to beta (kubernetes/kubernetes#101496, @ahg-g) [SIG API Machinery, Apps and Testing]
V1.Node .status.images[].names is now optional (kubernetes/kubernetes#102159, @roycaihw) [SIG Apps and Node]
"Auto" is now a valid value for the service.kubernetes.io/topology-aware-hints annotation. (kubernetes/kubernetes#100728, @robscott) [SIG Apps, Instrumentation and Network]
Kube-apiserver: --service-account-issuer can be specified multiple times now, to enable non-disruptive change of issuer. (kubernetes/kubernetes#101155, @zshihang) [SIG API Machinery, Auth, Node and Testing]
New "node-high" priority-level has been added to Suggested API Priority and Fairness configuration. (kubernetes/kubernetes#101151, @mborsz) [SIG API Machinery]
PodDeletionCost promoted to Beta (kubernetes/kubernetes#101080, @ahg-g) [SIG Apps]
SSA treats certain structs as atomic (kubernetes/kubernetes#100684, @Jefftree) [SIG API Machinery, Auth, Node and Storage]
Server Side Apply now treats all <Some>Selector fields as atomic (meaning the entire selector is managed by a single writer and updated together), since they contain interrelated and inseparable fields that do not merge in intuitive ways. (kubernetes/kubernetes#97989, @Danil-Grigorev) [SIG API Machinery]
The pods/ephemeralcontainers API now returns and expects a Pod object instead of EphemeralContainers. This is incompatible with the previous alpha-level API. (kubernetes/kubernetes#101034, @verb) [SIG Apps, Auth, CLI and Testing]
The pod/eviction subresource now accepts policy/v1 Eviction requests in addition to policy/v1beta1 Eviction requests (kubernetes/kubernetes#100724, @liggitt) [SIG API Machinery, Apps, Architecture, Auth, CLI, Storage and Testing]
Track ownership of scale subresource for all scalable resources i.e. Deployment, ReplicaSet, StatefulSet, ReplicationController, and Custom Resources. (kubernetes/kubernetes#98377, @nodo) [SIG API Machinery and Testing]
We have added a new Priority & Fairness rule that exempts all probes (/readyz, /healthz, /livez) to prevent restarting of "healthy" kube-apiserver instance(s) by kubelet. (kubernetes/kubernetes#100678, @tkashem) [SIG API Machinery]
fix: add missing change in latest regnerated lib (#184, @tomplus)
…objects are deprecated (kubernetes/kubernetes#99641, @pohly)
service.kubernetes.io/topology-aware-hints annotation. (kubernetes/kubernetes#100728, @robscott) [SIG Apps, Instrumentation and Network].spec.completionMode field to Job, with accepted values NonIndexed (default) and Indexed. This is an alpha field and is only honored by servers with the IndexedJob feature gate enabled. (kubernetes/kubernetes#98441, @alculquicondor) [SIG Apps and CLI]/debug/pprof and /debug/flags/v endpoint in kubelet by setting enableProfilingHandler and enableDebugFlagsHandler to false in the Kubelet configuration file. Options enableProfilingHandler and enableDebugFlagsHandler can be set to true only when enableDebuggingHandlers is also set to true. (kubernetes/kubernetes#98458, @SaranBalaji90)kubectl exec and kubectl portforward won't be interrupted. (kubernetes/kubernetes#97083, @knight42) [SIG API Machinery and CLI].spec.suspend field that can be used to suspend and resume Jobs. This is an alpha field which is only honored by servers with the SuspendJob feature gate enabled. (kubernetes/kubernetes#98727, @adtac)kubernetes.io/metadata.name label matching their metadata.name field to allow selecting any namespace by its name using a label selector. (kubernetes/kubernetes#96968, @jayunit100) [SIG API Machinery, Apps, Cloud Provider, Storage and Testing]immutable field in Secret or ConfigMap object to mark their contents as immutable. (kubernetes/kubernetes#97615, @wojtek-t) [SIG Apps, Architecture, Node and Testing]FailedAndUnresolvableNodes of ExtenderFilterResult struct. Nodes in this map will be skipped in the preemption phase. (kubernetes/kubernetes#92866, @cofyc) [SIG Scheduling]/var/run/secrets/kubernetes.io/serviceaccount/token to be time-limited, auto-refreshed, and invalidated when the containing pod is deleted.k8s.io/client-go version v11.0.0+ and v0.15.0+ reload tokens automatically.serviceaccount_stale_tokens_total can be used to monitor for workloads that are depending on the extended lifetime and are continuing to use tokens even after a refreshed token is provided to the container. If that metric indicates no existing workloads are depending on extended lifetimes, injected token lifetime can be shortened to 1 hour by starting kube-apiserver with --service-account-extend-token-expiration=false. (kubernetes/kubernetes#95667, @zshihang) [SIG API Machinery, Auth, Cluster Lifecycle and Testing]EndpointSliceController will not populate the deprecatedTopology field and will only provide topology information through the zone and nodeName fields. (kubernetes/kubernetes#99870, @swetharepakula)endpoints.kubernetes.io/over-capacity annotation to "warning" when an Endpoints resource contains more than 1000 addresses. In a future release, the controller will truncate Endpoints that exceed this limit. The EndpointSlice API can be used to support significantly larger number of addresses. (kubernetes/kubernetes#99975, @robscott) [SIG Apps and Network]{}) written to a policy/v1 PodDisruptionBudget now selects all pods in the namespace. The behavior of the policy/v1beta1 API remains unchanged. The policy/v1beta1 PodDisruptionBudget API is deprecated and will no longer be served in 1.25+. (kubernetes/kubernetes#99290, @mortent) [SIG API Machinery, Apps, Auth, Autoscaling, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Scheduling and Testing]EndpointSlice API is now GA. The EndpointSlice topology field has been removed from the GA API and will be replaced by a new per Endpoint Zone field. If the topology field was previously used, it will be converted into an annotation in the v1 Resource. The discovery.k8s.io/v1alpha1 API is removed. (kubernetes/kubernetes#99662, @swetharepakula)controller.kubernetes.io/pod-deletion-cost annotation can be set to offer a hint on the cost of deleting a Pod compared to other pods belonging to the same ReplicaSet. Pods with lower deletion cost are deleted first. This is an alpha feature. (kubernetes/kubernetes#99163, @ahg-g)managedFields that got corrupted by a mutating admission controller. (kubernetes/kubernetes#98074, @kwiesmueller)TopologyAwareHints feature gate. (kubernetes/kubernetes#99522, @robscott) [SIG API Machinery, Apps, Auth, Instrumentation, Network and Testing]kubectl.kubernetes.io/default-exec-container annotation in a Pod to preselect container for kubectl commands. (kubernetes/kubernetes#97099, @pacoxu) [SIG CLI]endpoints.kubernetes.io/over-capacity annotation to "warning" when an Endpoints resource contains more than 1000 addresses. In a future release, the controller will truncate Endpoints that exceed this limit. The EndpointSlice API can be used to support significantly larger number of addresses. (kubernetes/kubernetes#99975, @robscott) [SIG Apps and Network]{}) written to a policy/v1 PodDisruptionBudget now selects all pods in the namespace. The behavior of the policy/v1beta1 API remains unchanged. The policy/v1beta1 PodDisruptionBudget API is deprecated and will no longer be served in 1.25+. (kubernetes/kubernetes#99290, @mortent) [SIG API Machinery, Apps, Auth, Autoscaling, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Scheduling and Testing]TopologyAwareHints feature gate. (kubernetes/kubernetes#99522, @robscott) [SIG API Machinery, Apps, Auth, Instrumentation, Network and Testing].spec.completionMode field to Job, with accepted values NonIndexed (default) and Indexed (kubernetes/kubernetes#98441, @alculquicondor) [SIG Apps and CLI]deprecatedTopology field and will only provide topology information through the zone and nodeName fields. (kubernetes/kubernetes#99870, @swetharepakula) [SIG API Machinery, Apps, Auth, Network and Testing]controller.kubernetes.io/pod-deletion-cost annotation can be set to offer a hint on the cost of deleting a pod compared to other pods belonging to the same ReplicaSet. Pods with lower deletion cost are deleted first. This is an alpha feature. (kubernetes/kubernetes#99163, @ahg-g) [SIG Apps]/var/run/secrets/kubernetes.io/serviceaccount/token to be time-limited, auto-refreshed, and invalidated when the containing pod is deleted.k8s.io/client-go version v11.0.0+ and v0.15.0+ reload tokens automatically.serviceaccount_stale_tokens_total can be used to monitor for workloads that are depending on the extended lifetime and are continuing to use tokens even after a refreshed token is provided to the container. If that metric indicates no existing workloads are depending on extended lifetimes, injected token lifetime can be shortened to 1 hour by starting kube-apiserver with --service-account-extend-token-expiration=false. (kubernetes/kubernetes#95667, @zshihang) [SIG API Machinery, Auth, Cluster Lifecycle and Testing]FailedAndUnresolvableNodes of ExtenderFilterResult struct. Nodes in this map will be skipped in the preemption phase. (kubernetes/kubernetes#92866, @cofyc) [SIG Scheduling]kubectl exec and kubectl port-forward won't be interrupted. (kubernetes/kubernetes#97083, @knight42) [SIG API Machinery and CLI]Immutable field in Secrets or ConfigMap object to mark their contents as immutable. (kubernetes/kubernetes#97615, @wojtek-t) [SIG Apps, Architecture, Node and Testing]Regenerate protobuf code to fix CVE-2021-3121 (kubernetes/kubernetes#100501, @joelsmith) [SIG API Machinery, Apps, Auth, CLI, Cloud Provider, Cluster…
Kube-apiserver: Fixes handling of CRD schemas containing literal null values in enums (#104990, @liggitt) [SIG API Machinery, Apps and Network]
We have added a new Priority & Fairness rule that exempts all probes (/readyz, /healthz, /livez) to prevent restarting of "healthy" kube-apiserver instance(s) by kubelet. (kubernetes/kubernetes#101112, @tkashem) [SIG API Machinery]
Fixes using server-side apply with APIService resources (kubernetes/kubernetes#100714, @kevindelgado) [SIG API Machinery, Apps and Testing]
Regenerate protobuf code to fix CVE-2021-3121 (kubernetes/kubernetes#100501, @joelsmith) [SIG API Machinery, Apps, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Node and Storage]
Kubernetes is now built using go1.15.8 (kubernetes/kubernetes#98962, @cpanato) [SIG Cloud Provider, Instrumentation, Release and Testing]
TokenRequest and TokenRequestProjection features have been promoted to GA. This feature allows generating service account tokens that are not visible in Secret objects and are tied to the lifetime of a Pod object. See https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#service-account-token-volume-projection for details on configuring and using this feature. The TokenRequest and TokenRequestProjection feature gates will be removed in v1.21.
A new nofuzz go build tag now disables gofuzz support. Release binaries enable this. (kubernetes/kubernetes#92491, @BenTheElder) [SIG API Machinery]
Add WindowsContainerResources and Annotations to CRI-API UpdateContainerResourcesRequest (kubernetes/kubernetes#95741, @katiewasnothere) [SIG Node]
Add a serving and terminating condition to the EndpointSlice API.
serving tracks the readiness of endpoints regardless of their terminating state. This is distinct from ready since ready is only true when pods are not terminating.
terminating is true when an endpoint is terminating. For pods this is any endpoint with a deletion timestamp. (kubernetes/kubernetes#92968, @andrewsykim) [SIG Apps and Network]
Add dual-stack Services (alpha). This is a BREAKING CHANGE to an alpha API. It changes the dual-stack API wrt Service from a single ipFamily field to 3 fields: ipFamilyPolicy (SingleStack, PreferDualStack, RequireDualStack), ipFamilies (a list of families assigned), and clusterIPs (inclusive of clusterIP). Most users do not need to set anything at all, defaulting will handle it for them. Services are single-stack unless the user asks for dual-stack. This is all gated by the "IPv6DualStack" feature gate. (kubernetes/kubernetes#91824, @khenidak) [SIG API Machinery, Apps, CLI, Network, Node, Scheduling and Testing]
Add support for hugepages to downward API (kubernetes/kubernetes#86102, @derekwaynecarr) [SIG API Machinery, Apps, CLI, Network, Node, Scheduling and Testing]
Adds kubelet alpha feature, GracefulNodeShutdown which makes kubelet aware of node system shutdowns and result in graceful termination of pods during a system shutdown. (kubernetes/kubernetes#96129, @bobbypage) [SIG Node]
AppProtocol is now GA for Endpoints and Services. The ServiceAppProtocol feature gate will be deprecated in 1.21. (kubernetes/kubernetes#96327, @robscott) [SIG Apps and Network]
Automatic allocation of NodePorts for services with type LoadBalancer can now be disabled by setting the (new) parameter Service.spec.allocateLoadBalancerNodePorts=false. The default is to allocate NodePorts for services with type LoadBalancer which is the existing behavior. (kubernetes/kubernetes#92744, @uablrek) [SIG Apps and Network]
Certain fields on Service objects will be automatically cleared when changing the service's type to a mode that does not need those fields. For example, changing from type=LoadBalancer to type=ClusterIP will clear the NodePort assignments, rather than forcing the user to clear them. (kubernetes/kubernetes#95196, @thockin) [SIG API Machinery, Apps, Network and Testing]
Document that ServiceTopology feature is required to use service.spec.topologyKeys. (kubernetes/kubernetes#96528, @andrewsykim) [SIG Apps]
EndpointSlice has a new NodeName field guarded by the EndpointSliceNodeName feature gate.
External facing API podresources is now available under k8s.io/kubelet/pkg/apis/ (kubernetes/kubernetes#92632, @RenaudWasTaken) [SIG Node and Testing]
Fewer candidates are enumerated for preemption to improve performance in large clusters. (kubernetes/kubernetes#94814, @adtac)
Fix conversions for custom metrics. (kubernetes/kubernetes#94481, @wojtek-t) [SIG API Machinery and Instrumentation]
GPU metrics provided by kubelet are now disabled by default. (kubernetes/kubernetes#95184, @RenaudWasTaken)
If BoundServiceAccountTokenVolume is enabled, cluster admins can use metric serviceaccount_stale_tokens_total to monitor workloads that are depending on the extended tokens. If there are no such workloads, turn off extended tokens by starting kube-apiserver with flag --service-account-extend-token-expiration=false (kubernetes/kubernetes#96273, @zshihang) [SIG API Machinery and Auth]
Introduce alpha support for exec-based container registry credential provider plugins in the kubelet. (kubernetes/kubernetes#94196, @andrewsykim) [SIG Node and Release]
Introduces a metric source for HPAs which allows scaling based on container resource usage. (kubernetes/kubernetes#90691, @arjunrn) [SIG API Machinery, Apps, Autoscaling and CLI]
Kube-apiserver now deletes expired kube-apiserver Lease objects:
APIServerIdentity.identity-lease-garbage-collection-check-period-seconds (kubernetes/kubernetes#95895, @roycaihw) [SIG API Machinery, Apps, Auth and Testing]Kube-controller-manager: volume plugins can be restricted from contacting local and loopback addresses by setting --volume-host-allow-local-loopback=false, or from contacting specific CIDR ranges by setting --volume-host-cidr-denylist (for example, --volume-host-cidr-denylist=127.0.0.1/28,feed::/16) (kubernetes/kubernetes#91785, @mattcary) [SIG API Machinery, Apps, Auth, CLI, Network, Node, Storage and Testing]
Migrate scheduler, controller-manager and cloud-controller-manager to use LeaseLock (kubernetes/kubernetes#94603, @wojtek-t) [SIG API Machinery, Apps, Cloud Provider and Scheduling]
Modify DNS-1123 error messages to indicate that RFC 1123 is not followed exactly (kubernetes/kubernetes#94182, @mattfenwick) [SIG API Machinery, Apps, Auth, Network and Node]
Move configurable fsgroup change policy for pods to beta (kubernetes/kubernetes#96376, @gnufied) [SIG Apps and Storage]
New flag is introduced, i.e. --topology-manager-scope=container|pod. The default value is the "container" scope. (kubernetes/kubernetes#92967, @cezaryzukowski) [SIG Instrumentation, Node and Testing]
New parameter defaultingType for PodTopologySpread plugin allows to use k8s defined or user provided default constraints (kubernetes/kubernetes#95048, @alculquicondor) [SIG Scheduling]
NodeAffinity plugin can be configured with AddedAffinity. (kubernetes/kubernetes#96202, @alculquicondor) [SIG Node, Scheduling and Testing]
Promote RuntimeClass feature to GA. Promote node.k8s.io API groups from v1beta1 to v1. (kubernetes/kubernetes#95718, @SergeyKanzhelev) [SIG Apps, Auth, Node, Scheduling and Testing]
Reminder: The labels "failure-domain.beta.kubernetes.io/zone" and "failure-domain.beta.kubernetes.io/region" are deprecated in favor of "topology.kubernetes.io/zone" and "topology.kubernetes.io/region" respectively. All users of the "failure-domain.beta..." labels should switch to the "topology..." equivalents. (kubernetes/kubernetes#96033, @thockin) [SIG API Machinery, Apps, CLI, Cloud Provider, Network, Node, Scheduling, Storage and Testing]
Server Side Apply now treats LabelSelector fields as atomic (meaning the entire selector is managed by a single writer and updated together), since they contain interrelated and inseparable fields that do not merge in intuitive ways. (kubernetes/kubernetes#93901, @jpbetz) [SIG API Machinery, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Storage and Testing]
Services will now have a clusterIPs field to go with clusterIP. clusterIPs[0] is a synonym for clusterIP and will be synchronized on create and update operations. (kubernetes/kubernetes#95894, @thockin) [SIG Network]
The ServiceAccountIssuerDiscovery feature gate is now Beta and enabled by default. (kubernetes/kubernetes#91921, @mtaufen) [SIG Auth]
The status of v1beta1 CRDs without "preserveUnknownFields:false" now shows a violation, "spec.preserveUnknownFields: Invalid value: true: must be false". (kubernetes/kubernetes#93078, @vareti)
The usage of mixed protocol values in the same LoadBalancer Service is possible if the new feature gate MixedProtocolLBService is enabled. The feature gate is disabled by default. The user has to enable it for the API Server. (kubernetes/kubernetes#94028, @janosi) [SIG API Machinery and Apps]
This PR will introduce a feature gate CSIServiceAccountToken with two additional fields in CSIDriverSpec. (kubernetes/kubernetes#93130, @zshihang) [SIG API Machinery, Apps, Auth, CLI, Network, Node, Storage and Testing]
Users can try the CronJob controller v2 using the feature gate. This will be the default controller in future releases. (kubernetes/kubernetes#93370, @alaypatel07) [SIG API Machinery, Apps, Auth and Testing]
VolumeSnapshotDataSource moves to GA in 1.20 release (kubernetes/kubernetes#95282, @xing-yang) [SIG Apps]
WinOverlay feature graduated to beta (kubernetes/kubernetes#94807, @ksubrmnn) [SIG Windows]
API priority and fairness graduated to beta 1.19 servers with APF turned on should not be run in a multi-server cluster with 1.20+ servers. (kubernetes/kubernetes#96527, @adtac) [SIG API Machinery and Testing]
Add LoadBalancerIPMode feature gate (kubernetes/kubernetes#92312, @Sh4d1) [SIG Apps, CLI, Cloud Provider and Network]
Add WindowsContainerResources and Annotations to CRI-API UpdateContainerResourcesRequest (kubernetes/kubernetes#95741, @katiewasnothere) [SIG Node]
Add a 'serving' and terminating condition to the EndpointSlice API.
serving tracks the readiness of endpoints regardless of their terminating state. This is distinct from ready since ready is only true when pods are not terminating.
terminating is true when an endpoint is terminating. For pods this is any endpoint with a deletion timestamp. (kubernetes/kubernetes#92968, @andrewsykim) [SIG Apps and Network]
Add support for hugepages to downward API (kubernetes/kubernetes#86102, @derekwaynecarr) [SIG API Machinery, Apps, CLI, Network, Node, Scheduling and Testing]
Adds kubelet alpha feature, GracefulNodeShutdown which makes kubelet aware of node system shutdowns and result in graceful termination of pods during a system shutdown. (kubernetes/kubernetes#96129, @bobbypage) [SIG Node]
AppProtocol is now GA for Endpoints and Services. The ServiceAppProtocol feature gate will be deprecated in 1.21. (kubernetes/kubernetes#96327, @robscott) [SIG Apps and Network]
Automatic allocation of NodePorts for services with type LoadBalancer can now be disabled by setting the (new) parameter Service.spec.allocateLoadBalancerNodePorts=false. The default is to allocate NodePorts for services with type LoadBalancer which is the existing behavior. (kubernetes/kubernetes#92744, @uablrek) [SIG Apps and Network]
Document that ServiceTopology feature is required to use service.spec.topologyKeys. (kubernetes/kubernetes#96528, @andrewsykim) [SIG Apps]
EndpointSlice has a new NodeName field guarded by the EndpointSliceNodeName feature gate.
Fewer candidates are enumerated for preemption to improve performance in large clusters (kubernetes/kubernetes#94814, @adtac) [SIG Scheduling]
If BoundServiceAccountTokenVolume is enabled, cluster admins can use metric serviceaccount_stale_tokens_total to monitor workloads that are depending on the extended tokens. If there are no such workloads, turn off extended tokens by starting kube-apiserver with flag --service-account-extend-token-expiration=false (kubernetes/kubernetes#96273, @zshihang) [SIG API Machinery and Auth]
Introduce alpha support for exec-based container registry credential provider plugins in the kubelet. (kubernetes/kubernetes#94196, @andrewsykim) [SIG Node and Release]
Kube-apiserver now deletes expired kube-apiserver Lease objects:
APIServerIdentity.identity-lease-garbage-collection-check-period-seconds (kubernetes/kubernetes#95895, @roycaihw) [SIG API Machinery, Apps, Auth and Testing]Move configurable fsgroup change policy for pods to beta (kubernetes/kubernetes#96376, @gnufied) [SIG Apps and Storage]
New flag is introduced, i.e. --topology-manager-scope=container|pod. The default value is the "container" scope. (kubernetes/kubernetes#92967, @cezaryzukowski) [SIG Instrumentation, Node and Testing]
NodeAffinity plugin can be configured with AddedAffinity. (kubernetes/kubernetes#96202, @alculquicondor) [SIG Node, Scheduling and Testing]
Promote RuntimeClass feature to GA. Promote node.k8s.io API groups from v1beta1 to v1. (kubernetes/kubernetes#95718, @SergeyKanzhelev) [SIG Apps, Auth, Node, Scheduling and Testing]
Reminder: The labels "failure-domain.beta.kubernetes.io/zone" and "failure-domain.beta.kubernetes.io/region" are deprecated in favor of "topology.kubernetes.io/zone" and "topology.kubernetes.io/region" respectively. All users of the "failure-domain.beta..." labels should switch to the "topology..." equivalents. (kubernetes/kubernetes#96033, @thockin) [SIG API Machinery, Apps, CLI, Cloud Provider, Network, Node, Scheduling, Storage and Testing]
The usage of mixed protocol values in the same LoadBalancer Service is possible if the new feature gate MixedProtocolLBSVC is enabled. "action required" The feature gate is disabled by default. The user has to enable it for the API Server. (kubernetes/kubernetes#94028, @janosi) [SIG API Machinery and Apps]
This PR will introduce a feature gate CSIServiceAccountToken with two additional fields in CSIDriverSpec. (kubernetes/kubernetes#93130, @zshihang) [SIG API Machinery, Apps, Auth, CLI, Network, Node, Storage and Testing]
Users can try the CronJob controller v2 using the feature gate. This will be the default controller in future releases. (kubernetes/kubernetes#93370, @alaypatel07) [SIG API Machinery, Apps, Auth and Testing]
VolumeSnapshotDataSource moves to GA in 1.20 release (kubernetes/kubernetes#95282, @xing-yang) [SIG Apps]
TokenRequest and TokenRequestProjection features have been promoted to GA. This feature allows generating service account tokens that are not visible in Secret objects and are tied to the lifetime of a Pod object. See https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#service-account-token-volume-projection for details on configuring and using this feature. The TokenRequest and TokenRequestProjection feature gates will be removed in v1.21.Certain fields on Service objects will be automatically cleared when changing the service's type to a mode that does not need those fields. For example, changing from type=LoadBalancer to type=ClusterIP will clear the NodePort assignments, rather than forcing the user to clear them. (kubernetes/kubernetes#95196, @thockin) [SIG API Machinery, Apps, Network and Testing]
Services will now have a clusterIPs field to go with clusterIP. clusterIPs[0] is a synonym for clusterIP and will be synchronized on create and update operations. (kubernetes/kubernetes#95894, @thockin) [SIG Network]
Add dual-stack Services (alpha). This is a BREAKING CHANGE to an alpha API. It changes the dual-stack API wrt Service from a single ipFamily field to 3 fields: ipFamilyPolicy (SingleStack, PreferDualStack, RequireDualStack), ipFamilies (a list of families assigned), and clusterIPs (inclusive of clusterIP). Most users do not need to set anything at all, defaulting will handle it for them. Services are single-stack unless the user asks for dual-stack. This is all gated by the "IPv6DualStack" feature gate. (kubernetes/kubernetes#91824, @khenidak) [SIG API Machinery, Apps, CLI, Network, Node, Scheduling and Testing]
Introduces a metric source for HPAs which allows scaling based on container resource usage. (kubernetes/kubernetes#90691, @arjunrn) [SIG API Machinery, Apps, Autoscaling and CLI]
New parameter defaultingType for PodTopologySpread plugin allows to use k8s defined or user-provided default constraints (kubernetes/kubernetes#95048, @alculquicondor) [SIG Scheduling]
GPU metrics provided by kubelet are now disabled by default (kubernetes/kubernetes#95184, @RenaudWasTaken) [SIG Node]
New parameter defaultingType for PodTopologySpread plugin allows to use k8s defined or user provided default constraints (kubernetes/kubernetes#95048, @alculquicondor) [SIG Scheduling]
Server Side Apply now treats LabelSelector fields as atomic (meaning the entire selector is managed by a single writer and updated together), since they contain interrelated and inseparable fields that do not merge in intuitive ways. (kubernetes/kubernetes#93901, @jpbetz) [SIG API Machinery, Auth, CLI, Cloud Provider, Cluster Lifecycle, Instrumentation, Network, Node, Storage and Testing]
Status of v1beta1 CRDs without "preserveUnknownFields:false" will show violation "spec.preserveUnknownFields: Invalid value: true: must be false" (kubernetes/kubernetes#93078, @vareti) [SIG API Machinery]
A new nofuzz go build tag now disables gofuzz support. Release binaries enable this. (kubernetes/kubernetes#92491, @BenTheElder) [SIG API Machinery]
A new alpha-level field, SupportsFsGroup, has been introduced for CSIDrivers to allow them to specify whether they support volume ownership and permission modifications. The CSIVolumeSupportFSGroup feature gate must be enabled to allow this field to be used. (kubernetes/kubernetes#92001, @huffmanca) [SIG API Machinery, CLI and Storage]
Added pod version skew strategy for seccomp profile to synchronize the deprecated annotations with the new API Server fields. Please see the corresponding section in the KEP for more detailed explanations. (kubernetes/kubernetes#91408, @saschagrunert) [SIG Apps, Auth, CLI and Node]
Adds the ability to disable Accelerator/GPU metrics collected by Kubelet (kubernetes/kubernetes#91930, @RenaudWasTaken) [SIG Node]
Custom Endpoints are now mirrored to EndpointSlices by a new EndpointSliceMirroring controller. (kubernetes/kubernetes#91637, @robscott) [SIG API Machinery, Apps, Auth, Cloud Provider, Instrumentation, Network and Testing]
External facing API podresources is now available under k8s.io/kubelet/pkg/apis/ (kubernetes/kubernetes#92632, @RenaudWasTaken) [SIG Node and Testing]
Fix conversions for custom metrics. (kubernetes/kubernetes#94481, @wojtek-t) [SIG API Machinery and Instrumentation]
Generic ephemeral volumes, a new alpha feature under the GenericEphemeralVolume feature gate, provide a more flexible alternative to EmptyDir volumes: as with EmptyDir, volumes are created and deleted for each pod automatically by Kubernetes. But because the normal provisioning process is used (PersistentVolumeClaim), storage can be provided by third-party storage vendors and all of the usual volume features work. Volumes don't need to be empty; for example, restoring from snapshot is supported. (kubernetes/kubernetes#92784, @pohly) [SIG API Machinery, Apps, Auth, CLI, Instrumentation, Node, Scheduling, Storage and Testing]
Kube-controller-manager: volume plugins can be restricted from contacting local and loopback addresses by setting --volume-host-allow-local-loopback=false, or from contacting specific CIDR ranges by setting --volume-host-cidr-denylist (for example, --volume-host-cidr-denylist=127.0.0.1/28,feed::/16) (kubernetes/kubernetes#91785, @mattcary) [SIG API Machinery, Apps, Auth, CLI, Network, Node, Storage and Testing]
Kubernetes is now built with golang 1.15.0-rc.1.
Migrate scheduler, controller-manager and cloud-controller-manager to use LeaseLock (kubernetes/kubernetes#94603, @wojtek-t) [SIG API Machinery, Apps, Cloud Provider and Scheduling]
Modify DNS-1123 error messages to indicate that RFC 1123 is not followed exactly (kubernetes/kubernetes#94182, @mattfenwick) [SIG API Machinery, Apps, Auth, Network and Node]
The ServiceAccountIssuerDiscovery feature gate is now Beta and enabled by default. (kubernetes/kubernetes#91921, @mtaufen) [SIG Auth]
The kube-controller-manager managed signers can now have distinct signing certificates and keys. See the help about --cluster-signing-[signer-name]-{cert,key}-file. --cluster-signing-{cert,key}-file is still the default. (kubernetes/kubernetes#90822, @deads2k) [SIG API Machinery, Apps and Auth]
When creating a networking.k8s.io/v1 Ingress API object, spec.tls[*].secretName values are required to pass validation rules for Secret API object names. (kubernetes/kubernetes#93929, @liggitt) [SIG Network]
WinOverlay feature graduated to beta (kubernetes/kubernetes#94807, @ksubrmnn) [SIG Windows]
fix: watch returns raw_object if detection of returned objects fail (#177, @tomplus)
Regenerate protobuf code to fix CVE-2021-3121 (kubernetes/kubernetes#100515, @joelsmith) [SIG API Machinery, Auth, CLI, Cloud Provider, Cluster Lifecy…
SupportsFsGroup, has been introduced for CSIDrivers to allow them to specify whether they support volume ownership and permission modifications. The CSIVolumeSupportFSGroup feature gate must be enabled to allow this field to be used. (kubernetes/kubernetes#92001, @huffmanca) [SIG API Machinery, CLI and Storage].response.warnings field in the admission review response. (kubernetes/kubernetes#92667, @liggitt) [SIG API Machinery and Testing]status field was added; this field defaults to True, and may only be set to True for Approved, Denied, and Failed conditionslastTransitionTime field was addedFailed condition type was added to allow signers to indicate permanent failure; this condition can be added via the certificatesigningrequests/status subresource.Approved and Denied conditions are mutually exclusiveApproved, Denied, and Failed conditions can no longer be removed from a CSR (kubernetes/kubernetes#90191, @liggitt) [SIG API Machinery, Apps, Auth, CLI and Node]spec.versions[*].deprecated to true, and for optionally overriding the default deprecation warning with a spec.versions[*].deprecationWarning field. (kubernetes/kubernetes#92329, @liggitt) [SIG API Machinery]GenericEphemeralVolume feature gate, provide a more flexible alternative to EmptyDir volumes: as with EmptyDir, volumes are created and deleted for each pod automatically by Kubernetes. But because the normal provisioning process is used (PersistentVolumeClaim), storage can be provided by third-party storage vendors and all of the usual volume features work. Volumes don't need to be empt; for example, restoring from snapshot is supported. (kubernetes/kubernetes#92784, @pohly) [SIG API Machinery, Apps, Auth, CLI, Instrumentation, Node, Scheduling, Storage and Testing]+k8s:conversion-gen tags can be used with the k8s.io/code-generator component to generate conversions. (kubernetes/kubernetes#90018, @wojtek-t) [SIG API Machinery, Apps and Testing]--bind-address-hard-fail flag to treat failure to bind to a port as fatal (kubernetes/kubernetes#89350, @SataQiu) [SIG Cluster Lifecycle and Network]runOnce. (kubernetes/kubernetes#89128, @vincent178) [SIG Node]Immutable field in Secrets or ConfigMap object to mark their contents as immutable. (kubernetes/kubernetes#89594, @wojtek-t) [SIG Apps and Testing]BindTimeoutSeconds from schedule configuration KubeSchedulerConfiguration (kubernetes/kubernetes#91580, @cofyc) [SIG Scheduling and Testing]metadata.managedFields handling in update/patch requests submitted by older API clients (kubernetes/kubernetes#91748, @apelisse)CSIStorageCapacity feature gate, only works for CSI drivers and depends on support for the feature in a CSI driver deployment) (kubernetes/kubernetes#92387, @pohly) [SIG API Machinery, Apps, Auth, Scheduling, Storage and Testing]seccompProfile field is added to pod and container securityContext objects. Support for seccomp.security.alpha.kubernetes.io/pod and container.seccomp.security.alpha.kubernetes.io/... annotations is deprecated, and will be removed in v1.22. (kubernetes/kubernetes#91381, @pjbgf) [SIG Apps, Auth, Node, Release, Scheduling and Testing]spec.signerName is now required, and requests for kubernetes.io/legacy-unknown are not allowed to be created via the certificates.k8s.io/v1 APIspec.usages is now required, may not contain duplicate values, and must only contain known usagesstatus.conditions may not contain duplicate typesstatus.conditions[*].status is now requiredstatus.certificate must be PEM-encoded, and contain only CERTIFICATE blocks (kubernetes/kubernetes#91685, @liggitt) [SIG API Machinery, Architecture, Auth, CLI and Testing]--bootstrap-checkpoint-path option is now removed. (kubernetes/kubernetes#91577, @knabben) [SIG Apps and Node]--cloud-provider and --cloud-config options are now marked as deprecated. (kubernetes/kubernetes#90408, @knabben) [SIG Cloud Provider and Node]--enable-server and --provider-id option is now available via the Kubelet config file field enableServer and providerID respectively. (kubernetes/kubernetes#90494, @knabben) [SIG Node]--kernel-memcg-notification option is now available via the Kubelet config file field kernelMemcgNotification (kubernetes/kubernetes#91863, @knabben) [SIG Cloud Provider, Node and Testing]--really-crash-for-testing and --chaos-chance options are now marked as deprecated. (kubernetes/kubernetes#90499, @knabben) [SIG Node]--volume-plugin-dir option is now available via the Kubelet config file field VolumePluginDir. (kubernetes/kubernetes#88480, @savitharaghunathan) [SIG Node]DefaultIngressClass feature is now GA. The --feature-gate parameter will be removed in 1.20. (kubernetes/kubernetes#91957, @cmluciano) [SIG API Machinery, Apps, Network and Testing]DynamicAuditing feature gate and auditregistration.k8s.io/v1alpha1 API have been removed and are no longer supported. (kubernetes/kubernetes#91502, @deads2k) [SIG API Machinery, Auth and Testing]--cluster-signing-[signer-name]-{cert,key}-file. --cluster-signing-{cert,key}-file is still the default. (kubernetes/kubernetes#90822, @deads2k) [SIG API Machinery, Apps and Auth]series.state field, deprecated since v1.14, is removed from the events.k8s.io/v1beta1 and v1 Event types. (kubernetes/kubernetes#90449, @wojtek-t) [SIG Apps]Ingress and IngressClass resources have graduated to networking.k8s.io/v1. Ingress and IngressClass types in the extensions/v1beta1 and networking.k8s.io/v1beta1 API versions are deprecated and will no longer be served in 1.22+. Persisted objects can be accessed via the networking.k8s.io/v1 API. Notable changes in v1 Ingress objects (v1beta1 field names are unchanged):
spec.backend -> spec.defaultBackendserviceName -> service.nameservicePort -> service.port.name (for string values)servicePort -> service.port.number (for numeric values)pathType no longer has a default value in v1; "Exact", "Prefix", or "ImplementationSpecific" must be specified
Other Ingress API updates:path is no longer required to be a valid regular expression (kubernetes/kubernetes#89778, @cmluciano) [SIG API Machinery, Apps, CLI, Network and Testing]NodeResourcesLeastAllocated and NodeResourcesMostAllocated plugins now support customized weight on the CPU and memory. (kubernetes/kubernetes#90544, @chendave) [SIG Scheduling]PostFilter type is added to scheduler component config API on version v1beta1. (kubernetes/kubernetes#91547, @Huang-Wei) [SIG Scheduling]RequestedToCapacityRatioArgs encoding is now strict (kubernetes/kubernetes#91603, @pancernik) [SIG Scheduling]v1beta1 Scheduler Extender encoding is case-sensitive (v1alpha1/v1alpha2 was case-insensitive), its httpTimeout field uses duration encoding (for example, one second is specified as "1s"), and the enableHttps field in v1alpha1/v1alpha2 was renamed to enableHTTPS. (kubernetes/kubernetes#91625, @pancernik) [SIG Scheduling]SupportsFsGroup, has been introduced for CSIDrivers to allow them to specify whether they support volume ownership and permission modifications. The CSIVolumeSupportFSGroup feature gate must be enabled to allow this field to be used. (kubernetes/kubernetes#92001, @huffmanca) [SIG API Machinery, CLI and Storage]--cluster-signing-[signer-name]-{cert,key}-file. --cluster-signing-{cert,key}-file is still the default. (kubernetes/kubernetes#90822, @deads2k) [SIG API Machinery, Apps and Auth]GenericEphemeralVolume feature gate, provide a more flexible alternative to EmptyDir volumes: as with EmptyDir, volumes are created and deleted for each pod automatically by Kubernetes. But because the normal provisioning process is used (PersistentVolumeClaim), storage can be provided by third-party storage vendors and all of the usual volume features work. Volumes don't need to be empt; for example, restoring from snapshot is supported. (kubernetes/kubernetes#92784, @pohly) [SIG API Machinery, Apps, Auth, CLI, Instrumentation, Node, Scheduling, Storage and Testing]BindTimeoutSeconds from schedule configuration KubeSchedulerConfiguration (kubernetes/kubernetes#91580, @cofyc) [SIG Scheduling and Testing]spec.signerName is now required, and requests for kubernetes.io/legacy-unknown are not allowed to be created via the certificates.k8s.io/v1 APIspec.usages is now required, may not contain duplicate values, and must only contain known usagesstatus.conditions may not contain duplicate typesstatus.conditions[*].status is now requiredstatus.certificate must be PEM-encoded, and contain only CERTIFICATE blocks (kubernetes/kubernetes#91685, @liggitt) [SIG API Machinery, Architecture, Auth, CLI and Testing]--cloud-provider and --cloud-config options are now marked as deprecated. (kubernetes/kubernetes#90408, @knabben) [SIG Cloud Provider and Node]status field was added; this field defaults to True, and may only be set to True for Approved, Denied, and Failed conditionslastTransitionTime field was addedFailed condition type was added to allow signers to indicate permanent failure; this condition can be added via the certificatesigningrequests/status subresource.Approved and Denied conditions are mutually exclusiveApproved, Denied, and Failed conditions can no longer be removed from a CSR (kubernetes/kubernetes#90191, @liggitt) [SIG API Machinery, Apps, Auth, CLI and Node]--enable-server and --provider-id option is now available via the Kubelet config file field enableServer and providerID respectively. (kubernetes/kubernetes#90494, @knabben) [SIG Node]--really-crash-for-testing and --chaos-chance options are now marked as deprecated. (kubernetes/kubernetes#90499, @knabben) [SIG Node]DynamicAuditing feature gate and auditregistration.k8s.io/v1alpha1 API have been removed and are no longer supported. (kubernetes/kubernetes#91502, @deads2k) [SIG API Machinery, Auth and Testing]NodeResourcesLeastAllocated and NodeResourcesMostAllocated plugins now support customized weight on the CPU and memory. (kubernetes/kubernetes#90544, @chendave) [SIG Scheduling]PostFilter type is added to scheduler component config API on version v1beta1. (kubernetes/kubernetes#91547, @Huang-Wei) [SIG Scheduling]kubescheduler.config.k8s.io is now beta (kubernetes/kubernetes#91420, @pancernik) [SIG Scheduling]--really-crash-for-testing and --chaos-chance options are now marked as deprecated. (kubernetes/kubernetes#90499, @knabben) [SIG Node]NodeResourcesLeastAllocated and NodeResourcesMostAllocated plugins now support customized weight on the CPU and memory. (kubernetes/kubernetes#90544, @chendave) [SIG Scheduling]+k8s:conversion-gen tags can be used with the k8s.io/code-generator component to generate conversions. (kubernetes/kubernetes#90018, @wojtek-t) [SIG API Machinery, Apps and Testing]runOnce. (kubernetes/kubernetes#89128, @vincent178) [SIG Node]Immutable field in Secrets or ConfigMap object to mark their contents as immutable. (kubernetes/kubernetes#89594, @wojtek-t) [SIG Apps and Testing]series.state field, deprecated since v1.14, is removed from the events.k8s.io/v1beta1 and v1 Event types. (kubernetes/kubernetes#90449, @wojtek-t) [SIG Apps]--bind-address-hard-fail flag to treat failure to bind to a port as fatal (kubernetes/kubernetes#89350, @SataQiu) [SIG Cluster Lifecycle and Network]--volume-plugin-dir option is now available via the Kubelet config file field VolumePluginDir. (kubernetes/kubernetes#88480, @savitharaghunathan) [SIG Node]x-kubernetes-list-map-keys to specify properties that uniquely identify list items must make those properties required or have a default value, to ensure those properties are present for all list items. See https://kubernetes.io/docs/reference/using-api/api-concepts/#merge-strategy for details. (kubernetes/kubernetes#88076, @eloyekunle) [SIG API Machinery and Testing]x-kubernetes-list-type: map or x-kubernetes-list-type: set in CustomResources. (kubernetes/kubernetes#84920, @sttts) [SIG API Machinery]--profiling=false to disable. (kubernetes/kubernetes#88663, @deads2k) [SIG API Machinery, Cloud Provider and Scheduling].spec.SchedulerName. (kubernetes/kubernetes#88285, @alculquicondor) [SIG Apps, Scheduling and Testing]--show-hidden-metrics-for-version in kubelet can be used to show all hidden metrics that deprecated in the previous minor release. (kubernetes/kubernetes#85282, @serathius) [SIG Node]feat: load kubeconfig from dict (#169, @tomplus)
The following deprecated APIs can no longer be served:
Important Information:
The library versioning scheme has been changed. Starting from this release, the library uses a version format vY.Z.P where Y and Z are respectively from the Kubernetes version v1.Y.Z and P would incremented due to changes on the library side itself. Ref: https://github.com/kubernetes-client/python/issues/1244
API Deprecations:
apps/v1beta1 and apps/v1beta2 - use apps/v1 insteaddaemonsets, deployments, replicasets resources under extensions/v1beta1 - use apps/v1 insteadnetworkpolicies resources under extensions/v1beta1 - use networking.k8s.io/v1 insteadpodsecuritypolicies resources under extensions/v1beta1 - use policy/v1beta1 instead (#85903, @liggitt) [SIG API Machinery, Apps, Cluster Lifecycle, Instrumentation and Testing]API Change:
spec.behavior field that allows scale behavior to be configured. Behaviors are specified separately for scaling up and down. In each direction a stabilization window can be specified as well as a list of policies and how to select amongst them. Policies can limit the absolute number of pods added or removed, or the percentage of pods added or removed. (#74525, @gliush) [SIG API Machinery, Apps, Autoscaling and CLI]spec.ingressClassName replaces the deprecated kubernetes.io/ingress.class annotation, and allows associating an Ingress object with a particular controller.pathType field to allow indicating how the specified path should be matched against incoming requests. Valid values are Exact, Prefix, and ImplementationSpecific (#88587, @cmluciano) [SIG Apps, Cluster Lifecycle and Network]AnyVolumeDataSource enables PersistentVolumeClaim objects to use the spec.dataSource field to reference a custom type as a data source (#88636, @bswartz) [SIG Apps and Storage]ConfigurableFSGroupPolicy enables v1 Pods to specify a spec.securityContext.fsGroupChangePolicy policy to control how file permissions are applied to volumes mounted into the pod. (#88488, @gnufied) [SIG Storage]ServiceAppProtocol enables setting an appProtocol field in ServicePort and EndpointPort definitions. (#88503, @robscott) [SIG Apps and Network]ImmutableEphemeralVolumes enables an immutable field in both Secret and ConfigMap objects to mark their contents as immutable. (#86377, @wojtek-t) [SIG Apps, CLI and Testing]ServerSideApply enables tracking and managing changed fields for all new objects, which means there will be managedFields in metadata with the list of managers and their owned fields.ServiceAccountIssuerDiscovery enables publishing OIDC discovery information and service account token verification keys at /.well-known/openid-configuration and /openid/v1/jwks endpoints by API servers configured to issue service account tokens. (#80724, @cceckman) [SIG API Machinery, Auth, Cluster Lifecycle and Testing]x-kubernetes-list-map-keys to specify properties that uniquely identify list items must make those properties required or have a default value, to ensure those properties are present for all list items. See https://kubernetes.io/docs/reference/using-api/api-concepts/#merge-strategy for details. (#88076, @eloyekunle) [SIG API Machinery and Testing]x-kubernetes-list-type: map or x-kubernetes-list-type: set now enable validation that the list items in the corresponding custom resources are unique. (#84920, @sttts) [SIG API Machinery]To read the full CHANGELOG visit here.
fix: handle 401 returned by GKE (#154, @tomplus)
fix: Watch() raises exceptions for received errors (#151, @tomplus)
feat: add function to create objects from dict (#143, @tomplus)
create_from_yaml_single_item import (#133, @VideoSystemsTech)fix: remove checking headers if not preloaded content is returned from rest api (#123, @tomplus)
…version of CustomResourceDefinition is deprecated and will no longer be served in v1.19. (#79604, @liggitt)
API Change:
MutatingWebhookConfiguration and ValidatingWebhookConfiguration APIs have been promoted to admissionregistration.k8s.io/v1:
failurePolicy default changed from Ignore to Fail for v1matchPolicy default changed from Exact to Equivalent for v1timeout default changed from 30s to 10s for v1sideEffects default value is removed, and the field made required, and only None and NoneOnDryRun are permitted for v1admissionReviewVersions default value is removed and the field made required for v1 (supported versions for AdmissionReview are v1 and v1beta1)name field for specified webhooks must be unique for MutatingWebhookConfiguration and ValidatingWebhookConfiguration objects created via admissionregistration.k8s.io/v1AdmissionReview API sent to and received from admission webhooks has been promoted to admission.k8s.io/v1. Webhooks can specify a preference for receiving v1 AdmissionReview objects with admissionReviewVersions: ["v1","v1beta1"], and must respond with an API object in the same apiVersion they are sent. When webhooks use admission.k8s.io/v1, the following additional validation is performed on their responses:
response.patch and response.patchType are not permitted from validating admission webhooksapiVersion: "admission.k8s.io/v1" is requiredkind: "AdmissionReview" is requiredresponse.uid: "<value of request.uid>" is requiredresponse.patchType: "JSONPatch" is required (if response.patch is set) (#80231, @liggitt)CustomResourceDefinition API type is promoted to apiextensions.k8s.io/v1 with the following changes:
default feature in validation schemas is limited to v1spec.scope is no longer defaulted to Namespaced and must be explicitly specifiedspec.version is removed in v1; use spec.versions insteadspec.validation is removed in v1; use spec.versions[*].schema insteadspec.subresources is removed in v1; use spec.versions[*].subresources insteadspec.additionalPrinterColumns is removed in v1; use spec.versions[*].additionalPrinterColumns insteadspec.conversion.webhookClientConfig is moved to spec.conversion.webhook.clientConfig in v1spec.conversion.conversionReviewVersions is moved to spec.conversion.webhook.conversionReviewVersions in v1spec.versions[*].schema.openAPIV3Schema is now required when creating v1 CustomResourceDefinitionsspec.preserveUnknownFields: true is disallowed when creating v1 CustomResourceDefinitions; it must be specified within schema definitions as x-kubernetes-preserve-unknown-fields: trueadditionalPrinterColumns items, the JSONPath field was renamed to jsonPath in v1 (fixes https://github.com/kubernetes/kubernetes/issues/66531)
The apiextensions.k8s.io/v1beta1 version of CustomResourceDefinition is deprecated and will no longer be served in v1.19. (#79604, @liggitt)ConversionReview API sent to and received from custom resource CustomResourceDefinition conversion webhooks has been promoted to apiextensions.k8s.io/v1. CustomResourceDefinition conversion webhooks can now indicate they support receiving and responding with ConversionReview API objects in the apiextensions.k8s.io/v1 version by including v1 in the conversionReviewVersions list in their CustomResourceDefinition. Conversion webhooks must respond with a ConversionReview object in the same apiVersion they receive. apiextensions.k8s.io/v1 ConversionReview responses must specify a response.uid that matches the request.uid of the object they were sent. (#81476, @liggitt)conditions in apiextensions.v1beta1.CustomResourceDefinitionStatus and apiextensions.v1.CustomResourceDefinitionStatus is now optional instead of required. (#64996, @roycaihw)lastTransitionTime is now updated. (#69655, @CaoShuFeng)fix: watch closes http session (#104, @tomplus)
feat: regenerate client against openapi-generator v4.3.0 (context-manager and close function for http client instead of using __del__ method) (#99, @j
__del__ method)
(#99, @jnschaeffer)feat: add compatability for follow methods (#98, @playground-julia)
feat: regenerate library using the latest version of openapi-generator (4.3.x) (gen/#146, @tomplus)
API Change:
ExtensionsV1beta1RuntimeClassStrategyOptions and PolicyV1beta1RuntimeClassStrategyOptions. Add RuntimeClass restrictions & defaulting to PodSecurityPolicy kubernetes/kubernetes#73795V1WindowsSecurityContextOptions. Add Windows specific options in Pod Security Context and Container Security Context kubernetes/kubernetes#77147V1beta1Webhook into V1beta1MutatingWebhook and V1beta1ValidatingWebhook kubernetes/kubernetes#78491allow_watch_bookmarks in list options for requesting watch bookmarks from apiserver. The implementation in apiserver is hidden behind feature gate WatchBookmark (currently in Alpha stage) kubernetes/kubernetes#74074V1DeleteOptions parameters (dry_run, grace_period_seconds, orphan_dependents, propagation_policy) to delete collection APIs kubernetes/kubernetes#77843controller_expand_secret_ref in V1SecretReference to store CSI volume expansion secrets kubernetes/kubernetes#77516preemption_policy field to V1PriorityClass kubernetes/kubernetes#74614port configuration to service reference in Admission webhook configuration, AuditSink webhook configuration, CRD Conversion webhook configuration and kube-aggregator kubernetes/kubernetes#74855inline_volume_spec to V1PersistentVolumeSpec kubernetes/kubernetes#77703x_kubernetes_embedded_resource, x_kubernetes_int_or_string, x_kubernetes_preserve_unknown_fields to V1beta1JSONSchemaProps kubernetes/kubernetes#77207fix: when _preload_content=False Websocket Stream returns 401 error (#84, @kexirong)
The Ingress API is now available via NetworkingV1beta1Api. ExtensionsV1beta1Api Ingress objects are deprecated and will no longer be served in Kuberne…
create_form_yaml() functionality (#76, @PidgeyBE)API Change:
NetworkingV1beta1Api. ExtensionsV1beta1Api Ingress objects are deprecated and will no longer be served in Kubernetes v1.18 kubernetes/kubernetes#74057admissionregistration.k8s.io/v1alpha1 API version, Initializers admission plugin, and use of the metadata.initializers API field have been removed. Discontinue use of the alpha feature and delete any existing InitializerConfiguration API objects before upgrading. The metadata.initializers field will be removed in a future release. The parameter include_uninitialized has been removed. kubernetes/kubernetes#72972feat: check whether an object key is present on watch (#71, @mickours)
feat: switch to openapi-generator (#58, @tomplus)
Breaking Changes:
feat: watch improvement (context manager, close method) (#61, @hubo1016)
fix: use yaml.safe_load, yaml.safe_dump for security reasons (#57, @tomplus)
feat: remove dependency to urllib3 from kube_config, pin urlllib>=1.23 due to CVE-2018-20060 (#56, @tomplus)
fix: kubeconfig loading failure when server uri contains trailing slash (#53, @tomplus)
feat: upgrade to API spec from Kubernetes 1.12 (#50, @tomplus)
* first stable release 7.0.0
fix: option verify-ssl impacts on server certs only (#46, @tomplus)
feat: add OIDC auth support (#36, @bpicolo)
feat: regenerate with latest client gen to get custom object status scale api (#37, @juliantaylor)
feat: upgrade to spec from Kuberentes 1.11 (#34, @tomplus)
fix: aiohttp with verify_ssl=False (#33, @bpicolo)
feat: watch work forever if timeout is not specified (#30, @tomplus)
feat: support Python 3.7 (#28, @tomplus)
feat: make function load_kube_config asynchronous
load_kube_config asynchronousfeat: watch stops the iterator for empty responses and do not process ERROR responses (#22, @olitheolix)
* fix e2e and unit tests * fix Watch, Stream
* fix requirements.txt
Your coding agent can read these notes before it upgrades. Set up the MCP server →