kube-runtime

What's Changed

Maintenance release for 4.0.0. Fixes, and https proxy. MSRV 1.89. All PRs here.

Added

• add support for https proxy by @goenning in #2013

Fixed

• Update kube-cel requirement from 0.7.0 to 0.8.0 by @dependabot[bot] in #2017
• client: include sources in CommitError's error message by @iniw in #2020
• Include error sources in Display messages at client, core, runtime by @doxxx93 in #2021
• Remove extraneous Sync bound in Controller::reconcile_all_on by @bojidar-bg in #2029
• client: fall back to OS-native cert store via rustls-platform-verifier by @aviramha in #2030
• Avoid deep-cloning ObjectMeta per event in trigger_owners by @doxxx93 in #2039
• config: resolve relative exec plugin command paths by @deepu105 in #2037
• Construct typed CustomResourceDefinition in derived crd() by @doxxx93 in #2042

New Contributors

• @iniw made their first contribution in #2020
• @bojidar-bg made their first contribution in #2029
• @deepu105 made their first contribution in #2037

Full Changelog: 4.0.0...4.2.0

New Major

As per the release schedule to match up with the latest Kubernetes ハル release.
Lots of fixes and improvements. Thanks to everyone who contributed!

Kubernetes v1_36 support via k8s-openapi 0.28

Please upgrade k8s-openapi along with kube to avoid conflicts.

CEL Validation

A new optional crate kube-cel is being re-exported through kube::core::cel via #1954

Kubernetes CRDs support CEL validation rules via x-kubernetes-validations, and were supported from 3.0 via KubeSchema, but these rules could only be evaluated server-side by the API server.

The new crate allows evaluating these rules locally using rules matching the upstream Kubernetes CEL libraries.

While low-level, a higher-level CEL validator integrates with CustomResource via #[kube(cel)] from #2011 and can be used as;

#[derive(CustomResource, Serialize, Deserialize, Clone, KubeSchema)]
#[kube(group = "example.com", version = "v1", kind = "Foo", namespaced)]
#[kube(cel, validation = "self.spec.replicas >= 0")] // cel trigger + validation rule
struct FooSpec { replicas: i32 }

let foo = Foo::new("test", FooSpec { replicas: -1 });
foo.validate_cel()?;                     // new impl; checks creation rules
new_foo.validate_cel_update(&old_foo)?;  // new impl; checks transition rules

See examples/crd_derive_cel.rs for more details.

This is available under the kube/cel feature, courtesy of @doxxx93.

Config

A lot of improvements to config handling;

• better error handling of malformed client certs in #1966
• add missing Kubeconfig fields in #1965
• Kubeconfig future key compatibility for new fields by adding catch-all other key via #1964
• deserialization changed from serde-yaml to serde-saphyr to get rid of the long-deprecated dependency. #1975

Retry and Timeouts

Better timeout and retry handling to better deal with flaky network conditions, and busy or initializing apiservers.

• default global read timeouts has been unset in favor of watcher level timeouts in #1945 (see #1798 for context)
• regular (non-watch) queries now respect the RetryPolicy - now enabled by default in #2007.

Client

• properly handling rotating ca certs in cluster via #1962
• handle tls-server-name with openssl-tls via #1993
• auth exec: accept yaml output from exec plugins via #2003
• fix ws task leak and drop, and a deadlock on join() via #1978
• change: client tracing now opt-in due to issues. see #1972

Runtime

• watcher automatically uses the metadata_ api methods when called with PartialObjectMeta<K> via #1952
  • (this deprecates metadata_watcher in favor of an explicit change from Api::<K> to Api::<PartialObjectMeta<K>>)
• added wait::conditions::is_created as a counter to is_deleted #2000
• added Store::state_filtered and Store::state_filter_selector to allow more efficient slicing of the locked cache via #2002 + #1998

What's Changed

Added

• feat: add typed kubeconfig fields for client-go parity by @alex-lapuka in #1965
• Add CEL validation via kube-cel re-export by @doxxx93 in #1954
• Add AdmissionRequest::to_cel_request() for VAP CEL bridging by @doxxx93 in #1991
• runtime: implement Store::state_with and Store::state_filtered by @Alvov1 in #1998
• runtime: add wait::conditions::is_created helper by @orangecms in #2000
• refactor(runtime): rename Store::state_with/state_filtered per review feedback by @Alvov1 in #2002
• deps: bump kube-cel to 0.6.1 (validation surface flattened) by @doxxx93 in #2005
• Enable RetryPolicy::server_retry by default for Client by @Danil-Grigorev in #2007
• feat(derive): client-side CEL validation via #[kube(cel)] / #[x_kube(cel)] by @doxxx93 in #2011

Changed

• preserve unknown kubeconfig fields via serde(flatten) by @alex-lapuka in #1964
• Remove global read_timeout default, add watcher-level idle timeout by @doxxx93 in #1945
• Update tokio-tungstenite requirement from 0.28.0 to 0.29.0 by @dependabot[bot] in #1963
• convert from serde-yaml to serde-saphyr by @clux in #1975
• features: making client tracing opt-in by @mattklein123 in #1972
• client: reload in-cluster CA bundle on rotation (rustls-tls) by @chrnorm in #1962
• Api<PartialObjectMeta> should opportunistically degrade to metadata requests by @doxxx93 in #1952
• Chore(deps): Update garde requirement from 0.22.0 to 0.23.0 by @dependabot[bot] in #1989
• bump k8s-openapi to 0.28 by @clux in #2009
• Box a large runtime error in ReconcilerErr by @clux in #1880

Fixed

• fix: feature-flag CREATE_NO_WINDOW to not break stderr inheritance by @cristeigabriela in #1971
• Remove silent error when client-key/client-certificate is malformed by @goenning in #1966
• Fix AttachedProcess task leak on drop and join() deadlock by @SebTardif in #1978
• support auth exec yaml output by @aviramha in #2003
• fix(client): apply tls-server-name on the openssl-tls path by @dgunzy in #1993
• Use the resource's own name for the schema title by @cehoffman in #1985

New Contributors

• @alex-lapuka made their first contribution in #1965
• @cristeigabriela made their first contribution in #1971
• @mattklein123 made their first contribution in #1972
• @chrnorm made their first contribution in #1962
• @SebTardif made their first contribution in #1978
• @Alvov1 made their first contribution in #1998
• @orangecms made their first contribution in #2000
• @dgunzy made their first contribution in #1993
• @cehoffman made their first contribution in #1985

Full Changelog: 3.1.0...4.0.0

What's Changed

Maintenance release with fixes for schemas/validation, client exec blocking and proxy handling, as well as some smaller new features listed below. Internal changes and documentation improvements listed in the milestone.

Added

• allow empty body in const context by @tottoto in #1927
• Add typed printcolumn argument to derive macro by @cchndl in #1872
• kube-core: add optionalOldSelf to CEL Rule by @Immortal-Beyond-Oblivion in #1947
• Re-add support for basic auth in Proxy by @goenning in #1959

Fixed

• Fix OptionalEnum transform for complex enums by @doxxx93 in #1934
• Bump tower-http dependency to 0.6.4 by @abustany in #1939
• Add minimal-versions CI check by @doxxx93 in #1940
• fix: Produce valid CRDs containing flattened untagged enums by @NickLarsenNZ in #1942
• kube-client: Avoid blocking tokio worker during exec auth token refresh by @blakelawson in #1950

New Contributors

• @abustany made their first contribution in #1939
• @cchndl made their first contribution in #1872
• @Immortal-Beyond-Oblivion made their first contribution in #1947
• @gauravgahlot made their first contribution in #1949
• @blakelawson made their first contribution in #1950

Full Changelog: 3.0.1...3.1.0

What's Changed

Bugfix release for schemas, admission, and docs. Minor internal improvements listed in the milestone. Important fixes below.

Fixed

• Update API version of AdmissionResponse created via invalid call by @Magicloud in #1905
• Fix OptionalEnum transform skipping schemas with description by @doxxx93 in #1908
• Remove conflicting additionalProperties: false from schema by @doxxx93 in #1920

New Contributors

• @phenomenes made their first contribution in #1900
• @Magicloud made their first contribution in #1905

Full Changelog: 3.0.0...3.0.1

New Major

As per the new release schedule to match up with the new Kubernetes release.
Lots of additions, fixes and improvements. Thanks to everyone who contributed so heavily over the holidays! Happy new year.

Breaking Changes

Kubernetes v1_35 support via k8s-openapi 0.27

Please upgrade k8s-openapi along with kube to avoid conflicts.

jiff replaces chrono

Matching k8s-openapi's change, kube has also swapped out chrono. The biggest impact of this is for interacting with timestamps in metadata, but it also updates 2 smaller public interfaces in LogParams, Client::with_valid_until. See controller-rs#217 for an example change.

Changes: #1868 + #1870

ErrorResponse has been replaced with Status

ErrorResponse served as a partial metav1/Status replacement which ended up hiding error information to users. These structs have merged, more information is available on errors, and a type alias with a deprecation warning is in place for ErrorResponse which will be removed in a later version.

This creates a small breaking change for users matching on specific Error::Api codes;

     .map_err(|error| match error {
-        kube::Error::Api(kube::error::ErrorResponse { code: 403, .. }) => {
-            Error::UnauthorizedToPatch(obj)
-        }
+        kube::Error::Api(s) if s.is_forbidden() => Error::UnauthorizedToPatch(obj),
         other => Error::Other(other),
     })?;

#1875 + #1883 + #1891.

Predicates now has a TTL Cache

This prevents unbounded memory for controllers, particularly affecting ones watching quickly rotating objects with generated names (e.g. pods). By default the TTL is 1h. It can be configured via new PredicateConfig parameter. To use the default;

-        .predicate_filter(predicates::resource_version);
+        .predicate_filter(predicates::resource_version, Default::default());

Change in #1836. This helped expose and fix a bug in watches with streaming_lists now fixed in #1882.

Subresource Api

Some subresource write methods were public with inconsistent signatures that required less ergonomic use than any other write methods. They took a Vec<u8> for the post body, now they take a &K: Serialize or the actual subresource.
There affect Api::create_subresource, Api::replace_subresource, Api::replace_status, Api::replace_scale. In essence this generally means you do not have to wrap raw objects in json! and serde_json::to_vec for these calls and lean more on rust's typed objects rather than json! blobs which has some footguns for subresources.

-    let o = foos.replace_status("qux", &pp, serde_json::to_vec(&object)?).await?;
+    let o = foos.replace_status("qux", &pp, &object).await?;

See some more shifts in examples in the implementaion; #1884

Improvements

Support Kubernetes 1.30 Aggregated Discovery

Speeds up api discovery significantly by using the newer api with much less round-tripping.
To opt-in change Discovery::run() to Discovery::run_aggregated()

Changes; #1876 + #1873 + #1889

New Client RetryPolicy opt-in

Allows custom clients (for now) to enable exponential backoff'd retries for retryable errors by exposing a tower::retry::Policy for a tower::retry::Layer. See the new custom_client_retry example for details.

Enabled by a clonable body + the new RetryPolicy based on mirrord's solution*.

Rust 2024

While this is mostly for internal ergonomics, we would like to highlight this also simplifies the Condition implementors which had to deal with a lot of options;

    pub fn is_job_completed() -> impl Condition<Job> {
        |obj: Option<&Job>| {
-            if let Some(job) = &obj {
-                if let Some(s) = &job.status {
-                    if let Some(conds) = &s.conditions {
-                        if let Some(pcond) = conds.iter().find(|c| c.type_ == "Complete") {
-                            return pcond.status == "True";
-                        }
-                    }
-                }
+            if let Some(job) = &obj
+                && let Some(s) = &job.status
+                && let Some(conds) = &s.conditions
+                && let Some(pcond) = conds.iter().find(|c| c.type_ == "Complete")
+            {
+                return pcond.status == "True";

Change #1856 + #1792

Fixes

• streaming list bug fix - #1882
• watcher jitter bug fix - #1897
• schema fix for IntOrString - #1867
• schema fix for optional enums - #1853
• websocket keepalives for long running attaches - #1889

More

• Resize subresource impl for Pod - #1851
• add #[kube(attr="...") to allow custom attrs on derives - #1850
• example updates for admission w/axum - #1859

What's Changed

Added

• Add Resize subresource for Pod by @hugoponthieu in #1851
• feat: add #[kube(attr="...")] attribute helper to set attribute helper on the CR root type by @ngergs in #1850
• Rust 2024 let-chains to simplify wait Conditions by @clux in #1792
• Adds a try_clone method for kube_client::client::Body when it's Kind::Once by @meowjesty in #1867
• Implement client aggregated discovery API methods by @doxxx93 in #1873
• Implement aggregated discovery API methods by @doxxx93 in #1876
• Permit older version of API for v2 discovery for k8s < 1.30 (down to 1.27) by @Danil-Grigorev in #1889
• Add RetryPolicy for client-level request retries by @doxxx93 in #1894

Changed

• Update tokio-tungstenite requirement from 0.27.0 to 0.28.0 by @dependabot[bot] in #1829
• Predicates: add configurable cache TTL for predicate_filter by @doxxx93 in #1838
• Update darling requirement from 0.21.0 to 0.23.0 by @dependabot[bot] in #1861
• update to jsonpath-rust 1 by @tottoto in #1863
• Replace chrono with jiff by @ngergs in #1868
• Merge ErrorResponse and Status by @imp in #1875
• Make subresource methods more ergonomic by @doxxx93 in #1884
• Drop k8s v1.30, crono->jiff replacement for runtime & examples by @ngergs in #1870
• Add a metadata field to Status by @ryanpbrewster in #1891
• Bump k8s-openapi for Kubernetes 1.35 by @clux in #1898

Fixed

• fix: add use<> to kubelet_node_logs for rust edition 2024 by @co42 in #1849
• Transform optional enums to match pre kube 2.0.0 format by @Danil-Grigorev in #1853
• Distinguish between initial and resumed watch phases for streaming lists by @doxxx93 in #1882
• Re-export deprecated type alias and improve deprecation guidance by @clux in #1883
• Add nullable to optional fields with x-kubernetes-int-or-string by @doxxx93 in #1885
• send websocket ping to keep idle connections alive by @inqode-lars in #1887
• Fix watcher ExponentialBackoff jitter ignored by @doxxx93 in #1897

New Contributors

• @doxxx93 made their first contribution in #1836
• @elisaado made their first contribution in #1845
• @hugoponthieu made their first contribution in #1851
• @ngergs made their first contribution in #1850
• @meowjesty made their first contribution in #1867
• @inqode-lars made their first contribution in #1887
• @ryanpbrewster made their first contribution in #1891
• @uchiha-vivek made their first contribution in #1895

Full Changelog: 2.0.1...3.0.0

What's Changed

Fixes an accidental inclusion of a constraint added to Api::log_stream introduced in the 2.0.0 Rust 2024 upgrade.

Fixed

• Remove unused use<'_> from log_stream() by @pgerber in #1824

New Contributors

• @pgerber made their first contribution in #1824

Full Changelog: 2.0.0...2.0.1

Kubernetes v1_34 support via k8s-openapi 0.26

Please upgrade k8s-openapi along with kube to avoid conflicts.

Schemars 1.0

A fairly significant upgrade in #1780. Our external facing API should be unchanged, although some schemars public import paths have changed. Note that if you are implementing schemars traits directly, then see the upstream schemars/migrating (and maybe consider using KubeSchema for relevant schema overrides).

Please upgrade schemars along with kube for this version to avoid conflicts.

New Minimums

Minimum versions: MSRV 1.85.0 (for edition 2024), MK8SV: 1.30 (unchanged).

Highlights

This version is contains fixes, dependency clearups, and dependency updates. Noteworthy additions are TryFrom impls for Kubeconfig users in #1801, and a namespace accessor in Api in #1788

New Major

A new semver major for unstable, public facing dependency updates. As per the new release cycle, it is aligned with the Kubernetes release.

What's Changed

Added

• Add TryFrom conversions for Kubeconfig -> Config -> Client by @Danil-Grigorev in #1801
• Add pub fn namespace(&self) -> Option<&str> to Api by @tgrushka in #1788

Changed

• Update to schemars 1.0 by @Danil-Grigorev in #1780
• Bump Rust Edition to 2024 and MSRV to 1.85 by @clux in #1785
• Replace hyper-socks2 with hyper-util client-proxy feature by @tottoto in #1795
• Bump k8s-openapi to 0.26.0 by @clux in #1817

Fixed

• Clamp scheduling delay to 6 months by @dervoeti in #1779
• Update admission example and pin to a local crd by @clux in #1782
• Fix interactive auth mode to allow prompt messages by @gememma in #1800
• Make kube::runtime::controller::Action ctors const by @imp in #1804
• Fix oidc with openssl by @saif-88 in #1807

New Contributors

• @dervoeti made their first contribution in #1779
• @tgrushka made their first contribution in #1788
• @gememma made their first contribution in #1800
• @saif-88 made their first contribution in #1807

Full Changelog: 1.1.0...2.0.0

What's Changed

Missing attribute bugfix + extra standard derives on core::conversion structs.

Added

• Add missing derives on conversion types by @sbernauer in #1759

Fixed

• Emit #[schemars(crate)] attribute by @Techassi in #1764

Full Changelog: 1.0.0...1.1.0

A Major Version

It's been a long time coming, but time has come to draw the line in the sand. No alphas, no betas. Hope it finds you all well. Thanks to everyone who has contributed over the years.

This is a somewhat symbolic gesture, because semver-breaking changes are still hard to avoid with a large set of sub-1.0 dependencies we need to bump, as well as managing the large api surface of Kubernetes.

Therefore, the plan is to align our breaking changes and major bumps with Kubernetes versions / k8s-openapi versions for now, and this should allow our other releases to stream in. See #1688 for more information.

Kubernetes v1_33 support via k8s-openapi 0.25

Please upgrade k8s-openapi along with kube to avoid conflicts.

New minimum versions: MSRV 1.82.0, MK8SV: 1.30*

KubeSchema

The CELSchema alternate derive for JsonSchema has been renamed to KubeSchema to indicate the increased functionality.

In addition to being able to inject CEL rules for validations, it can now also inject x-kubernetes properties such as merge-strategy via #1750, handle #[validate] attributes #1749, and pass validation rules as string literals #1754 :

#[derive(CustomResource, Serialize, Deserialize, Debug, PartialEq, Clone, KubeSchema)]
#[kube(...properties)
struct DocumentSpec {
    /// New merge strategy support
    #[x_kube(merge_strategy = ListMerge::Set)]
    x_kubernetes_set: Vec<String>,

    /// CEL Validation now lives on x_kube and supports literal Rules:
    #[x_kube(validation = "!has(self.variantOne) || self.variantOne.int > 22")]
    complex_enum: ComplexEnum,
}

See kube.rs docs on validation for more info. Huge thanks to @Danil-Grigorev.

What's Changed

Added

• feat(deps): enable hyper-util/tracing feature flag by @cratelyn in #1734
• Permit literal string validation for CEL expressions by @Danil-Grigorev in #1754

Changed

• Support additional x-kubernetes-* schema extensions by @Danil-Grigorev in #1750
• Bump k8s-openapi to 0.25.0 by @clux in #1756

Removed

• Remove deprecated watcher::Event into_iter_* methods by @clux in #1738

Fixed

• docs: Adjust #[kube(scale(...)] doc example by @Techassi in #1733
• Add suffix to generated struct by CELSchema by @Danil-Grigorev in #1747
• Allow schemars validate attribute in CELSchema by @Danil-Grigorev in #1749
• fix: resolve conflict with schemars preserve_order feature by @HoKim98 in #1758

New Contributors

• @cratelyn made their first contribution in #1734
• @JanBerktold made their first contribution in #1752

Full Changelog: 0.99.0...1.0.0

Highlights

Dependency Cleanups

• backoff (unmaintained) replaced with backon in #1653
  • No change if you are using default_backoff natively, or through Controller.
  • Parameters configurable via ExponentialBackoff from backon::ExponentialBuilder into WatchStreamExt::backoff
• json-patch bumped and uses re-exported jsonptr for less version clashes #1718
• rand dependency no longer explicit as only rng is under ws feature via tungstenite's client::generate_key #1691
• ring (still maintained) now optional for rustls-tls feature (for alternate aws-lc-rs) #1717

Features

• Support for the v5.channel.k8s.io streaming ws protocol to allow closing streams properly (kubernetes.io blog) #1693
• CustomResource derive; typed attributes for #[kube(scale)] and #[kube(deprecated)] in #1656 + #1697
• Client::with_valid_until to handle short lived local client certs #1707
• New common conditions that can be awaited #1710

What's Changed

Added

• Add typed scale argument to derive macro by @Techassi in #1656
• Add deprecated argument to derive macro by @Techassi in #1697
• Add Api::get_metadata_opt_with by @sebsoto in #1708
• Add common wait conditions for Deployments, LoadBalancer Services, and Ingress by @detjensrobert in #1710
• Add Client::with_valid_until for client cert expiry by @goenning in #1707
• kube-runtime: make ExponentialBackoff public by @gdeleon2 in #1716

Changed

• Replace backoff with backon by @flavio in #1653
• Bump rand to 0.9 by @clux in #1686
• Remove rand dependency in favor of tungstenite fn by @clux in #1691
• Exec can return stdout data even after stdin is closed. by @esw-amzn in #1693
• Bump json-patch to 4 use bundled jsonptr to 0.7 by @clux in #1718
• Allow removing hyper-rustls/ring feature by @eliad-wiz in #1717

Fixed

• kube-runtime: fix exponential backoff max times by @eliad-wiz in #1713
• CustomResource derive; allow status attribute to take a path by @clux in #1704

New Contributors

• @esw-amzn made their first contribution in #1693
• @sebsoto made their first contribution in #1708
• @zhanluxianshen made their first contribution in #1709
• @detjensrobert made their first contribution in #1710
• @gdeleon2 made their first contribution in #1716

Full Changelog: 0.98.0...0.99.0

This tag was signed with the committer’s verified signature . The key has expired.

clux Eirik A

GPG key ID: 5D4B685DE5BEAE01 Expired

Verified Learn about vigilant mode .