apache-airflow-providers-cncf-kubernetes

Release Date: 2025-06-18

Features

KubernetesPodOperator push xcom after failed pod (#51475)

[KubernetesPodOperator] Reads Kubernetes events and writes them into log (#50192)

Added detection of terminated pod during get_or_create_pod (#51158)

Bug Fixes

Add xcom sidecar terminated detection (#51144)

Fix 'EksPodOperator' in deferrable mode (#51255)

Fix a bug where Kube config "worker_pod_pending_fatal_container_state_reasons" is parsed wrongly (#50931)

Misc

Relax Kubernetes client version support to <33.0.0 (#51195)

Adjust code path in executors to be compatible with airflow task run (#51009)

Port ''ti.run'' to Task SDK execution path (#50141)

Bump some provider dependencies for faster resolution (#51727)

Prevent legacy static hybrid executors to be running in Airflow 3 (#51733)

Doc-only

Update the executor and provider doc to highlight the two statically coded hybrid executors are no longer supported in Airflow 3.0.0+ (#51715)

Update operators.rst - fix typo Kubernetes Pod Operator doc (#51574)

Release Date: 2025-05-21

Note

This release of provider is only available for Airflow 2.10+ as explained in the Apache Airflow providers support policy < https://github.com/apache/airflow/blob/main/PROVIDERS.rst#minimum-supported-version-of-airflow-for-community-managed-providers >_.

Features

Add fast fail for ErrImagePull and InvalidImageName for KubernetesPodOperator (#49867)

KubernetesPodOperator uses different timeouts to check for schedule timeout and startup timeout (#49784)

Add configurable automountServiceAccountToken for the KubernetesPodOperator (#50223)

Add 'test_connection' method to 'KubernetesHook' (#47881)

Add '@task.kuberenetes_cmd' (#46913)

Bug Fixes

Pass workload to KubernetesExecutor using command args instead of init container (#50448)

Fix KubernetesPodOperator AutomountServiceAccountToken from Pod Template (#50800)

Fix broken imports of cncf.kubernetes for Airflow 2 (#50651)

Misc

Move SQS message queue to Amazon provider (#50057)

Remove AIRFLOW_2_10_PLUS conditions (#49877)

Bump min Airflow version in providers to 2.10 (#49843)

Release Date: 2025-04-19

Bug Fixes

Remove 'subdir' arg from CLI commands (#49317)

Misc

Use contextlib.suppress(exception) instead of try-except-pass and add SIM105 ruff rule (#49251)

remove superfluous else block (#49199)

Remove unused db method in k8s provider (#49186)

Release Date: 2025-04-14

Misc

Make '@task' import from airflow.sdk (#48896)

Update ECS executor to support Task SDK (#48513)

Release Date: 2025-04-09

Bug Fixes

Handle missing container_statuses in pod_manager.get_container_status (#47936)

Misc

Remove change_sensor_mode_to_reschedule from base executor (#48649)

No need to exclude executor_config specifically for KE (#48826)

Release Date: 2025-03-31

Features

KubernetesPodOperator: add base_container_name to the templated fields (#47864)

Bug Fixes

cncf.kubernetes: make the base container status check polling interval configurable (#47774)

Switch to non-interactive mode where stdin is disabled (#47568)

Retry k8s API requests in 'KubernetesPodTrigger' (#47187)

Misc

Remove extra method for async hook getting (#47313)

Release Date: 2025-03-13

Misc

Get rid of google-re2 as dependency (#47493)

AIP-72: Port over executor_config for K8sExecutor when using task sdk (#46926)

Update templated fields in KPO doc-string + wording (#46970)

Upgrade flit to 3.11.0 (#46938)

Release Date: 2025-02-26

Note

This version contains no code changes. It was released to replace a previous version that was yanked due to a packaging issue.

Release Date: 2025-02-21

Features

KubernetesPodOperator new callbacks and allow multiple callbacks (#44357)

Bug Fixes

(fix): Render a pod spec using the pod_template_file override, if passed to the executor (#46374)

fix: kpo async kube_config_path (#45571)

Allow passing empty labels in the spark kubernetes driver config (#45864)

fix: spark operator label (#45353)

Allow internal retries when pending k8s pod is deleted (#45184)

fix: correctly shutdown xcom after job is complete (#45455)

Fix kubernetes executor watcher kube_client_request_args shadowing (#45528)

Misc

Fix wrong operator name in doc string of @task.kubernetes (#46873)

AIP-83 amendment: Add logic for generating run_id when logical date is None. (#46616)

Remove overlooked 'pod_generator_deprecated.py' (#46587)

More meaningful '@task.kubernetes' pod naming (#46535)

chore(kubernetes): reduce redundant code (#46574)

Add dynamic task mapping into TaskSDK runtime (#46032)

Expand KPO template_fields, fix Spark k8s operator tests (#46268)

Swap CeleryExecutor over to use TaskSDK for execution. (#46265)

Move Pod*Exceptions to separate module (#45759)

Add other instances of Context type hints (#45657)

AIP-72: Support better type-hinting for Context dict in SDK (#45583)

AIP-72: Swap KubernetesExecutor to use taskSDK for execution (#46860)

Release Date: 2024-12-26

Note

This release of provider is only available for Airflow 2.9+ as explained in the Apache Airflow providers support policy .

Features

Add logging support for init containers in KubernetesPodOperator (#42498) (#43853)

Implement KubernetesInstallKueueOperator + (#44568)

support grouping of log lines for KubernetesPodOperator (#44428)

Bug Fixes

Compare k8s executor against alias, not full ExecutorName repr (#44967)

Fix failing KubernetesExecutor (#44931)

Misc

Remove references to AIRFLOW_V_2_9_PLUS (#44987)

Bump minimum Airflow version in providers to Airflow 2.9.0 (#44956)

Consistent way of checking Airflow version in providers (#44686)

Fix yoda-conditions (#44466)

Release Date: 2024-11-27

Bug Fixes

Bugfix KubernetesJobOperator.on_kill() by removing unexpected 'job' argument from the k8s job client method 'delete_namespaced_job' invocation (#44131)

Re-queue tassk when they are stuck in queued (#43520)

Misc

Rename execution_date to logical_date across codebase (#43902)

Release Date: 2024-11-18

Breaking changes

Warning

All deprecated classes, parameters and features have been removed from the Kubernetes provider package. The following breaking changes were introduced:

• Helpers

Remove add_pod_suffix method from kubernetes_helper_functions.py . Use add_unique_suffix instead.

Remove make_unique_pod_id method from PodGenerator . Use add_unique_suffix in kubernetes_helper_functions instead.

Remove create_pod_id method from kubernetes_helper_functions.py . Use create_unique_id instead.

Remove gen_pod method from PodGenerator .

Remove add_xcom_sidecar method from PodGenerator . Use airflow.providers.cncf.kubernetes.utils.xcom_sidecar.add_xcom_sidecar instead.

Remove the option to using a dictionary for the executor_config from_obj function in PodGenerator . Use a kubernetes.client.models.V1Pod class with a “pod_override” key.

Remove from_legacy_obj method from PodGenerator .

Remove airflow.providers.cncf.kubernetes.pod_launcher_deprecated module. Use airflow.providers.cncf.kubernetes.utils.pod_manager instead.

• Operators

Remove airflow.providers.cncf.kubernetes.operators.kubernetes_pod . Use airflow.providers.cncf.kubernetes.operators.pod instead.

Remove is_delete_operator_pod parameters from KubernetesPodOperator . Use on_finish_action instead.

Remove progress_callback parameters from KubernetesPodOperator . Use callbacks instead.

Remove execute_complete method from KubernetesPodOperator . Use trigger_reentry instead.

Remove xcom_push parameter from SparkKubernetesOperator . Use do_xcom_push .

• Triggers

Remove should_delete_pod parameter from KubernetesPodTrigger . Use on_finish_action instead.

• Utils

Remove progress_callback parameter from PodManager .

Remove follow_container_logs method from PodManager . Use fetch_container_logs instead.

Warning

Set the default value of namespace in @task.kubernetes to None , so it uses the cluster namespace when in_cluster is True. Be sure to specify a namespace when using this decorator. To retain the previous behavior, set namespace="default"

Remove deprecations cncf.kubernetes (#43689)

Change default value of 'namespace' in 'task.kubernetes' to be None (#43402)

Bug Fixes

Add random_name_suffix to SparkKubernetesOperator (#43847)

terminate kubernetes watch in case of unknown error (#43645)

Update 'namespace' priority for 'find_pod' function (#43762)

Misc

AIP-72: Remove DAG pickling (#43667)

Release Date: 2024-10-31

Bug Fixes

(fix): HybridExecutor tasks of other executor rescheduled in kubernetes executor (#43003)

(fix): use instance base_container_name to fetch logs on trigger_reentry (#42960)

Misc

Add kubernetes_conn_id to templated fields (#42786)

Allow node_selector templating in KPO (#43051)

Bump max version of Kubernetes Python SDK to 31.0.0 (#42907)

Release Date: 2024-10-14

Breaking changes

Warning

Support for identifying pods by execution_date during the upgrade from Airflow 1 to 2 has been removed. This may result in duplicate pods being launched for tasks originally started by Airflow 1, but only one of the task pods will succeed.

kubernetes executor cleanup_stuck_queued_tasks optimization (#41220)

Bug Fixes

All executors should inherit from BaseExecutor (#41904)

Fix mark as success when pod fails while fetching log (#42815)

Fix SparkKubernetesOperator spark name. (#42427)

KubernetesPodOperator never stops if credentials are refreshed (#42361)

Restructured 'await_xcom_sidecar_container_start' method. (#42504)

KubernetesHook kube_config extra can take dict (#41413)

Misc

Drop python3.8 support core and providers (#42766)

Remove airflow_version from k8s executor pod selector (#42751)

Release Date: 2024-09-24

Misc

Deprecated configuration removed (#42129)

docstring update in AsyncKubernetesHook (#41929)

Remove Airflow 2.7 support from Kubernetes Provider (#41746)

remove deprecated soft_fail from providers (#41710)

Release Date: 2024-08-28

Bug Fixes

fix: 'KubernetesExecutor' failing the task in case the watcher receives an event with the reason ProviderFailed (#41186)

fix: 'do_xcom_push' and 'get_logs' functionality for KubernetesJobOperator (#40814)

fix: 'KubernetesHook' loading config file with '_is_in_cluster' set as False (#41464)

fix: Missing 'slots_occupied' in 'CeleryKubernetesExecutor' and 'LocalKubernetesExecutor' (#41602)

Release Date: 2024-08-22

Note

This release of provider is only available for Airflow 2.8+ as explained in the Apache Airflow providers support policy .

Misc

Bump minimum Airflow version in providers to Airflow 2.8.0 (#41396)

Describe behaviour in docstring correctly (#41458)

Remove deprecated SubDags (#41390)

reorder docstring of 'SparkKubernetesOperator' (#41372)

Release Date: 2024-08-06

Bug Fixes

Pass content of kube/config file to triggerer as a dictionary (#41178)

Fix confusing log message in kubernetes executor (#41035)

Fix ApiException handling when adopting completed pods (#41109)

Release Date: 2024-07-12

Bug Fixes

Solve failing KPO task with task decorator and imported typing elements (#40642)

Misc

Release Date: 2024-06-27

Bug Fixes

Fix Scheduler restarting due to too many completed pods in cluster (#40183)

Misc

Bump minimum kubernetes lib version to kubernetes 29.0.0 (#40253)

Release Date: 2024-06-09

Bug Fixes

Fixes KubernetesPodTrigger failing running pods with timeout (#40019)

Refresh properties on KubernetesPodOperator on token expiration also when logging (#39789)

Fix reattach_on_restart parameter for the sync mode (#39329)

Avoid resetting adopted task instances when retrying for kubernetes executor (#39406)

Misc

Include fatal reason for pod pending events (#39924)

Release Date: 2024-06-03

Features

Add timeout when watching pod events in k8s executor (#39551)

Add retry logic for KubernetesCreateResourceOperator and KubernetesJobOperator (#39201)

Bug Fixes

Fix deprecated calls in 'cncf.kubernetes' provider (#39381)

Handling exception getting logs when pods finish success (#39296)

fix wrong arguments in read_namespaced_pod_log call (#39874)

Misc

Move Kubernetes cli to provider package (#39587)

Remove compat code for 2.7.0 - its now the min Airflow version (#39591)

Simplify 'airflow_version' imports (#39497)

Replace pod_manager.read_pod_logs with client.read_namespaced_pod_log in KubernetesPodOperator._write_logs (#39112)

Add a warning message to KPO to warn of one second interval logs duplication (#39861)

Release Date: 2024-05-06

Note

This release of provider is only available for Airflow 2.7+ as explained in the Apache Airflow providers support policy .

Features

Add missing informative logs in KPO trigger until container has not finished (#37546)

Bug Fixes

fixes templated env vars for k8s pod operator (#39139)

Fix SparkKubernetesOperator when using initContainers (#38119)

Refresh properties on KubernetesPodOperator when k8s fails due to token expiration (#39325)

Misc

Bump minimum Airflow version in providers to Airflow 2.7.0 (#39240)

Remove unnecessary validation from cncf provider. (#39238)

Moves airflow import in deprecated pod_generator to local (#39062)

KPO xcom sidecar PodDefault usage (#38951)

Release Date: 2024-04-21

Bug Fixes

Avoid logging empty line KPO (#38247)

Release Date: 2024-04-13

Features

KPO Add follow log in termination step (#38081)

Add GKECreateCustomResourceOperator and GKEDeleteCustomResourceOperator operators (#37616)

Implement deferrable mode for KubernetesJobOperator (#38251)

Create KubernetesPatchJobOperator operator (#38146)

Implement delete_on_status parameter for KubernetesDeleteJobOperator (#38458)

Implement deferrable mode for GKEStartJobOperator (#38454)

Use startup_check_interval_seconds instead of poll_interval to check pod while startup (#38075)

Implement wait_until_job_complete parameter for KubernetesJobOperator (#37998)

Bug Fixes

Use SIGINT signal number instead of signal name (#37905)

Fix spark operator log retrieval from driver (#38106)

Fix dynamic allocation specs handling for custom launcher (#38223)

Fix case if 'SparkKubernetesOperator.application_file' is templated file (#38035)

fix: reduce irrelevant error logs for pod events. (#37944)

Misc

Add GKEListJobsOperator and GKEDescribeJobOperator (#37598)

removed usage of deprecated function for naming the pod in provider k8s pod.py (#38638)

Create DeleteKubernetesJobOperator and GKEDeleteJobOperator operators (#37793)

Refactor GKE hooks (#38404)

Release Date: 2024-03-08

Bug Fixes

Immediately fail the task in case of worker pod having a fatal container state (#37670)

Skip pod cleanup in case of pod creation failed (#37671)

Misc

Avoid non-recommended usage of logging (#37792)

Migrate executor docs to respective providers (#37728)

Release Date: 2024-02-20

Breaking changes

In the case of Kube API exceeded quota errors, we have introduced the task_publish_max_retries flag to control the re-queuing task behavior. Changed the default behavior from unlimited retries to 0. The default behavior is no retries ( task_publish_max_retries==0 ). For unlimited retries, set task_publish_max_retries=-1 . For a fixed number of retries, set task_publish_max_retries to any positive integer.

Fix: The task is stuck in a queued state forever in case of pod launch errors (#36882)

Features

Add logging_interval in KubernetesPodOperator to log container log periodically (#37279)

Create GKEStartJobOperator and KubernetesJobOperator (#36847)

Bug Fixes

Fix occasional attr-undefined for the python_kubernetes_script (#37318)

Fix hanging KPO on deferrable task with do_xcom_push (#37300)

Fix rendering 'SparkKubernetesOperator.template_body' (#37271)

Fix assignment of template field in 'init' in 'KubernetesPodOperator' (#37010)

KPO Maintain backward compatibility for execute_complete and trigger run method (#37454)

Fix KPO task hanging when pod fails to start within specified timeout (#37514)

Fix KeyError when KPO exits too soon (#37508)

Misc

feat: Switch all class, functions, methods deprecations to decorators (#36876)

Kubernetes version bump (#37040)

Add GKEStartKueueInsideClusterOperator (#37072)

Convert Kubernetes ApiException status code to string to ensure it's correctly checked (#37405)