NewYour coding agent can read the release notes before it upgrades.Set up the MCP server โ
PyPI
PyPI, #1563 most downloaded on PyPI
Programmatically author, schedule and monitor data pipelines
Latest stable
3.4.0
Last release 16 days ago
06 Nov 2026
Ships fairly regularly
a new release about every 2 weeks
Nearly every release is documented
notes for 60 of the last 60 stable releases
4 versions withdrawn
withdrawn after publishing
10 years old
306 releases ยท first in 2017
Drag across the ruler to compare, or tap a release to open it.
Resolve NPM vulnerabilities in Simple auth manager
๐ฆ PyPI: https://pypi.org/project/apache-airflow/3.4.0/
๐ Docs: https://airflow.apache.org/docs/apache-airflow/3.4.0/
๐ Release Notes: https://airflow.apache.org/docs/apache-airflow/3.4.0/release_notes.html
๐ณ Docker Image: "docker pull apache/airflow:3.4.0"
๐ Constraints: https://github.com/apache/airflow/tree/constraints-3.4.0
Airflow 3.4.0 introduces Task Loops (AIP-111): a decorated task group can be repeated a number of
times decided at runtime using .loop(). A conditional loop works like a do ... while loop -
the body runs at least once and an until condition decides when to stop, bounded by
max_iterations. Each iteration can use the previous iteration's result and iterations run one
after another (tasks within an iteration can still run in parallel). Retried attempts are kept
under a per-attempt UUID so each iteration's data is preserved and can be cleared selectively.
For detailed usage instructions, see :doc:/authoring-and-scheduling/loops.
.. note::
Downgrading to any version earlier than 3.4.0 can be slow, depending on the number of
XCom rows that have been written after upgrading. Upgrading was tested on a
300m row XCom table, so please don't worry about upgrade time (we made sure
that the XCom data didn't get touched or read during migration to keep it
fast).
This is because this release contains a migration that starts storing new XCom
rows in an xcom_v2 table, and in order to downgrade we have to move those over
to the old table, and there is no way of doing this without copying the
written xcom values.
The downgrade refuses to continue when it finds data it cannot map back
safely. Each migration commits separately, so the later 3.4.0 migrations
have already been reversed at that point: fix the data and re-run the
downgrade, or run airflow db migrate to return to 3.4.
On MySQL a failure after those checks leaves the schema partly downgraded,
as MySQL cannot roll back DDL. Taking a backup before attempting the
downgrade on MySQL is advised, so it can be restored in that case.
Airflow 3.4.0 can author an entire Dag - schedule, tasks, and dependencies - in a language other
than Python. AIP-85 adds a pluggable Dag Importer in the Task SDK that teaches the Dag processor to
recognize source formats beyond .py: each format registers an importer, the processor runs the
matching language runtime, and the resulting Dags surface in Airflow like Python Dags. The formats
that land on it are the TypeScript, Go, and Java language SDKs and a native YAML Dag format (a
.yaml file with a $schema marker, validated against a schema and built into real operators,
with explicit templating). The Dag Importer framework and the language SDKs are experimental in 3.4
and their APIs may change.
For detailed usage instructions, see :doc:/administration-and-deployment/dag-importers.
BaseExecutor.executor_queues (#63491)Executor workload state is now stored on the unified BaseExecutor.executor_queues mapping keyed
by WorkloadType and scheduled through trigger_workloads. The previous per-type attributes
and entrypoints (queued_tasks, queued_callbacks, supports_callbacks, trigger_tasks,
order_queued_tasks_by_priority) and the Airflow 3.3 connection-test surface are folded into
this mapping and kept as backward-compatible shims that emit RemovedInAirflow4Warning (to be
removed in Airflow 4.0). Out-of-tree executors should declare supported_workload_types and
implement trigger_workloads. Connection tests are now admitted under the same per-heartbeat
budget as tasks and prioritized ahead of them. See
#63491 <https://github.com/apache/airflow/pull/63491>_ for the full migration guide.
psycopg3 (psycopg) is now the default driver for the metadata database, for both the sync and
the async engine, replacing psycopg2 and asyncpg. A bare postgresql:// (or a legacy
postgres:// / postgres+psycopg2://) sql_alchemy_conn is rewritten to
postgresql+psycopg://, and the derived async URL uses postgresql+psycopg_async:// when
psycopg3 is installed (falling back to asyncpg otherwise). Airflow also disables psycopg3's
server-side statement preparation (prepare_threshold = None) so the metadata database works out
of the box behind transaction-mode PgBouncer, the recommended production setup. To keep psycopg2 or
asyncpg, install the apache-airflow-providers-postgres[psycopg2] / [asyncpg] extra and
configure the connection explicitly; see #68496 and #69469 for details, including how to re-enable
statement preparation.
Single-asset schedules and asset OR expressions now create one Dag run per event instead of
consuming all queued events in one run. This also applies to AssetOrTimeSchedule and
PartitionedAssetTimetable.
To retain batching, set batch_asset_events=True on the timetable or enable [scheduler] batch_asset_events and reparse the affected Dags. Already serialized timetables without this
option retain their previous batching behavior until reparsed.
Schedules requiring several events, including schedule=[a, b], asset AND expressions, and
partitioned timetables using RollupMapper, continue to batch by default. Explicitly setting
batch_asset_events=False for these schedules raises a timetable validation error.
ui_color and ui_fgcolor (#68760)The ui_color and ui_fgcolor attributes on operators and TaskGroups color the graph view
again -- the node fill from ui_color and the operator label from ui_fgcolor, as they did in
Airflow 2. They were silently ignored since Airflow 3.0.
The value can be a raw color (a hex code like #e8b7e4 or a CSS name) or a Chakra theme token --
a palette token such as blue.500 or a semantic token such as brand.solid that adapts to
light and dark mode (including tokens added through custom UI theming); a token resolves through a
theme-controlled CSS variable. The node border keeps reflecting the task-instance run state; the
custom color is applied to the fill and label only.
validate_key now raises ValueError instead of AirflowException (#68890)The airflow.utils.helpers.validate_key utility now raises ValueError instead of
AirflowException when an invalid key is provided (for example an invalid format or a
length-limit violation).
Dataset alias has been removed from the airflow module (#70154)Use Asset from airflow.sdk instead.
AUDIT_LOGS_ALL view (#70759)Rows in the audit log whose dag_id is NULL record operations that are not tied to a Dag,
such as Connection, Variable and Pool changes. They were returned to any caller that could read
event logs at all, because there is no per-Dag key to authorize them on and both read paths fell
back to a Dag-level audit log check.
They are now gated on a dedicated AccessView.AUDIT_LOGS_ALL, in the same way
AccessView.IMPORT_ERRORS_ALL gates import errors for files with no registered Dag. Rows bound
to a Dag keep their existing per-Dag check.
GET /eventLogs no longer returns rows with a NULL dag_id to callers without the newtotal_entries and pagination. GET /eventLogs/{event_log_id}403 for such a row. An id matching no row still returns 404.ADMIN role only. With the FAB authAll Audit Logs resource, which ships in the Admin role; customAll Audit Logs.can_read explicitly.AccessView now denies that view and emits aUserWarning instead of raising KeyError. This affects auth managers older than the coreairflow teams create and airflow teams sync now accept only names matching
(?!.*__)[a-z0-9_-]{3,50} -- lower case letters, digits, hyphens and single underscores. Upper
case letters and __ were previously accepted.
A team name is embedded in the AIRFLOW_CONN__<TEAM>___<ID> environment variable namespace that
holds team scoped secrets, and the backend upper-cases the name when it builds that variable.
data_eng and Data_Eng therefore resolved a single namespace between them, and a name
containing ___ could not be told apart from the separator.
airflow db migrate lower-cases every stored team name, moving that team's Dag bundles,
Connections, Variables, Pools and Triggers with it. Two stored names that differ only in case would
collapse onto one row and silently merge two teams, so migration refuses to start while any such
pair exists -- rename all but one first. Names that stay invalid for another reason, such as
containing __, are migrated as they are; they keep working, but airflow teams sync exits
non-zero until they are renamed.
Deleting asset event records with airflow db clean now also removes any asset events that are
still pending for a downstream Dag. This mainly affects Dags waiting on a multi-asset condition,
where a pending event can be purged before the condition is met. Deleting an asset event now
consistently means it will not trigger a run; previously such a pending trigger could still fire
after the event had been purged.
Relatedly, an asset event that was queued for a Dag while it was scheduled on that asset is still
used to trigger the run, and appears among the run's triggering events, even if the asset is later
removed from the Dag's schedule. Previously, they were silently dropped.
A worker now redeems a callback's single-use token with the API server (PATCH /execution/callbacks/{callback_id}/run) before importing and running the callback. Redeeming the
token moves the callback from queued to running and swaps it for a short-lived execution-scoped
token. Before this change a callback ran without the worker making any authenticated call, so the
token minted for it was never checked.
Because the token is single-use, a redelivered or replayed message that reaches a callback that is
already running or finished is refused rather than run again. A worker on 3.4 needs an API server
that serves this endpoint; a worker talking to an older API server negotiates the API version and
behaves as though the endpoint is not there.
Clearing a task instance now discards its task_state_store entries, so the next attempt starts
from the beginning instead of resuming from a checkpoint or reconnecting to an external job
recorded by the cleared attempt. Retries are unaffected and keep task state as before. This applies
to clearing individual task instances (the clear endpoint/dialog and airflowctl tasks/dags clear); clearing a whole Dag run still keeps task state (tracked in
#72929 <https://github.com/apache/airflow/issues/72929>_). To preserve the old behaviour, pass
keep_task_state=True (or tick "keep task state" in the clear dialog; a per-browser default is
available under Settings > Clearing). Note a task-instance clear now also deletes
task_state_store rows for that task instance.
[core] execute_tasks_new_python_interpreter now applies to Airflow 3 task processes (#72164)On Airflow 3 the option had no effect on task execution (only the Edge worker read it). When set to
True, the task supervisor now exec\ s a fresh interpreter right after forking the task
process, which prevents the fork from inheriting a lock held by a supervisor thread (a permanent
hang at the task's first TLS call). Deployments that kept the option True from Airflow 2 get
this behaviour, and its per-task interpreter start-up cost, on upgrade without further action; set
it to False to keep bare fork. The task process reads the global value, so a team-scoped Edge
setting alone does not turn it on. Edge workers with the option True already start a fresh
interpreter for the supervisor and will now start a second one for the task. The Dag processor, the
triggerer and task callbacks (which the callback supervisor runs from a closure that cannot be
named for an exec'd child) are not affected and keep bare fork.
Airflow now supports a draining scheduling state for Dags. Draining stops the scheduler from
creating new scheduled or asset-triggered Dag runs while allowing already queued and running Dag
runs to continue. Explicitly triggered runs remain allowed.
After outstanding Dag runs finish, the Dag automatically transitions to paused. Users can
cancel draining to return the Dag to active.
Manual runs, backfills and asset materializations on a paused Dag can drain the Dag instead of
unpausing it: choose Drain in the UI, or set drain_dag in the REST API request. Draining
lets unfinished runs of the Dag start or resume, except those held by paused backfills, creates no
scheduled runs, and pauses the Dag again once all unfinished runs finish. Paused backfills remain
paused and must be resumed for their runs to finish and draining to complete.
The UI, REST API, and airflowctl expose the active, draining, and paused scheduling
states, and the Dags list can pause, drain, or unpause several selected Dags at once. Existing
is_paused API usage remains supported; use scheduling_state when the draining state must be
selected or reported explicitly.
REPARSE_ALL view (#72896)A file that failed to import before defining any Dag has no per-Dag key to authorize a reparse
against, so PUT /parseDagFile/{file_token} previously returned 404 for it. Reparsing such a
file is now gated on a dedicated AccessView.REPARSE_ALL -- a write action of its own, distinct
from the permission to view import errors -- and the Dag import errors modal gains a reparse action
for these files.
PUT /parseDagFile/{file_token} for a file with no registered Dag authorizes againstREPARSE_ALL and returns 403 when the caller lacks it, instead of 404. A file that hascan_edit check.ADMIN role only. With the FAB authAll Reparses resource, which ships in the Admin role; customAll Reparses.can_read explicitly.--file-io option from the airflow info command (#73054)--file-io uploaded the report to file.io and printed a shareable link. file.io no longer
accepts anonymous uploads, so the option could not succeed; it also ended in an unhandled traceback
rather than reporting the failure.
Redirect the output instead::
airflow info --anonymize > airflow-info.txt
Note that --file-io applied --anonymize implicitly. Pass --anonymize explicitly when
sharing a report: plain airflow info prints connection strings, including passwords, in
cleartext.
[workers] execution_api_retries (#73244)Heartbeats had two overlapping retry mechanisms: the Task SDK API client retried every request,
including heartbeats, and the supervisor separately counted consecutive heartbeat failures against
[workers] max_failed_heartbeats. The client retries also had the unwanted side-effect of
blocking the supervisor's monitor loop while it backed off, so during an API server outage the
supervisor would be delayed forwarding task logs and/or noticing that the task subprocess had
exited.
Heartbeat retries are now governed solely by max_failed_heartbeats, whose default has been
adjusted from 3 to 30 to keep roughly the same tolerance for an unreachable API server as
the two mechanisms combined gave before.
If you have set [workers] max_failed_heartbeats explicitly, you may wish reconsider value you
have chosen. It is now the whole retry budget: with the default [workers] min_heartbeat_interval of 5 seconds, the worker tolerates roughly 5 * max_failed_heartbeats
seconds of API unavailability before terminating a running task. You should likely keep the
resulting value below any configured [scheduler] task_instance_heartbeat_timeout (default 300
seconds).
DeadlineAlert now refuses a callback that is not exactly AsyncCallback or
SyncCallback, so a subclass fails when the Dag is parsed rather than when the scheduler reads
it back. The scheduler decodes stored callbacks by exact class name so no class named by a Dag
author is imported or instantiated there, which means a subclass could be written and then never
read.
Passing a number as the interval is deprecated and now emits RemovedInAirflow4Warning. It
is still accepted and taken as a count of seconds, but the supported types are timedelta and
VariableInterval. A timedelta subclass such as pendulum.duration(...) is accepted and
normalized to a plain timedelta, where previously it passed validation and then failed during
serialization. interval=True is refused, because bool is an int subclass and would
otherwise be taken as one second.
POST /assets/{asset_id}/materialize endpoint now requires a JSON request body (#73328)Previously a request with no body was accepted and the asset was materialized with default values.
The endpoint now requires an explicit JSON body, consistent with the other mutating endpoints: a
request without a body is rejected with 422, and an empty HTML form is not a valid input
anymore. All body fields remain optional, so an empty object ({}) is a valid body.
.. note::
If you have any external systems that look at task logs or TI state for an up-for-retry TI, you
will need to be aware of this change and update your code.
For example, if you made a request to GET /api/v2/dags/{dag_id}/dagRuns/{dag_run_id}/taskInstances/{task_id} expecting to get info about
the failed try, you will now need to look at GET /api/v2/dags/{dag_id}/dagRuns/{dag_run_id}/taskInstances/{task_id}/tries/{try_number - 1}
When a task is up_for_retry, try_number now identifies the upcoming try. To retrieve the try that
failed, request /tries/{try_number - 1}. The UI shows the pending try and provides a link to
the previous try's logs.
Airflow now allocates a task instance's UUID and try number together immutably so they consistently
identify the same try.
Previously when a task entered the retry cooldown period, it would be given a new UUID, but keep on
try number N and have the state of UP_FOR_RETRY until that period was up (at which point that try
would be marked FAILED and the try number incremented). Instead, the failed try is immediately
marked with state of FAILED, and a new try created in state UP_FOR_RETRY.
A Dag whose TaskGroups depend on each other in a cycle, when each TaskGroup is treated as a single
unit, now issues a TaskGroupCycleDeprecationWarning at parse time and shows a Dag warning in
the UI that names the TaskGroups and tasks involved. These Dags still parse and run, because their
task dependencies are acyclic, but Airflow 3.5 is planned to reject them at parse time.
A dependency into or out of any task in a TaskGroup counts as a dependency of the whole group, so a
path that leaves a TaskGroup and comes back into it is a cycle. For example, a >> bridge >> b,
with a and b in the same TaskGroup and bridge outside it, makes the group depend on
bridge and bridge depend on the group, even if a >> b is also set.
bridge into the group, or move b out of it.-W error::airflow.sdk.exceptions.TaskGroupCycleDeprecationWarning. The check runs when a Dag isDagBag, so only tests that load your Dags into a DagBag and assert there areSee "Cyclic TaskGroup dependencies" in the Dags documentation.
The experimental jars_root (Java) and executables_root (Go) coordinator kwargs are removed.
Put the artifacts in a Dag bundle registered in [dag_processor] dag_bundle_config_list and set
task_handler_bundle_name in the coordinator's kwargs, or leave it unset to use the task's
own Dag bundle. A config that still sets a removed kwarg fails with Cannot instantiate coordinator '<key>'.
Airflow now requires Python 3.11 or newer. Python 3.10 reaches its end of life in October 2026, and
Airflow drops support for a Python version in main as soon as it reaches end of life. The
apache-airflow distribution, the Task SDK, airflowctl and all providers released from
main declare requires-python = ">=3.11".
required_kwargs attribute for custom deadline references (#70714)Implement _evaluate_with() with a dagrun parameter instead.
Airflow's CI and production images are now based on Docker Hardened Images <https://dhi.io>_ for
Python and Debian 13 "trixie".
Python is no longer compiled from source during the image build - it comes from the hardened base
image instead, which removes the source download, its signature verification and the compilation
itself from the build. Python still lives in /usr/python, which is now a symlink to the
/opt/python location the base image uses, so paths inside the image are unchanged.
The BASE_IMAGE build argument now defaults to
ghcr.io/apache/airflow/base/python:<version>-debian13-dev, Airflow's public mirror of the
upstream dhi.io/python image, so building an image needs no registry credentials. Anyone
overriding BASE_IMAGE with a plain debian:bookworm-slim image has to switch to an image that
already provides Python, or build the legacy flavor.
The base OS moved from Debian 12 "bookworm" to Debian 13 "trixie". Library packages renamed in
trixie's 64-bit time_t transition (for example libssl3 is now libssl3t64) have to be
updated in custom RUNTIME_APT_DEPS / ADDITIONAL_RUNTIME_APT_DEPS, and the MySQL-compatible
client now comes from MariaDB 11.8.
For the 3.4.x line, production images built the previous way - Python compiled from sources on
debian:bookworm-slim - are still published as the deprecated legacy flavor, selected with
the new AIRFLOW_IMAGE_FLAVOR="legacy" build argument. They will be removed in Airflow 3.5.0.
The PYTHON_LTO build argument is gone. To build a FIPS-compliant image, point BASE_IMAGE at a
FIPS variant of the hardened image instead, for example dhi.io/python:3.13.16-debian13-fips-dev.
certifi (#73116)With [api] client_use_public_certs enabled (the default), the Task SDK client that talks to the
Execution API now trusts the operating system's certificate store through truststore instead of
the certifi CA bundle. CAs from [api] ssl_ca_file and [api] ssl_cert are still trusted
in addition to the system store, and certifi is no longer a Task SDK dependency.
Workers must have the public CAs they rely on installed in the system store (for example, the
ca-certificates package on Debian-based images). On macOS and Windows, certificate validation
is delegated to the operating system, which can reject server certificates that OpenSSL accepted,
such as private-CA certificates without the serverAuth extended key usage.
next_kwargs hold custom classes (#73577)When a deferred task resumes on a worker with an older Task SDK, the API server no longer reconstructs arbitrary classes from the stored next_kwargs to convert them to the legacy format. Built-in value types, assets and asset aliases (including asset access control) are still converted, without constructing them. A deferred task whose next_kwargs hold any other class now fails on such a worker with an error naming the type and asking to upgrade the Task SDK, instead of crashing on a payload it cannot decode.
What to do: if you run workers with apache-airflow-task-sdk 1.1.x or older and your deferrable operators or triggers pass instances of custom classes in next_kwargs (for example via self.defer(..., kwargs=...) or in the trigger event payload), upgrade the Task SDK on those workers to 1.2.0 or later. Newer workers receive next_kwargs unchanged and are not affected. Until all workers are upgraded, pass only built-in values, assets and asset aliases in next_kwargs of tasks that may resume on older workers.
The keyword arguments of Deadline Alert callbacks are no longer decoded by the scheduler. They
are kept in their serialized form and decoded where the callback runs: by the triggerer for an
AsyncCallback, and by the callback runner on the worker for a SyncCallback.
Workers that run SyncCallback Deadline Alerts need a Task SDK that includes this change, so
upgrade apache-airflow and apache-airflow-task-sdk on workers together. A worker with
an upgraded apache-airflow and an older Task SDK fails such callbacks with an error instead
of running them. Callbacks created before the upgrade run as before.
serialize_value must implement store_value to support task loops (#74858)Every iteration of a task inside a loop is its own task instance, but serialize_value
receives the same dag_id, run_id, task_id, map_index and key for all of
them, so a backend that builds its storage location from those makes every iteration's XCom
read back as the last iteration's value. Backends should override store_value and
load_value instead, and build the storage location with XComCoordinate.build_path,
which is unique per task instance, or per write when the write cannot know the task instance.
A Dag that uses a loop fails on startup when the configured XCom backend still stores values
through serialize_value, even if it already writes to a unique location. This includes
XComObjectStorageBackend from apache-airflow-providers-common-io 1.10.1 and earlier;
upgrade the provider before using task loops with it.
Custom Task State Store backends will likewise need to include
scope.region_id in their storage key to support mapped tasks inside a loop
and loop iterations regenerated by clearing.
airflow.utils.email is deprecated in favor of the SMTP provider (#75414)The functions in airflow.utils.email (send_email, send_email_smtp, send_mime_email,
build_mime_message and get_email_address_list) now live in
airflow.providers.smtp.utils.emailer. The old names still work and raise a DeprecatedImportWarning.
Airflow now requires apache-airflow-providers-smtp>=3.2.0.
Keep [email] email_backend set to airflow.utils.email.send_email_smtp. Airflow treats that value as
"send task emails with SmtpNotifier", which reads all SMTP settings from the Connection. Changing it to
the provider path sends through the [smtp] settings instead.
If you call these functions directly:
BaseHook.get_connection. A script that runs outside Airflow'sairflow.configuration.conf.set() are not seen. airflow.cfg and environment variables work as before.airflow.utils.email.send_mime_email no longer intercept the call made bysend_email_smtp. Patch airflow.providers.smtp.utils.emailer.send_mime_email instead.The API server uses a second database connection pool
API server workers now run some database work on the async engine as well as the sync one, and each engine has its own connection pool, sized by the same [database] sql_alchemy_pool_size and [database] sql_alchemy_max_overflow settings. What uses the async pool depends on the --apps option of airflow api-server:
execution: task heartbeats and several Execution API reads.core: the audit log entry recorded for each write request through the REST API or the UI.all (the default): both of the above.In every mode a worker can therefore open up to twice sql_alchemy_pool_size + sql_alchemy_max_overflow connections (30 with the defaults) and keep up to twice sql_alchemy_pool_size of them idle. Account for this, per [api] workers, when sizing PgBouncer or the database's max_connections.
The async pool is used since #67800, #73403, #73407 and #76416. (#76430)
airflow db clean (#74498)ChainRetryPolicy to compose retry policies in order (#73508)is_scheduled REST API parameter (#72029)Note truncated.
Your coding agent can read these notes before it upgrades. Set up the MCP server โ
Apache Airflow 3.4.0rc3
Apache Airflow 3.4.0rc3
Apache Airflow 3.4.0rc2
Apache Airflow 3.4.0rc2
Apache Airflow 3.4.0rc1
Apache Airflow 3.4.0rc1
Nothing published for this version
Nothing published for this version
Fix resolution of deprecated imports in airflow.utils.helpers
๐ฆ PyPI: https://pypi.org/project/apache-airflow/3.3.2/ ๐ Docs: https://airflow.apache.org/docs/apache-airflow/3.3.2/ ๐ Release Notes: https://airflow.apache.org/docs/apache-airflow/3.3.2/release_notes.html ๐ณ Docker Image: "docker pull apache/airflow:3.3.2" ๐ Constraints: https://github.com/apache/airflow/tree/constraints-3.3.2
The four routes that name a backfill in their path -- GET /backfills/{backfill_id}
and the pause, unpause and cancel routes -- resolved the Dag they authorize
against from the dag_id supplied on the request whenever the path's id matched no row.
An unknown id and a backfill on a Dag the caller cannot see therefore answered differently,
which enumerates backfill ids across Dags.
The backfill named in the path is now the only thing those routes authorize against.
Behaviour changes:
404
(Backfill not found) -- the same response an unknown id gets -- instead of the
403 returned before. A caller who can read the Dag still gets 403 for a write
they are not allowed to make.backfill_id in the path is never authorized against a dag_id in the request body
or query string. GET /backfills, POST /backfills and POST /backfills/dry_run
name no backfill in their path and keep authorizing off the request.Backfill not found.
The pause, unpause and cancel routes previously answered
Could not find backfill with id {backfill_id}. Clients matching on detail
must be updated. (#71113)get_user() is written to prefer an explicit bearer token, then OAuth2, then the
session cookie, but that precedence was unreachable whenever a cookie was present.
JWTRefreshMiddleware runs
first, resolves a user from the _token cookie alone and stamps it on
request.state, and get_user() returned that cached user before looking at either
explicit credential. The effective order on every core-API route was cookie over bearer.
A request carrying both a session cookie and an explicit credential therefore executed, and was recorded in the audit log, as the cookie's principal rather than the identity the client presented. The cached user is now honoured only when the request carries no explicit credential.
Behaviour changes:
_token cookie and an Authorization: Bearer headeris now resolved as the bearer token's principal, where it was previously resolved as the
cookie's. The same applies to a cookie combined with an OAuth2 token.
401/403 evenwhen a valid ``_token`` cookie accompanies it. Previously the cookie silently took over
and the request succeeded as the cookie's principal; the failure is now loud.
the token-refresh behaviour of ``JWTRefreshMiddleware`` unchanged.
service account's bearer token while a user session cookie was present, and expected the
user's identity to apply -- will now act as the bearer token's principal. Remove the
header, or the cookie, to select the intended identity explicitly. (#72225)
dag_run join (#72944)airflow db clean never purging the callback table (#72899)DAG subclasses or aliased-module imports (#72898)DeadlockImminentError when a connection is resolved inside an async task (#72895)XCom values that already parse as JSON during the bytea-to-JSONB migration (#72886)airflow.utils.helpers (#72868)LIMIT 1 to avoid scanning large tables (#72842)airflow info --file-io uploading an empty report (#72832)partition_key behind the 2026-06-30 Execution API version (#72827)DagRun fields (#72812)is_authorized_hitl_task (Human-in-the-loop) hook runs (#72807)td_format rendering of negative durations (#72798)airflow jobs check --allow-multiple with --limit 0 (#72744)XCom existence lookups with LIMIT 1 to avoid full scans (#72702)KeyError for removed-task task instances (#72620)hierarchical_alphabetical sort order breaking the graph and grid (#72618)@task-decorated callable errors when extra positional arguments are passed (#72616)airflow db migrate failing under the PyMySQL driver when a schema migration drops unique constraints (#72613)airflow providers get --full mutating cached provider metadata (#72601)airflow connections test returning a success exit code on failure (#72583)airflow standalone leaking components when one fails to start (#72568)DAG.cli() crashing on dags pause and dags unpause (#72565)limit search param in the task overview duration chart (#72357)AIRFLOW_TEST_MODE from airflow tasks test without --env-vars (#72320)with block (#71956)Variable.set rewriting the team_name of existing variables (#71904)/assets/events to the Dags the caller may read (#71785)dag_discovery_safe_mode is False (#71714)FORWARDED_ALLOW_IPS when the API server runs under gunicorn (#71708)airflow config lint staying silent on conditional removal rules (#71651)clearTaskInstances returning HTTP 500 instead of 422 on an invalid body (#71559)airflow dags list (#71481)_team_name missing from DagRun passed to some listener calls (#71262)SerializedVariableInterval for deadline alerts (#72244)FlexibleForm (#71573)BaseDeadlineReference and deadline_reference to the SDK public interface (#71208)create_async_metadata_engine example to the docs (#72804)dag.test() for testing custom operators in the docs (#71587)Apache Airflow 3.3.2rc1
Apache Airflow 3.3.2rc1
Fix npm vulnerabilities in the simple auth manager
๐ฆ PyPI: https://pypi.org/project/apache-airflow/3.3.1/ ๐ Docs: https://airflow.apache.org/docs/apache-airflow/3.3.1/ ๐ Release Notes: https://airflow.apache.org/docs/apache-airflow/3.3.1/release_notes.html ๐ณ Docker Image: "docker pull apache/airflow:3.3.1" ๐ Constraints: https://github.com/apache/airflow/tree/constraints-3.3.1
pandas 3 exposes its public classes from the pandas namespace, so a DataFrame is qualified as
pandas.DataFrame instead of pandas.core.frame.DataFrame. XComs record that name alongside the
serialized value, so the name written into the metadata database depends on the pandas version of the
component that pushed the value. Airflow registers both names, and a DataFrame written by either
pandas version can be read by either -- no configuration change is needed, and existing XComs stay
readable.
What you should do:
Roll this Airflow version out to every component before pandas 3 reaches any of them -- workers in particular. A component that predates this change cannot read a DataFrame XCom written under pandas 3, and fails the pull with:
.. code-block:: text
ImportError: pandas.DataFrame was not found in allow list for deserialization imports.
To allow it, add it to allowed_deserialization_classes in the configuration
The message points at configuration, but the allow list is not the cause and changing it does not help. The rows are not corrupt: they become readable again as soon as the reader is upgraded.
Treat a downgrade as a one-way door for those XComs. Rolling back to an Airflow version without this change strands any DataFrame XCom written while on pandas 3, with the same error, until you roll forward again.
Review Dags that inspect the dtypes of a pulled DataFrame. The pandas version of the reader
determines what a pulled DataFrame looks like, not the version that wrote it. Under pandas 3, a
column of strings comes back as str rather than object, and its missing values
come back as nan rather than None. Values are unchanged, but downstream code that branches
on dtype == "object", checks cells with is None, or compares against a reference frame with
DataFrame.equals() can behave differently after the upgrade.
The 0082_3_1_0_make_bundle_name_not_nullable migration assigned every legacy row
bundle_name='dags-folder', so triggering a DagRun raised Requested bundle 'dags-folder' is not configured. on any deployment that uses a bundle other than the default dags-folder.
DagFileProcessorManager now runs a one-shot, best-effort backfill at startup that routes each
affected Dag to the correct bundle based on its file path; unmatched Dags self-heal on the next
successful parse (or run airflow dags reserialize to force it immediately).
Configuration options are registered as sensitive under their base section, so until now only the
base spelling of an option was masked. A team scoped override -- set in a [<team>=<section>]
config file section, or through an AIRFLOW__<TEAM>___<SECTION>__<KEY> environment variable --
was not recognized as the same option and was returned in full.
Sensitivity is now decided after resolving the team scoped spelling back to the base option, so a team scoped value is masked exactly as the base value already was.
Behaviour changes:
AirflowConfigParser.as_dict(display_sensitive=False), GET /config,
GET /config/section/{section}/option/{option} and airflow config list now return
< hidden > for a team scoped value of an option registered as sensitive. Deployments that
read a team's real value through any of these will now receive the mask; use
display_sensitive=True where a real value is required and appropriate._cmd and _secret entries are replaced with < hidden > in place, rather
than being resolved into their value and removed as they are in a base section. Resolving them
is not supported for a team, so the command string or secret path is no longer shown either.display_sensitive=True continues to return real
values.TriggerDagRunOperator gets a 404 (#71083)deadline_reference decorator's no-parentheses form (#70966)bundle_name during upgrade from 2.x to 3.x (#70662)airflow partitions clear (#69547)json_logs is enabled (#70669)none_failed_min_one_success tasks in mapped task groups (#70318)on_failure_callback for heartbeat-timed-out retries (#69824)TaskFailedEvent, instead of always failing terminally (#71163)TaskInstance mark-success downstream default (#70143)update --option/--ignore-option never matching options (#70757)TypeError in airflow db shell when the database name is missing (#70752)-o commands so structured output stays machine-readable (#70747)Variable values stored as JSON lists (#71069)Variable/Connection updates (#71043)KubernetesPodOperator (#70756)structlog>=26.1.0 and croniter>=6.2.2 to fix memory leaks (#70749)dag and note missing from Dag-run state-change listener events (#70245)email_on_failure/email_on_retry task alerts silently ignoring a custom [email] email_backend and always routing through SmtpNotifier; an email_backend that cannot be imported now errors loudly instead of silently falling back to SMTP (#70129)Trigger Again showing empty config for the selected run (#70288)FanOutMapper and wait policies from airflow.partition_mappers (#69513)task.execute OpenTelemetry span around task execution (#69359)run_type tag to the dagrun.duration.failed metric (#70731)he) translations (#70566)ar) translations (#70510)pl) translation (#70507)el) translations (#70471)ResumableJobMixin an abstract base class (subclasses must implement its methods) (#70810)?) and clean up the graph/grid view (#69978)ResumableJobMixin (#70792)jwt_secret/_secret and LocalFilesystemBackend config support (#70730)logging_config_class contract and document REMOTE_TASK_LOG (#70592)AssetAlias usage (#71087)AssetPartitionDagRun provisional-run docstring (#70104)CronDataIntervalTimetable and DeltaDataIntervalTimetable (#70434)FanOutMapper docs (#69511)dev/README.md (#70107)fr) UI translations to 100% coverage (#70387)nl) translations (#70004)zh-CN) UI translations (#70417, #70418, #70419)zh-TW) translation gaps (#70195, #70379, #69707)ko) translations and backport from main (#70807, #70832)Nothing published for this version
Nothing published for this version
One column per quarter.