NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
PyPI · #1758 most downloaded on PyPI
dlt is an open-source python-first scalable data loading library that does not require any backend to run.
Last release 15 days ago
10 Sep 2026
Ships fairly regularly
a new release about every 2 weeks
Nearly every release is documented
notes for 59 of the last 60 stable releases
13 versions withdrawn
withdrawn after publishing
9 years old
183 releases · first in 2018
adds support for http and path style for filesystem sql_client with s3 by @sh-rp in https://github.com/dlt-hub/dlt/pull/2343
Full Changelog: https://github.com/dlt-hub/dlt/compare/1.8.0...1.8.1
rewrites python - arrow backend for sql_database, better type conversion by @zilto in https://github.com/dlt-hub/dlt/pull/2295
sql_database, better type conversion by @zilto in https://github.com/dlt-hub/dlt/pull/2295json modules by @joscha in https://github.com/dlt-hub/dlt/pull/2286make_hints by @rudolfix in https://github.com/dlt-hub/dlt/pull/2365Full Changelog: https://github.com/dlt-hub/dlt/compare/1.7.0...1.8.0
One column per quarter.
Nothing published for this version
dlt 1.7.0 is mostly a maintenance release that should not impact your existing pipelines when upgrading.
dlt 1.7.0 is mostly a maintenance release that should not impact your existing pipelines when upgrading.
Improvements
Bugs
Docs
Full Changelog: https://github.com/dlt-hub/dlt/compare/1.6.1...1.7.0
Nothing published for this version
Nothing published for this version
Python version support changes:
Python version support changes:
Python 3.13 package depends on dlt-pendulum (datetime library). It is a fork of pendulum that we are going to support until we are able to merge our changes into the original repo. There are a few nice things in it (we got rid of test dependencies and now it works on Alpine linux etc.)
Note: other Python versions still use original pendulum.
A big improvement of Databricks destination:
staging filesystem pipelineImprovements to SQLALchemy destination:
Full Changelog: https://github.com/dlt-hub/dlt/compare/1.5.0...1.6.0
Nothing published for this version
This is alpha release before Python version support changes.
This is alpha release before Python version support changes.
Python 3.13 requires pendulum (our datetime library) release. It is available via dltHub PyPI. If you are on Python 3.13 use the following command to install dlt
pip install dlt==1.6.0a0 --index=https://pypi.dlthub.com
Full Changelog: https://github.com/dlt-hub/dlt/compare/1.5.0...1.6.0a0
After several weeks of experimenting we release dataset API. You can now read data in your destination with a neat, unified interface that works the s
After several weeks of experimenting we release dataset API. You can now read data in your destination with a neat, unified interface that works the same way for warehouses, relational databases, sql alchemy dialects, local and remote files, iceberg and delta tables.
You can use simple dot notation to access tables, execute sql or use data-frame expression (compiled to SQL with ibis). We materialize your data as panda frames, arrow tables or dbapi compatible records (also in batches). Here's main intro:
https://dlthub.com/docs/general-usage/dataset-access/dataset
Together with this we release our backend-less, catalog-less (well, ad hoc technical catalog is created) Iceberg implementation. You can use append and replace write dispositions, create partitions and write to the bucket. Be aware of limitations, we are just starting!
https://dlthub.com/docs/dlt-ecosystem/destinations/delta-iceberg
iceberg table format support for filesystem destination by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/2067--eject source code of the core sources (ie. sql_database) to allow hacking-in customizations by @rudolfix in https://github.com/dlt-hub/dlt/pull/2150ℹ️ Note on add_limit: now you can use it to chunk large resources and load them in pieces. We support chunks created
based on maximum number of rows or after a specified time. Please read the docs: your resource should return ordered rows or
be able to get data from checkpoint. Also note that we apply add_limit after all processing steps (ie. incremental), before we were limiting generator directly. This was a necessary change to implement chunking and is backward compatible regarding produced data but your resource can be queried many times the get "new" item that ie. is not filtered out by incremental.
https://dlthub.com/docs/examples/backfill_in_chunks
endpoint_url config in GCS by @trymzet in https://github.com/dlt-hub/dlt/pull/2120add_limit to do large backfills in steps by @sh-rp in https://github.com/dlt-hub/dlt/pull/2131Code got updated to 1.x.x dlt and tests work again. We are accepting contributions again.
ℹ️ 0.5 sources are on 0.5 tag. If you are still on dlt 0.5.x access this tag via dlt init sql_database duckdb --branch 0.5
Full Changelog: https://github.com/dlt-hub/dlt/compare/1.4.1...1.5.0
Docs: update deprecated paginator type in examples by @burnash in https://github.com/dlt-hub/dlt/pull/2093
We release important bugfix and define identifier normalization behavior for compound identifiers. practically
read more here: https://dlthub.com/docs/devel/general-usage/naming-convention#compound-flattened-identifiers
adds engine adapter and passes incremental and engine to query adapter by @rudolfix in https://github.com/dlt-hub/dlt/pull/2070
path can also be a URL by @joscha in https://github.com/dlt-hub/dlt/pull/2099data_selector by @joscha in https://github.com/dlt-hub/dlt/pull/2101Full Changelog: https://github.com/dlt-hub/dlt/compare/1.4.0...1,4,1
Nothing published for this version
Nothing published for this version
Fix the deprecation warning in .common.configuration.container by @burnash in https://github.com/dlt-hub/dlt/pull/2025
exclude_keys() to dlt.common.utils by @burnash in https://github.com/dlt-hub/dlt/pull/1966JSONResponseCursorPaginator with empty string cursor value by @kang8 in https://github.com/dlt-hub/dlt/pull/2016delta partitioning on arrow normalizer load id by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/2022delta table format by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/2048dlt init by @burnash in https://github.com/dlt-hub/dlt/pull/2028dlt dataset public interface and docs coming next week.
.common.configuration.container by @burnash in https://github.com/dlt-hub/dlt/pull/2025Full Changelog: https://github.com/dlt-hub/dlt/compare/1.3.0...1.4.0
Nothing published for this version
Nothing published for this version
Nothing published for this version
Fix try/except in from_reference shadowing MissingDependencyException by @burnash in https://github.com/dlt-hub/dlt/pull/1939
references table hint and reflect them in sql_database by @steinitzu in https://github.com/dlt-hub/dlt/pull/1925Below we expose a new pipeline._dataset and dlt._dataset interfaces that provide unified access to data loaded into destination. We also implement duckdb-based SQL client on a filesystem destination to access data in data lakes. We'll add documentation once we stabilize dataset interface. However already now you can benefit from new cursor implementation of sql_client that allows to take data frames, arrow tables also in batches:
PRs below adds pluggy and a few first plugin hooks. The idea is to make a lot of functionalities in dlt pluggable. Currently you can plug new cli command (or upgrade existing) and you can also plug your own runtime environment (how dlt looks for data, secrets etc.)
Full Changelog: https://github.com/dlt-hub/dlt/compare/1.2.0...1.3.0
Sqlalchemy merge support by @steinitzu in https://github.com/dlt-hub/dlt/pull/1842
scd2 with merge_key by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1818Full Changelog: https://github.com/dlt-hub/dlt/compare/1.1.0...1.2.0
fix intermittent delta panic issue by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1832
delta panic issue by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1832processing_steps by @burnash in https://github.com/dlt-hub/dlt/pull/1872Full Changelog: https://github.com/dlt-hub/dlt/compare/1.0.0...1.1.0
This is a major `dlt` release. Please check the list of breaking changes and deprecations: https://github.com/dlt-hub/dlt/issues/1778
This is a major dlt release. Please check the list of breaking changes and deprecations: https://github.com/dlt-hub/dlt/issues/1778
foreign_key, adds nested references (row_key - parent_key) by @rudolfix in https://github.com/dlt-hub/dlt/pull/1774complex data type, changes to json by @rudolfix in https://github.com/dlt-hub/dlt/pull/1792cursor_path missing or None value by @willi-mueller in https://github.com/dlt-hub/dlt/pull/1576Full Changelog: https://github.com/dlt-hub/dlt/compare/0.5.4...1.0.0
Nothing published for this version
Nothing published for this version
BigQuery project_id may be different from credentials project_id by @VioletM in https://github.com/dlt-hub/dlt/pull/1680
merge write disposition with delta table format by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1742storage_options to DeltaTable.create by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1686delta table dangling Parquet file bug by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1695delta table partitioning support by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1696scd2 record reinsert by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1707scd2 custom "valid from" / "valid to" value feature by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1709psutil by @deepyaman in https://github.com/dlt-hub/dlt/pull/1733Full Changelog: https://github.com/dlt-hub/dlt/compare/0.5.3...0.5.4
Nothing published for this version
Add support for continuously starting load jobs as slots free up in the loader. This will significantly speed up loading packages with many files. by
get_delta_tables helper function to optimize and vacuum tables by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1664sql_database @steinitzuFull Changelog: https://github.com/dlt-hub/dlt/compare/0.5.2...0.5.3
Add upsert merge strategy for Postgres and Snowflake, by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1466
upsert merge strategy for Postgres and Snowflake, by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1466upsert support for delta table format in filesystem destination by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1600autodetect_schema by @rudolfix in https://github.com/dlt-hub/dlt/pull/1591upsert merge strategy for more SQL destinations (Athena, BigQuery, Databricks, mssql) by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1628current.pipeline() access by @rudolfix in https://github.com/dlt-hub/dlt/pull/1581delta table format by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1586delta table format on filesystem destination by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1617convert parameter in rest_api source incremental config by @willi-mueller in https://github.com/dlt-hub/dlt/pull/1649Full Changelog: https://github.com/dlt-hub/dlt/compare/0.5.1...0.5.2
Nothing published for this version
Nothing published for this version
Nothing published for this version
This is a major release (0.4 -> 0.5) in our versioning scheme so please review the breaking changes below. Most of them are relevant only for platform…
This is a major release (0.4 -> 0.5) in our versioning scheme so please review the breaking changes below. Most of them are relevant only for platform builders that use dlt internals. Some of the long-deprecated components were removed as well
PageNumberPaginator takes base_page and page arguments instead of initial_page. This allows to paginate APIs that number pages ie. from 0 or from 1. #1509credentials argument was removed from dlt.pipeline. #1537 Please use destination factories to instantiate destinations with explicit credentials. (https://dlthub.com/devel/general-usage/destination#pass-explicit-credentials)dlt.source or dlt.resource decorated function is passed a None in a default argument during a function call, it will be handled exactly like in regular Python function call. Previously such None would request argument injection from configuration. Please read more here: (https://github.com/dlt-hub/dlt/pull/1430)dlt.config.value and dlt.secrets.value were evaluating to None at runtime. Now they will evaluate to a sentinel value. All the existing code should be backward compatible. (https://github.com/dlt-hub/dlt/pull/1430)full_refresh flag of dlt.pipeline will be deprecated and replaced with dev_mode. (https://github.com/dlt-hub/dlt/pull/1063) and (https://dlthub.com/devel/general-usage/pipeline#do-experiments-with-dev-mode)round_robin from fifo as a default setting. You can switch back to the previous behavior and learn more about what this means here: (https://dlthub.com/docs/reference/performance#resources-extraction-fifo-vs-round-robin)SnowflakeCredentials) it will not be marked as resolved even if all required fields are provided. previously some were resolving and some were not. https://github.com/dlt-hub/dlt/pull/1489parse_native_representation never marks config as resolved. previously some were resolving and some were not. https://github.com/dlt-hub/dlt/pull/1489delta tables with delta-rs on top of filesystem destination. (https://github.com/dlt-hub/dlt/pull/1382)LanceDB destination and examples (https://github.com/dlt-hub/dlt/pull/1375)sql_cs_v1 (case sensitive) and sql_ci_v1 (case insensitive) to create SQL safe identifiers without snake case transformation (https://dlthub.com/devel/general-usage/naming-convention#available-naming-conventions)refresh modes to pipeline.run that allow to drop and recreate tables - with different granularity. (https://dlthub.com/devel/general-usage/pipeline#refresh-pipeline-data-and-state)filesystem destination only the bucket component is taken into account #1516requests.Session.send by @burnash in https://github.com/dlt-hub/dlt/pull/1452Pipeline.sql_client credentials forwarding by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1499dremio destination capabilities by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1527init_request by @burnash in https://github.com/dlt-hub/dlt/pull/1442We worked intensively on rest_api and sql_database:
sql_database @rudolfix https://github.com/dlt-hub/verified-sources/pull/478Full Changelog: https://github.com/dlt-hub/dlt/compare/0.4.12...0.5.1
Nothing published for this version
Nothing published for this version
Nothing published for this version
feat(pipeline): add an ability to auto truncate staging dataset by @IlyaFaer in https://github.com/dlt-hub/dlt/pull/1292
rest_client.paginate for pagination by @burnash in https://github.com/dlt-hub/dlt/pull/1287Full Changelog: https://github.com/dlt-hub/dlt/compare/0.4.11...0.4.12
RESTClient: building blocks (auths, paginators, response extractors etc.) to write REST API pipelines by @burnash
merge write disposition for athena Iceberg by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1315Full Changelog: https://github.com/dlt-hub/dlt/compare/0.4.10...0.4.11
Nothing published for this version
Nothing published for this version
Clickhouse destination by @Pipboyguy in https://github.com/dlt-hub/dlt/pull/1097
scd2 extension: pick your active record literal, defaults to NULL by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1275Full Changelog: https://github.com/dlt-hub/dlt/compare/0.4.9...0.4.10
SCD2 support by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1168 https://dlthub.com/devel/general-usage/incremental-loading#scd2-strategy
staging-optimized replace strategy for synapse by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1231pyarrow, pandas, connectorx or sqlalchemy backends when reading tables with sql_database. See README for details. https://github.com/dlt-hub/verified-sources/pull/425Full Changelog: https://github.com/dlt-hub/dlt/compare/0.4.8...0.4.9
SCD2 support by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1168 We are still working on BigQuery support) https://dlthub.com/devel/genera
staging-optimized replace strategy for synapse by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1231Final release is scheduled for next week
Nothing published for this version
Nothing published for this version
Add Dremio as a destination by @maxfirman in https://github.com/dlt-hub/dlt/pull/1026
spark flavor is not a default any longer by @rudolfix in https://github.com/dlt-hub/dlt/pull/1185
https://github.com/dlt-hub/dlt/pull/1185synapse and mssql bugfixes and improvements (INSERT VALUES UNION) by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/1174my_db default database and makes password / token mandatory by @rudolfix inFull Changelog: https://github.com/dlt-hub/dlt/compare/0.4.7...0.4.8
Nothing published for this version
Nothing published for this version
Custom destinations with @dlt.destination decorator by @sh-rp in https://github.com/dlt-hub/dlt/pull/1065
@dlt.destination decorator by @sh-rp in https://github.com/dlt-hub/dlt/pull/1065dlt.mark marker to materialize table schemas without data by @rudolfix in https://github.com/dlt-hub/dlt/pull/1122Full Changelog: https://github.com/dlt-hub/dlt/compare/0.4.6...0.4.7
Nothing published for this version
feat(airflow): expose the Airflow runner method to create custom DAGs by @IlyaFaer in https://github.com/dlt-hub/dlt/pull/1014
sql_database recoginizes end_value and row_order to return rows in range and optionally ordered. backfill and proper Airflow intervals support @rudolfix https://github.com/dlt-hub/verified-sources/pull/388/Full Changelog: https://github.com/dlt-hub/dlt/compare/0.4.5...0.4.6
enables google drive filesystem for sources and destinations (second one experimental, google drive listings are only eventually consistent!) by @Ilya
hard_delete and dedup_sort columns hint for merge by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/960 and https://dlthub.com/docs/general-usage/incremental-loading#delete-recordsFull Changelog: https://github.com/dlt-hub/dlt/compare/0.4.4...0.4.5
Nothing published for this version
passes incremental from apply hints to resource function by @rudolfix in https://github.com/dlt-hub/dlt/pull/953
sql_database @rudolfix (https://github.com/dlt-hub/verified-sources/pull/351)Full Changelog: https://github.com/dlt-hub/dlt/compare/0.4.3...0.4.4
Databricks destination by @steinitzu and @phillem15 in https://github.com/dlt-hub/dlt/pull/892
Full Changelog: https://github.com/dlt-hub/dlt/compare/0.4.2...0.4.3
Nothing published for this version
Fix the data type used in the from_db_type() method from MsSqlTypeMapper by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/863
from_db_type() method from MsSqlTypeMapper by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/863dlt.destination.adapters by @rudolfix in https://github.com/dlt-hub/dlt/pull/854Full Changelog: https://github.com/dlt-hub/dlt/compare/0.4.1...0.4.2
There are no significant breaking changes, but minor ones exist, please refer to #763 for details
This is a major dlt release (as per our semantic versioninghttps://github.com/dlt-hub/dlt?tab=readme-ov-file#adding-as-dependency). It brings several interesting new features like: schema evolution control, data contracts, deeper Pydantic integration, parametrized destinations, improvements to parallelism and data lineage + many more
There are no significant breaking changes, but minor ones exist, please refer to #763 for details
dlt.destinations module and instantiate them: by @steinitzu in https://github.com/dlt-hub/dlt/pull/746Plus some tooling changes
Full Changelog: https://github.com/dlt-hub/dlt/compare/0.3.25...0.4.1
Nothing published for this version
fixed attribute check: getuid -> geteuid by @jorritsandbrink in https://github.com/dlt-hub/dlt/pull/823
Full Changelog: https://github.com/dlt-hub/dlt/compare/0.4.1a1...0.4.1a2
load_id is generated in extract step and carried till the end to improve data lineage by @rudolfix in https://github.com/dlt-hub/dlt/pull/790
load_id is generated in extract step and carried till the end to improve data lineage by @rudolfix in https://github.com/dlt-hub/dlt/pull/790consult https://github.com/dlt-hub/dlt/issues/763 for a list of major changes compared to 0.3.x version
Full Changelog: https://github.com/dlt-hub/dlt/compare/0.4.1a0...0.4.1a1
Parametrized destinations by @steinitzu in https://github.com/dlt-hub/dlt/pull/746
Full Changelog: https://github.com/dlt-hub/dlt/compare/0.3.24...0.4.1a0
Your coding agent can read these notes before it upgrades. Set up the MCP server →