NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →

PyPI · #2032 most downloaded on PyPI

dbt-databricks

1.12.63.9M downloads/modatabricks/dbt-databricks ↗New releases in your daily digest, changelog included.

The Databricks adapter plugin for dbt

Last release 3 days ago
01 Oct 2026
Ships fairly regularly
a new release about every 3 weeks
Nearly every release is documented
notes for 60 of the last 60 stable releases
10 versions withdrawn
withdrawn after publishing
5 years old
186 releases · first in 2021

Release timeline

186 releases since 2021

202120222023202420252026

20212026

One column per quarter.

Releases

2.0.0rc18 May 2024pre-releasewithdrawn: Renaming as 1.8.0
Nothing published for this version
1.12.61 Oct 20263 sources agree
Support auth_type: env-oidc and auth_type: file-oidc for authenticating via workload identity federation, with a new oidc_token_filepath profile confi
Features

Support auth_type: env-oidc and auth_type: file-oidc for authenticating via workload identity federation, with a new oidc_token_filepath profile config for the latter (thanks @Gerrit-K!) (#1666)

Fixes

Handle SHOW GRANTS result columns case-insensitively so grant reconciliation does not crash when connectors return lowercase names (thanks @TangoEnSkai!) (#1650 resolves #782)

Keep the existing table when a V1 view model that replaces it fails to build, matching V2 behavior by @sd-db in #1688

Use in-place ALTER for an ordinary view only when the existing relation is also an ordinary view by @sd-db in #1687

Apply only changed table- and column-level databricks_tags to materialized views and streaming tables while preserving tags across replacements by @sd-db in #1686

Refresh a materialized view without an automatic schedule when a run applies in-place configuration changes, instead of leaving its data stale by @sd-db in #1686

Under the Hood

Raise the dbt-core upper bound to <1.12.6 to include dbt-core 1.12.4 and 1.12.5 by @sd-db in #1689

Emit only changed databricks_tags keys in ALTER … SET TAGS by @sd-db in #1667

Emit only changed column-level databricks_tags keys in ALTER COLUMN … SET TAGS, without unsetting remote-only tags by @sd-db in #1668

Document serverless environment configuration for Python models (thanks @TangoEnSkai!) (#1649 resolves #1055)

New Contributors

@TangoEnSkai made their first contribution in #1649

@Gerrit-K made their first contribution in #1666

Full Changelog: v1.12.5...v1.12.6

1.12.329 Jul 20263 sources agree

dbt-databricks 1.12.3 (Jul 29, 2026)

Features

Support catalog_database in v2 catalogs.yml to route Unity catalog models to a physical catalog independent of the dbt catalog name (requires dbt-core>=1.12 and dbt-adapters>=1.24.4). (thanks @aahel!) (#1590)

Fixes

Prevent concurrent connection opens from racing during lazy SDK configuration initialization by @sd-db in #1606
Avoid treating unchanged Streaming Table databricks_tags as configuration changes by diffing against existing table tags. by @sd-db in #1602 (resolves #1601)
Allow dropping a column that has governed tags by @saishreeeee in #1597 (resolves #1323)
Fix view materialization incorrectly producing a no-op instead of forcing recreation when --full-refresh is provided alongside view_update_via_alter: true and use_materialization_v2: true (thanks @aarushisingh04!) (#1456 resolves #1404)
Support dbt clone and rebuilds over an existing shallow clone by @saishreeeee in #1592 (resolves #1165, follow-up by @sd-db in #1608)
Fix managed Iceberg Python models failing with MANAGED_TABLE_FORMAT by emitting .format("iceberg") instead of the parquet sentinel from resolve_file_format (thanks @Divya-Kovvuru-0802!) (#1593 resolves #1591)
Quote generated column identifiers in incremental strategies so non-ASCII column names no longer fail on subsequent runs (thanks @ash2shukla!) (#1595 resolves #1594)
Handle missing or empty view-definition metadata when creating materialized views from streaming tables or newly-created materialized views (thanks @aarushisingh04!) (#1462 resolves #1459)

Under the Hood

Remove the unused keyring dependency and its transitive packages by @sd-db in #1588
Raise the dbt-core upper bound to <1.12.1 to include dbt-core 1.12.0 by @sd-db in #1605

New Contributors

@saishreeeee made their first contribution in #1587
@ash2shukla made their first contribution in #1595
@Divya-Kovvuru-0802 made their first contribution in #1593

Full Changelog: v1.12.2...v1.12.3

Open source →

1.12.29 Jul 20263 sources agree

dbt-databricks 1.12.2 (Jul 9, 2026)

Features

Add catalogs.yml v2 support (requires use_catalogs_v2: true in dbt-core) (#1440)
Add skip_optimize model config to opt out of the post-materialization OPTIMIZE call without dropping zorder / liquid_clustered_by / auto_liquid_cluster from the table definition. Useful when OPTIMIZE is delegated to Predictive Optimization or scheduled out of band. Complements the existing run-wide DATABRICKS_SKIP_OPTIMIZE var by allowing project-, folder-, or model-level opt-out via standard dbt config inheritance (#703).
Support the connector's Rust kernel backend via connection_parameters: {use_kernel: true} for SQL warehouses, with personal access token or Databricks OAuth (M2M/U2M) auth (requires databricks-sql-connector[kernel] on Python 3.10+; Azure service principals are not supported by the kernel) (#1576)

Fixes

Stop dropping existing constraints on incremental runs when contract.enforced is false (#1557)
Recognize the skip_not_matched_step merge config in the adapter config schema. It was previously declared with a typo (skip_non_matched_step); the merge macro already read the correct key, so merge behavior is unchanged (#1562).
Honor the expression field on primary_key constraints on the V1 materialization path. A primary key declared with expression: RELY (or any trailing clause) previously had its expression silently dropped. (#1551)
Apply column-level databricks_tags for incremental models on the V1 materialization path (#1520 closes #1307)
Raise a DbtRuntimeError when a Python model job run terminates with a non-success result_state (e.g. FAILED/TIMEDOUT) instead of returning silently (#1477)
Fix PK/FK constraints declaring an expression (e.g. RELY) being dropped and re-added on every incremental run. Regression: changing the expression on an existing PK/FK (RELY↔NORELY, or an expression-form FK's target) is no longer applied on incremental runs — use --full-refresh. (#1552 closes #1513)
Honor incremental_apply_config_changes in the V1 incremental merge path, allowing users to skip metadata diff queries (tags, column_tags, constraints, column_masks, tblproperties, describe_extended) when set to false. Matches the existing V2 behavior. (#1467 partially resolves #1402)
Fix column-level databricks_tags on Unity Catalog views updated via ALTER (view_update_via_alter: true) (#1526 closes #1525)
Apply tblproperties to metric_view models at create time, not only on a later alter/replace run (#1530 closes #1527)
Only emit INSERT ... BY NAME in the replace_where/microbatch strategies on DBR 18.0+ (and SQL warehouses), since older clusters reject the BY NAME ... REPLACE WHERE combination with a parse error (#1539 resolves #1532)
Fix materialized views always rebuilding because Databricks-internal tblproperties were read as configuration drift; the diff now compares only the configured properties (#1350 closes #1314).
Stop metric views with view_update_via_alter from re-issuing a redundant ALTER VIEW ... AS on every run (#1546)
Fix column comments being permanently dropped from views when view_update_via_alter issues ALTER VIEW AS; reapply persisted column comments after the query update (#1357)

Under the Hood

Raise the databricks-sql-connector upper bound to <4.3.1 to support 4.3.0 (#1518)
Raise the dbt-adapters upper bound to <1.25.0 (#1507)
Raise the databricks-sdk upper bound to <0.118.0 to pick up 0.117.0, which fixes WorkspaceClient construction failing with CONTEXT_UNAVAILABLE_FOR_REMOTE_CLIENT on Spark Connect clusters (#1517 closes #1252)
Raise the dbt-core upper bound to <1.11.13 to include dbt-core 1.11.12 (#1578)
Instrument add_query, get_relation_config, is_uniform, has_dbr_capability, and is_cluster for dbt's record/replay framework (test-only, no runtime impact) (#1508)
Add a weekly Kernel Integration Tests workflow that runs the functional suite against the SQL-warehouse profile through the connector's Rust kernel backend (DBT_DATABRICKS_USE_KERNEL=1) (test-only, no runtime impact) (#1576).
Substantially expand adapter test coverage across the functional and unit suites: broaden functional coverage for materializations (table replace-in-place, view→table conversion), incremental strategies and full-refresh recreate, constraints and row filters, tags and column tags, streaming tables, materialized and metric views, copy_into, clone, seeds, SQL UDFs, split_part, sync_all_columns type widening, catalog table_format, and Python models — shifting assertions to server-observable state rather than compiled SQL or log output; add unit coverage for the live event/logging classes; make stateful functional tests rerun-safe; and add CI upkeep (a weekly Python-model notebook-folder purge and a longer unit-test timeout) (test-only, no runtime impact). (#1511, #1512, #1514, #1521, #1522, #1523, #1524, #1528, #1529, #1531, #1533, #1534, #1535, #1536, #1537, #1538, #1541, #1542, #1543, #1544, #1545, #1548, #1550, #1553, #1558, #1559, #1564, #1565, #1566, #1568, #1569, #1570, #1571, #1573, #1579, #1580, #1581, #1582)

New Contributors

@aahel made their first contribution in #1440
@moomindani made their first contribution in #1512
@tomaioo made their first contribution in #1469
@ajhlee-dbt made their first contribution in #1508

Full Changelog: v1.12.1...v1.12.2

Open source →

1.12.110 Jun 2026

Expose job_id, job_run_id, and task_run_id from the Databricks Jobs dbt_task runtime in adapter_response, enabling correlation between dbt runs and Da

dbt-databricks 1.12.1 (June 10, 2026)

Features

Expose job_id, job_run_id, and task_run_id from the Databricks Jobs dbt_task runtime in adapter_response, enabling correlation between dbt runs and Databricks workflow executions via run_results.json by @sd-db in #1451, closes #722
Add support for SPOG (Single Point of Gateway) hosts: account-level vanity URLs with ?o=<workspace-id> in http_path route correctly for both data-plane (SQL) and control-plane (REST/Jobs/Workspace API) traffic. Requires databricks-sql-connector >= 4.2.6 and databricks-sdk >= 0.76.0. by @sd-db in #1479

Fixes

Ignore the server-set delta.parquet.compression.codec tblproperty when diffing relation configs, so streaming tables and materialized views are not flagged as changed after the backend started stamping it automatically by @sd-db in #1489
Fix missing f-string prefix in JobRunsApi.submit debug log by @Dev-X25874 in #1471
Fix capability-branching macros falling through to their legacy path at parse/compile time on SQL warehouses. The parse-time stub of has_dbr_capability now returns True on warehouse profiles for capabilities flagged sql_warehouse_supported, so macros select the modern branch during compilation instead of the legacy fallback. by @sd-db in #1449, closes #1331
Fix snapshots not applying databricks_tags on columns by @themattmorris in #1442, closes #1441
Skip DESCRIBE TABLE EXTENDED ... AS JSON for foreign/federated tables in get_columns_in_relation, avoiding repeated failures and extra latency on those sources by @willweld in #1472
EXTRACT_CLUSTER_ID_FROM_HTTP_PATH_REGEX now stops the capture at ? / &, so any trailing query string on http_path no longer corrupts the extracted cluster id. Latent issue on legacy hosts; the fix unblocks SPOG cluster paths. by @sd-db in #1479
Gate column-level constraints on contract.enforced to match the existing model-level gate, ensuring column-level NOT NULL / PK / FK / CHECK constraints are only applied when contract.enforced: true under use_materialization_v2: true by @sd-db in #1470, closes #1381
Fix managed Iceberg incremental models configured with partition_by silently losing their clustering after the first incremental run. Managed Iceberg stores partition_by as liquid clustering server-side, so the reconciler now treats partition_by as the desired clustering and no longer issues a spurious ALTER TABLE ... CLUSTER BY NONE by @sd-db in #1496, closes #1495

Under the Hood

Defer SDK Config construction to connection-open time so offline paths (dbt parse/list/compile) don't trigger the host-metadata probe introduced in databricks-sdk>=0.103; as a side effect, auth errors now surface at first connection rather than during profile parsing. by @sd-db in #1474
Bump ceilings on databricks-sdk (now <0.105.0) and databricks-sql-connector[pyarrow] (now <4.3.0) to admit newer releases; floors unchanged. by @sd-db in #1474
BREAKING: users who relied on column-level constraints (NOT NULL, primary key, foreign key, check) being applied under use_materialization_v2: true without contract.enforced: true must now set contract.enforced: true explicitly on the model. by @sd-db in #1470
Bump upper bound of dbt-core to <1.11.12 to include dbt-core 1.11.9, 1.11.10, and 1.11.11 by @sd-db in #1505

Full Changelog: https://github.com/databricks/dbt-databricks/compare/v1.12.0...v1.12.1

Open source →

Additional notes2 sources agree

Features

Expose job_id, job_run_id, and task_run_id from the Databricks Jobs dbt_task runtime in adapter_response, enabling correlation between dbt runs and Databricks workflow executions via run_results.json (#1451 closes #722)
Add support for SPOG (Single Point of Gateway) hosts: account-level vanity URLs with ?o=<workspace-id> in http_path route correctly for both data-plane (SQL) and control-plane (REST/Jobs/Workspace API) traffic. Requires databricks-sql-connector >= 4.2.6 and databricks-sdk >= 0.76.0. (#1479)

Fixes

Ignore the server-set delta.parquet.compression.codec tblproperty when diffing relation configs, so streaming tables and materialized views are not flagged as changed after the backend started stamping it automatically (#1489)
Fix missing f-string prefix in JobRunsApi.submit debug log (#1471)
Fix capability-branching macros falling through to their legacy path at parse/compile time on SQL warehouses. The parse-time stub of has_dbr_capability now returns True on warehouse profiles for capabilities flagged sql_warehouse_supported, so macros select the modern branch during compilation instead of the legacy fallback. (#1449 closes #1331)
Fix snapshots not applying databricks_tags on columns (#1442 closes #1441)
Skip DESCRIBE TABLE EXTENDED ... AS JSON for foreign/federated tables in get_columns_in_relation, avoiding repeated failures and extra latency on those sources (#1472)
EXTRACT_CLUSTER_ID_FROM_HTTP_PATH_REGEX now stops the capture at ? / &, so any trailing query string on http_path no longer corrupts the extracted cluster id. Latent issue on legacy hosts; the fix unblocks SPOG cluster paths.
Gate column-level constraints on contract.enforced to match the existing model-level gate, ensuring column-level NOT NULL / PK / FK / CHECK constraints are only applied when contract.enforced: true under use_materialization_v2: true (#1470 closes #1381)
Fix managed Iceberg incremental models configured with partition_by silently losing their clustering after the first incremental run. Managed Iceberg stores partition_by as liquid clustering server-side, so the reconciler now treats partition_by as the desired clustering and no longer issues a spurious ALTER TABLE ... CLUSTER BY NONE (#1496 closes #1495)

Under the Hood

Make the incremental constraint functional tests rerun-safe so a pytest --reruns retry no longer inherits mutated state (rewritten schema.yml, half-built relations) from the failed attempt (test-only, no runtime impact). (#1503)
Make the column-tag functional tests rerun-safe so a pytest --reruns retry no longer inherits mutated state (updated schema.yml, leftover column tags, a running streaming-table query) from the failed attempt (test-only, no runtime impact). (#1499)
Raise the dbt-tests-adapter test-dependency floor to >=1.20.0 to pick up its persist_docs fixture typo fix (test-only, no runtime impact) (#1490)
Defer SDK Config construction to connection-open time so offline paths (dbt parse/list/compile) don't trigger the host-metadata probe introduced in databricks-sdk>=0.103; as a side effect, auth errors now surface at first connection rather than during profile parsing. (#1474)
Bump ceilings on databricks-sdk (now <0.105.0) and databricks-sql-connector[pyarrow] (now <4.3.0) to admit newer releases; floors unchanged. (#1474)
Tighten the databricks-sql-connector ceiling to patch level (<4.3.0 → <4.2.7) so patch upgrades require an intentional bump; the locked version stays at 4.2.6. (#1497, #1498)
Stabilize the TestChangingSchema* Python-model functional tests under min-deps (dbt-core 1.11.2), where a sibling class's source schema.yml could leak into their parse and fail with EnvVarMissingError. (#1488)
BREAKING: users who relied on column-level constraints (NOT NULL, primary key, foreign key, check) being applied under use_materialization_v2: true without contract.enforced: true must now set contract.enforced: true explicitly on the model.
Bump upper bound of dbt-core to <1.11.12 to include dbt-core 1.11.9, 1.11.10, and 1.11.11 (#1505)

1.12.018 May 20263 sources agree

Add support for metric views as a materialization by @benc-db in #1285

Features

Add support for metric views as a materialization by @benc-db in #1285
Add support for row filters by @sd-db in #1294
Add support for Python UDFs by @sd-db in #1336
Add support for key-only databricks_tags for table and column tagging. This can now be configured by setting tag values to empty strings "" or None. by @canbekley in #1339
Fetch relation metadata like constraints, column masks, row filters, etc with a single DESCRIBE TABLE EXTENDED ... AS JSON call, replacing multiple information_schema queries. Falls back to information_schema on older runtimes. Gated behind use_describe_as_json_for_relation_metadata behavior flag, off by default. by @tejassp-db in #1432
Support SCHEDULE EVERY and TRIGGER ON UPDATE refresh modes for materialized views and streaming tables, with parser and diff coverage so relations whose actual refresh is not CRON no longer crash on subsequent runs by @sd-db in #1434, closes #1293

Fixes

Fix metric_view failing with METRIC_VIEW_INVALID_VIEW_DEFINITION when models use bare {{ ref(...) }} for the source: field by @sd-db in #1430, closes #1361
Fix RefreshConfig.__eq__ self/other typo where two configs with the same cron but different time_zone_value compared equal by @sd-db in #1434
Fix streaming-table DROP-SCHEDULE path that was silently filtered out of the changeset by @sd-db in #1434
Use pydantic v1-compatible API in refresh.py so the adapter imports on environments shipping pydantic v1 by @sd-db in #1461

Under the Hood

BREAKING: databricks_tags defined at different hierarchy levels (e.g. project-level and model-level) now merge additively instead of the child config completely replacing the parent. by @canbekley in #1340
Skip information_schema.tags and information_schema.column_tags metadata fetches when table tags and column tags are not configured on a model. by @tejassp-db in #1387

Full Changelog: v1.11.8...v1.12.0

Open source →

1.11.811 May 2026

feat: Add invocation_id to default query comment by @GuyEshdat in #1378

What's Changed

Features

feat: Add invocation_id to default query comment by @GuyEshdat in #1378

Fixes

fix: enforce 255-character identifier length limit for Databricks relations by @psaikaushik in #1391
fix: prevent spurious MicrobatchConcurrency warning on every run (#1406) by @sd-db in #1409
fix: prevent _cache_dbr_capabilities from caching None-version entries (#1398) by @sd-db in #1414
fix: missing metadata handling in is_hudi and is_iceberg by @aarushisingh04 in #1435
test: fix failing TestStreamingTableLiquidClusteringChanges by @sd-db in #1436
fix: stop emitting insert_overwrite noise warning on every run (#1305) by @sd-db in #1425
fix: warn on contract.enforced=true for materialized_view by @sd-db in #1421
fix: suppress use_managed_iceberg warning for non-Iceberg projects by @tejassp-db in #1429
fix: refresh materialized_view when databricks_tags is set (#1419) by @sd-db in #1424
fix: prevent agate type-conflict during catalog merge (#1392) by @sd-db in #1428
fix: suppress warning on materialization v2 behavior flags. by @tejassp-db in #1431

New Contributors

@psaikaushik made their first contribution in #1391
@GuyEshdat made their first contribution in #1378

Full Changelog: v1.11.7...v1.11.8

Open source →

Additional notes2 sources agree

Features

Add invocation_id to the default query comment (#1377)

Fixes

Fix latent AssertionError when evaluating is_hudi or is_iceberg properties on relations without metadata
Validate relation identifier length at creation time and raise a clear error when it exceeds Databricks' 255-character limit (#1309)
Fix spurious MicrobatchConcurrency behavior-change warning firing on every run regardless of whether the project contained microbatch models (#1406)
Stop emitting the insert_overwrite will perform a dynamic insert overwrite warning on every insert_overwrite run on SQL warehouses; warn instead only when use_replace_on_for_insert_overwrite is enabled but the cluster's DBR version does not support REPLACE ON (#1305)
Fix DBR capability cache being permanently poisoned by a transient version-query failure (#1398)
Fix spurious use_materialization_v2 behavior-change warning firing on table, view, seed, and incremental materializations (#1089)
Fix spurious use_managed_iceberg behavior-change warning firing on every run for projects that don't use Iceberg (#1266)
Warn when contract.enforced: true is set on a materialized_view model (#1279)
Fix materialized_view models with databricks_tags silently going stale on dbt run. MaterializedViewAPI._describe_relation was not fetching information_schema.tags, so existing tags always parsed as empty, producing a spurious tag diff that routed the materialization to ALTER ... SET TAGS instead of REFRESH MATERIALIZED VIEW (#1419)
Fix dbt docs generate failing with RuntimeError: Tables contain columns with the same names ... but different types during catalog merge across schemas (#1392)

1.11.720 Apr 2026

feat: Add Notebook-scoped packages for command submits or notebook job run by @fedemgp in #1321

What's Changed

Features

feat: Add Notebook-scoped packages for command submits or notebook job run by @fedemgp in #1321

Fixes

fix: convert workflow job spec to SDK dataclasses before jobs.create() (#1338) by @aarushisingh04 in #1364
fix: enable TestWorkflowJob functional test that was unreachable on all profiles by @sd-db in #1401
fix: prevent duplicate alias in --empty mode for inline ref/source by @sd-db in #1397
fix: add insert-by-name support to microbatch and replace_where strategies (#1338) by @aarushisingh04 in #1348
fix: capability detection for named compute (databricks_compute) by @trouze in #1355

Under the Hood

chore: bump dbt-core upper bound to <1.11.9 by @sd-db in #1400
feat(PECOBLR-2328): bump databricks-sql-connector upper bound to <4.1.6 by @tejassp-db in #1363

New Contributors

@tejassp-db made their first contribution in #1365
@trouze made their first contribution in #1355
@fedemgp made their first contribution in #1321

Full Changelog: v1.11.6...v1.11.7

Open source →

Additional notes2 sources agree

Features

Enable Notebook scoped python packages installation

Fixes

Fix capability detection (e.g., table_format='iceberg') failing on named compute (databricks_compute) due to lazy DBR version caching (#1355)
Fix workflow_job Python model submission method failing with dictionary attribute error (#1360)
Fix TestWorkflowJob functional test that was unreachable on all profiles due to incorrect skip list, wrong model fixture, and invalid max_retries parameter (#1360)
Fix column order mismatch in microbatch and replace_where incremental strategies by using INSERT BY NAME syntax (#1338)
Fix dbt run --empty failing with inline ref() / source() aliases (dbt-labs/dbt-adapters#660)

Under the Hood

Bump databricks-sql-connector upper bound to <4.1.6, enabling users to opt into _respect_server_retry_after_header (available in connector 4.1.5) for server-directed retry behavior via connection_parameters (#1363)
Bump upper bound of dbt-core to <1.11.9 to include dbt-core 1.11.8

Open source →

1.11.03 Nov 2025Breaking3 sources agree

Breaking Change: Requires Databricks Runtime 12.2 LTS or higher

Features

Support databricks_tags for MV/STs
Add support for scalar SQL functions (SQL UDFs) (1197)
Add liquid clustering config for materialized views and streaming tables (thanks @reflection!) (1101)
BEHAVIOR CHANGE: Add official support for insert_overwrite incremental strategy for SQL warehouses. This strategy now uses REPLACE ON syntax for all compute types (previously INSERT OVERWRITE). This behavior is gated behind behavior flag use_replace_on_for_insert_overwrite which defaults true (1025)
Add support for Databricks query tags; NOTE: This feature is still in private preview, so it is not generally usable from the adapter yet.
Add support for managed iceberg when table_format is set to iceberg. This behavior is gated behind behavior flag use_managed_iceberg which defaults to false
Support delete+insert incremental strategy (thanks @canbekley!) (1217); if you have a current DBR (17.1+), this uses REPLACE ON for efficient replacement. Otherwise uses a 'delete from' statement follow by an insert statement (i.e. it is not atomic).

Fixes

BREAKING: Fix column order mismatch bug in incremental models by using INSERT BY NAME syntax (#1211)
- When using on_schema_change: sync_all_columns, dbt previously used positional column matching in INSERT statements, causing values to be inserted into wrong columns when column order changed
- Now uses Databricks INSERT BY NAME syntax to match columns by name instead of position, preventing data corruption
- Breaking Change: Requires Databricks Runtime 12.2 LTS or higher
- Users on older runtimes should pin to dbt-databricks 1.10.x
- Affects all incremental strategies: append, insert_overwrite, replace_where, and merge (via table creation)
Fix case-sensitivity issues with column name handling in persist_docs and config diff operations (#1215)
- Fixed KeyError when column names in models had different casing than YAML schema definitions
- Improved efficiency of column tags and comments change detection to use case-insensitive comparison
Use backtick quoting for everything to avoid errors with special characters (1186)
Ensure column compare always uses lower case names (since Databricks stores internally as lower case) (1190)
Fix incompatible schema error during streaming table creation (1235)
Reintroduce support for external table type so as not to break users (1240)

Under the Hood

Materialized views now uses CREATE OR REPLACE where appropriate, instead of DROP + CREATE
Refactor to use Databricks SDK for API calls (1185)
Update dependency versions, and start using uv (1199)
Upgrade ruff and mypy (1207)
Allow create or replace semantics on full refresh in Mat V2 (1210)
Add centralized DBR capability system for managing version-dependent features with per-compute caching (#1218)
BREAKING: Removing the 'use_info_schema_for_columns' behavior flag, as we have a better mechanism for getting complex type information - DESCRIBE EXTENDED ... AS JSON. This is a breaking change because it requires a modern DBR (or SQL Warehouse) in order to function (1226). If this is breaking for you, you will either need to upgrade your cluster to 16.4+ or pin dbt-databricks to 1.10.x.
Use atomic CREATE OR REPLACE instead of DROP + CREATE for managed Iceberg tables
BREAKING: Drop support for python 3.9, adds 3.13 (1240)
BREAKING: Flipping the default for USE_USER_FOLDER_FOR_PYTHON to true (1248)

Open source →

1.10.113 Sept 2025

fix: Set correct cluster type on databricks connection by @ericj-db in https://github.com/databricks/dbt-databricks/pull/1158

What's Changed

fix: Set correct cluster type on databricks connection by @ericj-db in https://github.com/databricks/dbt-databricks/pull/1158
fix: Handle ANSI mode for pandas DataFrame conversion by @benc-db in https://github.com/databricks/dbt-databricks/pull/1157
chore: Modify integration workflow to simplify running external PRs by @benc-db in https://github.com/databricks/dbt-databricks/pull/1162
fix: add check in _get to validate model had gettable config by @colin-rogers-dbt in https://github.com/databricks/dbt-databricks/pull/1164
fix: Remove external path on intermediate tables for python incremental models by @gsolasab in https://github.com/databricks/dbt-databricks/pull/1161
fix: get_columns_in_relation branching for STs by @ericj-db in https://github.com/databricks/dbt-databricks/pull/1167
fix: Improve perf on schema enumeration/validation by @benc-db in https://github.com/databricks/dbt-databricks/pull/1168
fix: don't pass nodes that don't have database attribute to build_catalog relation by @colin-rogers-dbt in https://github.com/databricks/dbt-databricks/pull/1170
Fix: model-level override logic for is_cluster by @ericj-db in https://github.com/databricks/dbt-databricks/pull/1172

Full Changelog: https://github.com/databricks/dbt-databricks/compare/v1.10.10...v1.10.11

Open source →

Additional notes2 sources agree

Fixes

Improve ANSI mode error handling for Python models and add debug instrumentation (1157)
Remove external path on intermediate tables for incremental models (with Materialization V2) (1161)
Fix get_columns_in_relation branching logic for streaming tables to prevent it from running AS JSON
Fix model-level compute override connection logic that was causing invalid spark configs to be set on SQL warehouses

Under the hood

Improve performance of schema enumeration/validation (1168)

Open source →

1.10.529 Jul 2025withdrawn: Materialization is broken for older DBR versions due to limited DESCRIBE TABLE EXTENDED .. AS JSON support

Fix inefficient query when getting column schema for MV/STs by @ericj-db in https://github.com/databricks/dbt-databricks/pull/1080

What's Changed

Fix inefficient query when getting column schema for MV/STs by @ericj-db in https://github.com/databricks/dbt-databricks/pull/1080
Fix TypedConstraint equality check by @ericj-db in https://github.com/databricks/dbt-databricks/pull/1084
Add cloning for the external tables by @samgans in https://github.com/databricks/dbt-databricks/pull/1079
Do cascading drop when primary key constraint is removed from config by @ericj-db in https://github.com/databricks/dbt-databricks/pull/1088
[FIX]: Fix issue with cloning of external tables by @samgans in https://github.com/databricks/dbt-databricks/pull/1095
Ensure we can append with parquet, and delegate to DBSQL for possible errors by @benc-db in https://github.com/databricks/dbt-databricks/pull/1099
Handle complex data types that are truncated by DESCRIBE TABLE by @ericj-db in https://github.com/databricks/dbt-databricks/pull/1093
Fix MV/ST materializations with complex data types by @ericj-db in https://github.com/databricks/dbt-databricks/pull/1103
Removing outdated docs and directing users to the main hosted documentation. by @benc-db in https://github.com/databricks/dbt-databricks/pull/1108

New Contributors

@samgans made their first contribution in https://github.com/databricks/dbt-databricks/pull/1079

Full Changelog: https://github.com/databricks/dbt-databricks/compare/v1.10.4...v1.10.5

Open source →

Additional notes2 sources agree

Features

Add cloning support for the the external tables (thanks @samgans!) (1079)

Fixes

Fix inefficient query when getting column schema for MV/STs (1074)
Fix bug causing false positives in diffing constraints between existing relation and model config for incremental runs (1081)
Fix bug causing "main is not being called during running model" errors for some view updates (1077)
Fix bug that causes materialization (V2) to fail when data type is long enough to be truncated by DESCRIBE TABLE (1083)
Fix the bugs with external tabls cloning 1095 (thanks @samgans!)
Fix MV/ST materializations with complex data types (1100)

Under the Hood

Dropping primary key constraints in incremental runs now trigger cascading deletes (i.e. foreign key constraints referencing it will also be dropped)

Open source →

1.10.424 Jun 2025

Wait for library installation on python submission by @ericj-db in https://github.com/databricks/dbt-databricks/pull/1031

What's Changed

Wait for library installation on python submission by @ericj-db in https://github.com/databricks/dbt-databricks/pull/1031
Fix insert overwrite compile check when using model level compute override by @ericj-db in https://github.com/databricks/dbt-databricks/pull/1035
Fix multiple foreign key constraints bug by @ericj-db in https://github.com/databricks/dbt-databricks/pull/1039
Downgrade INSERT_OVERWRITE error to warning for SQL warehouses by @ericj-db in https://github.com/databricks/dbt-databricks/pull/1038
Fix location_root case sensitivity by @ericj-db in https://github.com/databricks/dbt-databricks/pull/1043
Set column comments on create for MV/ST by @ericj-db in https://github.com/databricks/dbt-databricks/pull/1046
Add column mask support for tables and incremental by @ericj-db in https://github.com/databricks/dbt-databricks/pull/1033
Fix insert_overwrite handling of schema update by @ericj-db in https://github.com/databricks/dbt-databricks/pull/1058
Add metric_view as a relation type by @ericj-db in https://github.com/databricks/dbt-databricks/pull/1065
Add column-level tag support for table, MV, ST, incremental by @ericj-db in https://github.com/databricks/dbt-databricks/pull/1062
Fix ACL permissions not applied to notebook jobs created by DBT Python model by @case-k-git in https://github.com/databricks/dbt-databricks/pull/1015
Disable unsetting table and column tags by @ericj-db in https://github.com/databricks/dbt-databricks/pull/1071

Full Changelog: https://github.com/databricks/dbt-databricks/compare/v1.10.2...v1.10.4

Open source →

Additional notes2 sources agree

Features

Support column tags (with Materialization V2) (649)
Support column masking (with Materialization V2) (670)

Fixes

Fix bug where schema update causes insert_overwrite strategy to fail on subsequent runs (1057)
Fix ACL permissions not applied to notebook jobs created by DBT Python model (1015)
Fix bug where model run fails if catalog contains a metric view (1045)
Disable removing table tags and column tags to allow dbt tags to coexist with automatically added Databricks tags (1069)
Fix bug where python model run starts before all libraries are installed on the cluster (1028)
Fix bug where incorrect insert overwrite exception was being thrown when overriding compute at model level (1032)
Fix bug that was causing failures when multiple foreign key constraints are defined in a model (1034)
Fix location_root config not being treated case sensitive (1037)
Fix column comments for streaming tables and materialized views (1049)

Under the Hood

Update to dbt-core 1.10.1
Update to dbt-common 1.24.0

Open source →

1.10.129 Apr 2025

Fix: DBT Python Model Canceled Notebook Job Treated as Successful by @case-k-git in https://github.com/databricks/dbt-databricks/pull/985

What's Changed

Fix: DBT Python Model Canceled Notebook Job Treated as Successful by @case-k-git in https://github.com/databricks/dbt-databricks/pull/985
Adding internal table property delta.columnMapping.maxColumnId to ignore_list (#987) by @iamatharkhan in https://github.com/databricks/dbt-databricks/pull/991
Strip trailing semi-colons. by @benc-db in https://github.com/databricks/dbt-databricks/pull/990
Fix regression related to insert_overwrite by @benc-db in https://github.com/databricks/dbt-databricks/pull/992
Replace illegal characters in unique temporary table names by @geo-martino in https://github.com/databricks/dbt-databricks/pull/995
Unfreeze requires_full_refresh on DatabricksRelationChangeSet by @geo-martino in https://github.com/databricks/dbt-databricks/pull/1000
Drop pre-existing backup name from the adapter's cache by @geo-martino in https://github.com/databricks/dbt-databricks/pull/999
Fixing view replace logic by @benc-db in https://github.com/databricks/dbt-databricks/pull/1004

New Contributors

@iamatharkhan made their first contribution in https://github.com/databricks/dbt-databricks/pull/991
@geo-martino made their first contribution in https://github.com/databricks/dbt-databricks/pull/995

Full Changelog: https://github.com/databricks/dbt-databricks/compare/v1.10.0...v1.10.1

Open source →

Additional notes2 sources agree

Fixes

Fix: DBT Python Model Canceled Notebook Job Treated as Successful (985)
Fix failures when models end with ';' (990)
Added internal table property delta.columnMapping.maxColumnId to ignore_list to allow enabling delta.columnMapping.mode without breaking concurrent job runs (thanks @ iamatharkhan!) (991)
For insert_overwrite, raise exception when using SQL Warehouse and inform users that such use causes truncate + insert (992)
Remove illegal characters in unique temporary table names which prevented dropping these tables on session close (thanks @geo-martino!) (995)
Dropping a pre-existing backup relation now removes the pre-existing relation name from the cache (thanks @geo-martino!). (999)
Error no longer thrown when setting requires_full_refresh attribute after comment is changed on view and view_update_via_alter is true (thanks @geo-martino!) (1000)
Many fixes related to how we handle view replacements, but specifically allowing views to function properly with using unique temporary names (thanks @geo-martino for help with validation!) (1004)

Open source →

1.10.08 Apr 2025

Support databricks OAuth M2M auth type. Updated OAuth readme doc with instructions.(827)

Brief summary

Features

Support databricks OAuth M2M auth type. Updated OAuth readme doc with instructions.(827)
Introduced use_materialization_v2 flag for gating materialization revamps. (844)
Introduce use_safer_relation_operations which will not change the production view/table unless new data can safely be ingested (927)
adding view_update_via_alter, to allow updating views by altering, instead of recreating (954)
Remover blockers to custom incremental strategies (972)

Under the Hood

Update pinned python SDK version to 0.46.0 (969)
Implement new constraint logic for use_materialization_v2 flag (846), (876)
Quote constraint names if we quote column names (thanks @samuelberntzen!) (966)
Streamlining debug logging to make it more usable (946)
Upgrading Databricks SQL Connector to V4 (962)
Validation of sample mode (961)
Move liquid clustering behavior to on_config_change approach for incremental (968)
Move column coments to on_config_change approach (981)

Fixes

Switch to using full_data_type column when using info schema for column info (950)

While waiting for full documentation (coming shortly to the dbt Labs doc site), you can look at the discussion for the 1.10.0 alpha release in the discussion section for how to use the new features.

Open source →

Additional notes2 sources agree

Features

Support databricks OAuth M2M auth type. Updated OAuth readme doc with instructions.(827)
Introduced use_materialization_v2 flag for gating materialization revamps. (844)
Introduce Tables V2, including safe_table_create which will not change the production table unless new data can safely be ingested (927)
Views V2, including renaming safe_table_create to use_safer_relation_operations, and adding view_update_via_alter, to allow updating views by altering, instead of recreating (954)
Remover blockers to custom incremental strategies (972)

Under the Hood

Update pinned python SDK version from 0.17.0 to 0.41.0. (827)
Update pinned python SDK version from 0.41.0 to 0.46.0 (969)
Implement new constraint logic for use_materialization_v2 flag (846), (876)
- Quote constraint names if we quote column names (thanks @samuelberntzen!) (966)
Streamlining debug logging to make it more usable (946)
Upgrading Databricks SQL Connector to V4 (962)
Validation of sample mode (961)
Move liquid clustering behavior to on_config_change approach for incremental (968)
Move column coments to on_config_change approach (981)

Fixes

Switch to using full_data_type column when using info schema for column info (950)

1.9.09 Dec 2024Deprecated3 sources agree

Prepare for python typing deprecations (837)

What's Changed

Features

Add config for generating unique tmp table names for enabling parralel merge (thanks @huangxingyi-git!) (854)
Add support for serverless job clusters on python models (706)
Add 'user_folder_for_python' behavior to switch writing python model notebooks to the user's folder (835)
Merge capabilities are extended (739) to include the support for the following features (thanks @mi-volodin):
- with schema evolution clause (requires Databricks Runtime 15.2 or above);
- when not matched by source clause, only for delete action
- matched, not matched and not matched by source condition clauses;
- custom aliases for source and target tables can be specified and used in condition clauses;
- matched and not matched steps can now be skipped;
Allow for the use of custom constraints, using the custom constraint type with an expression as the constraint (thanks @roydobbe). (792)
Add "use_info_schema_for_columns" behavior flag to turn on use of information_schema to get column info where possible. This may have more latency but will not truncate complex data types the way that 'describe' can. (808)
Add support for table_format: iceberg. This uses UniForm under the hood to provide iceberg compatibility for tables or incrementals. (815)
Add include_full_name_in_path config boolean for external locations. This writes tables to {location_root}/{catalog}/{schema}/{table} (823)
Add a new workflow_job submission method for python, which creates a long-lived Databricks Workflow instead of a one-time run (thanks @kdazzle!) (762)
Allow for additional options to be passed to the Databricks Job API when using other python submission methods. For example, enable email_notifications (thanks @kdazzle!) (762)
Support microbatch incremental strategy using replace_where (825)

Fixes

Replace array indexing with 'get' in split_part so as not to raise exception when indexing beyond bounds (839)
Ensure columns that are added get backticked (859)

Under the Hood

Significant refactoring and increased testing of python_submissions (830)
Fix places where we were not properly closing cursors, and other test warnings (713)
Drop support for Python 3.8 (713)
Upgrade databricks-sql-connector dependency to 3.5.0 (833)
Prepare for python typing deprecations (837)
Fix behavior flag use in init of DatabricksAdapter (thanks @VersusFacit!) (836)
Restrict pydantic to V1 per dbt Labs' request (843)
Switching to Ruff for formatting and linting (847)
Refactoring location of DLT polling code (849)
Switching to Hatch and pyproject.toml for project config (853)

New Contributors

@huangxingyi-git made their first contribution in https://github.com/databricks/dbt-databricks/pull/811
@jelmerk made their first contribution in https://github.com/databricks/dbt-databricks/pull/820
@VersusFacit made their first contribution in https://github.com/databricks/dbt-databricks/pull/836

Full Changelog: https://github.com/databricks/dbt-databricks/compare/v1.8.6...v1.9.0

Open source →