PyPI · #2135 most downloaded on PyPI

pylance

12.0.03.5M downloads/molance-format/lance ↗New releases in your daily digest, changelog included.

python wrapper for Lance columnar format

Last release 17 days ago
17 Sep 2026
Ships fairly regularly
a new release about every 2 weeks
Most releases are documented
notes for 44 of the last 60 stable releases
2 versions withdrawn
withdrawn after publishing
4 years old
193 releases · first in 2022

Release timeline

193 releases since 2022

20222023202420252026

20222026

One column per quarter.

Releases

12.0.017 Sept 2026Deprecated
fix(substrait): decode deprecated timestamp literals as microseconds by @jonasdedden in #8832
What's Changed

Breaking Changes 🛠

feat(io)!: paginated directory listing on ObjectStore by @wjones127 in #8606

perf(mem_wal)!: serve the shard manifest from the store that wrote it by @hamersaw in #8640

feat(table): predecessor-conditioned publication for external manifest stores by @wkalt in #8800

feat(namespace)!: accept multiple columns for the merge insert on key by @wjones127 in #8915

feat: make data file parts serializable by @Xuanwo in #9072

fix(index)!: do not route json_extract predicates to JSON indices by @westonpace in #9101

New Features 🎉

feat(java): support segment-based distributed vector search by @summaryzb in #7169

feat(index): share IVF partition scans across batch vector queries by @sezruby in #7640

feat(mem_wal): make write backpressure injectable and account index memory by @hamersaw in #7831

feat(python): add Bitmap binding for RoaringBitmap by @wjones127 in #7837

feat(python): add LanceDataset.slice() method by @Sapnilb15 in #8059

fix(mem-wal): resolve writer shard spec identity by @u70b3 in #8112

feat(python): expose more scanner options on LanceFragment by @keunhong in #8429

perf(java): reduce fragment statistics allocations by @everySympathy in #8452

feat(python): expose ObjectStoreProvider registration in pylance by @farazshaikh in #8522

feat: add mixed data file version capability by @Xuanwo in #8580

feat: validate mixed V2 data file versions by @Xuanwo in #8581

feat: support per-operation V2 write targets by @Xuanwo in #8582

feat: propagate exact V2 versions across dataset operations by @Xuanwo in #8583

feat: target exact V2 versions in compaction by @Xuanwo in #8584

feat(cleanup): support cleaning specific dataset versions by @hfutatzhanghb in #8617

feat: resolve stable file format to 2.2 by @Xuanwo in #8657

feat: add column slice stitching by @Xuanwo in #8660

feat: support BinaryView in packed blob writer via iterator-based extraction by @lichuang in #8700

feat(python): restore index retraining by @ddupg in #8786

feat(encoding): add CANDIDATE_BATCH_SIZES and plan_decoded_bytes trait defaults by @westonpace in #8791

feat(java): expose createIndex progress callbacks by @hfutatzhanghb in #8823

feat(java): align inverted index options by @ddupg in #8829

feat(mem_wal): let a writer read its own indexed prefix by @hamersaw in #8835

fix(mem_wal): honor distance_range in the LSM vector planner by @hamersaw in #8845

feat(python): return matched row offsets from fragment update_columns by @zhangyue19921010 in #8891

feat(java): expose merge insert write mode by @zhangyue19921010 in #8918

feat: budget blob materialization readahead by @jackye1995 in #8919

feat: add caller-managed data file parts by @Xuanwo in #8923

feat(java): complete transaction operation mappings by @everySympathy in #8925

feat(index): default ivf_rq to 5-bit quantization by @BubbleCal in #8936

feat(java): expose referenced file metadata by @everySympathy in #8962

feat(mem_wal): record what a flushed SSTable holds by @xuanyu-z in #8981

feat(index): proportional hierarchical k-means for IVF training by @BubbleCal in #9050

feat(index): split IVF partitions to target size and join undersized ones in one optimize pass by @BubbleCal in #9051

feat: expose public accessor for the fragment reuse index by @brendanclement in #9112

feat(dataset): expose lance dataset deep clone to python by @dentiny in #9181

feat(search): select metric-aware initial IVF probe budgets by @BubbleCal in #9195

Bug Fixes 🐛

fix: make float filters treat -0.0 and 0.0 as the same value by @LuciferYang in #6236

fix(arrow): avoid duplicate field when merging identical List columns by @LuciferYang in #7494

fix: don't fail dataset open on an undecodable inline transaction by @jackye1995 in #7740

fix(core): reject non-finite values in AimdConfig::validate by @LuciferYang in #7942

fix(index): replace scalar cache entries on store rotation by @u70b3 in #7962

fix: validate primary key nullability on every path that can change it by @xuanyu-z in #8101

fix(index): filter stale vector segment rows before top-k by @app/lance-gatefixer in #8351

fix: cast nested columns during schema evolution by @app/lance-gatefixer in #8379

fix(encoding): read wrapped miniblock level counts by @app/lance-gatefixer in #8410

fix(index): keep an index this build cannot read out of the erase path by @wombatu-kun in #8427

fix(index): keep a commit alive when an index cannot be opened by @wombatu-kun in #8441

fix(python): declare LanceSchema.field and field_case_insensitive in the type stub by @jonasdedden in #8486

fix(python): correct the misspelled materialize_deletions_threshold option by @jonasdedden in #8487

fix(python): declare CompactionOptions with total=False by @jonasdedden in #8488

fix(python): accept None for max_rows_per_group by @jonasdedden in #8489

fix(python): let create_index_uncommitted accept an IndexConfig by @jonasdedden in #8490

fix(python): declare the index training helpers as returning FixedSizeListArray by @jonasdedden in #8491

fix(python): annotate LanceFragment.data_files and deletion_file by @jonasdedden in #8492

fix(python): annotate count_rows, versions and optimize_indices by @jonasdedden in #8493

fix(python): annotate BatchUDF and the batch_udf decorator by @jonasdedden in #8494

fix: reject lossy null structs in v2.0 by @app/lance-gatefixer in #8507

fix: support Time32 integer casts in filters by @app/lance-gatefixer in #8508

fix(compaction): reject unsafe binary column layouts by @app/lance-gatefixer in #8510

fix: keep LabelList index current after reordered merge insert by @app/lance-gatefixer in #8511

fix(index): remap ZoneMap search results by @app/lance-gatefixer in #8512

fix(compaction): avoid stranded remainder fragments by @app/lance-gatefixer in #8513

fix(index): skip unrecognized index types by @app/lance-gatefixer in #8529

fix(linalg): pick the sum-of-squares accumulator per element type by @LuciferYang in #8533

fix: defer blank rows when a fully deleted batch has no row to copy by @LuciferYang in #8612

fix(sql): include system columns when requested for SQL-based query by @dentiny in #8626

fix(linalg): enforce the hamming_batch_u64 length contract in release by @LuciferYang in #8639

fix(linalg): report AVX-512 test coverage by @app/lance-gatefixer in #8664

fix: refuse to drop a path that is not a Lance dataset by @xiaguanglei in #8665

fix: preserve typed DataFusion errors through Shared and ReplayExec by @app/copilot-swe-agent in #8677

fix(linalg): correct multivector distance aggregation by @ddupg in #8710

fix(io): prevent io_uring requests from hanging by @Xuanwo in #8725

fix(io): reject unordered scheduler ranges by @app/lance-gatefixer in #8728

fix(linalg): enforce cosine u8 length contract by @app/lance-gatefixer in #8737

fix(fts): preserve global scoring for no-impact indexes by @BubbleCal in #8748

fix(fts): canonicalize fuzzy expansion across segments by @BubbleCal in #8758

fix(python): spell misnamed without a hyphen so the typos gate passes by @jackylee-ch in #8766

fix(index): stream bulk index file copies by @jackye1995 in #8770

fix(merge_insert): make analyze_plan follow execute's source routing by @LuciferYang in #8772

fix(datagen): align random primitive buffers by @app/lance-gatefixer in #8776

fix(fts): use substring-safe ngram defaults by @app/lance-gatefixer in #8779

fix(python): don't seed IVF centroids with NaN vectors by @jonasdedden in #8782

fix(ci): stabilize test_scan_slice against page-cache state by @RupertMaiti2005 in #8815

fix(encoding): normalize out-of-range null dictionary keys by @app/lance-gatefixer in #8827

fix(substrait): decode deprecated timestamp literals as microseconds by @jonasdedden in #8832

fix(hnsw): make a persisted HNSW cover its rows and reject invalid ones by @xuanyu-z in #8834

fix(mem_wal): seal the memtable on max_memtable_rows by @hamersaw in #8837

fix(fs): fix dataset files permission on local filesystem by @dentiny in #8839

fix(encoding): handle empty all-null dictionaries by @app/lance-gatefixer in #8841

fix(index): widen the RaBitQ FastScan accumulator above 1024 dims by @wombatu-kun in #8842

fix(index): declare FM index row address results by @app/lance-gatefixer in #8855

fix(linalg): widen u8 distance accumulators by @app/lance-gatefixer in #8888

fix: resolve clone sources with source commit handler by @Xuanwo in #8902

fix(index): key metadata cache on manifest etag by @ChilePiquin in #8904

fix(linalg): distinguish arrow batch type errors by @app/lance-gatefixer in #8911

fix(fts): align v1 merge reads with writer batches by @BubbleCal in #8922

fix: preserve complete blob v2 logical schemas by @Xuanwo in #8929

fix(io): carry batch_size_bytes through distributed FilteredReadOptions by @lichuang in #8933

fix(encoding): accept 1-bit booleans in the single-row full-zip fallback by @zhangyue19921010 in #8934

fix: avoid panic when rebasing a commit whose read version was cleaned up by @qiuyuhang in #8935

fix(io): reject non-lowercase GooseFS storage_options keys by @XuQianJin-Stars in #8940

fix(io): parse GooseFS chunk/block size suffixes like 4MB by @XuQianJin-Stars in #8943

fix: remove namespace-managed clone handler plumbing by @Xuanwo in #8964

fix(sql): classify malformed SQL queries as invalid input by @wjones127 in #8974

fix: let analyze_plan run under a caller-provided TaskContext by @LuQQiu in #8980

fix(index): prevent maintenance from undoing index drops by @app/lance-gatefixer in #8984

fix(java): remove duplicate IntoJava impl for &BasePath by @LuciferYang in #8990

fix: count only visible nulls in the 2.1+ writer by @jackye1995 in #9020

fix(mem_wal): run grace eviction on its own task, not the flush handler's ticker by @xuanyu-z in #9022

fix(cache): bypass Moka single-flight for zero-capacity caches by @jiaqizho in #9038

fix: copy sliced arrays from logical offsets by @jackye1995 in #9048

fix(index): keep every HNSW node reachable after a parallel build by @BubbleCal in #9053

fix(fts): preserve prewarmed state across request store wrappers by @BubbleCal in #9075

fix: stamp overlay row lineage at commit by @app/lance-gatefixer in #9084

fix: honor zero scanner limits by @app/lance-gatefixer in #9111

fix(encoding): prevent bits_per_value=0 in FullZipLayout for FSL with AllNull child by @westonpace in #9130

fix: load external fragment reuse index details from the correct base path by @LuQQiu in #9141

fix: skip size probes for bounded blob reads by @Xuanwo in #9146

fix(python): preserve filter pushdown with row IDs by @app/lance-gatefixer in #9154

fix(dataset): handle lance dataset blobs on deep clone by @dentiny in #9185

fix(scanner): respect negated flag when converting InList filters to take operations by
Note truncated.

11.0.030 Aug 2026

refactor!: compose exact current-format readers by @Xuanwo in #8024

What's Changed

Breaking Changes 🛠

refactor!: compose exact current-format readers by @Xuanwo in #8024
refactor: centralize exact dataset write policy by @Xuanwo in #8025
refactor: move file-version selectors to lance-file by @Xuanwo in #8026
refactor: centralize dataset version policies by @Xuanwo in #8027
refactor: make index file versions exact by @Xuanwo in #8028
feat(mem_wal): return the sealed generation from force_seal_active by @hamersaw in #8051
feat(mem_wal): expose whether an index type can be maintained by @hamersaw in #8095
fix(cache): deduplicate deep size accounting by @app/lance-gatekeeper in #8159
fix(encoding): return errors for malformed variable-width offsets by @app/lance-gatekeeper in #8172
fix(index): align HNSW construction with reference by @app/lance-gatekeeper in #8188
fix(core)!: stop reusing fragment ids across an overwrite by @wjones127 in #8206
feat(compaction)!: support max_source_rows and max_source_bytes limits by @zhangyue19921010 in #8235
fix(dataset)!: conflict a NOT NULL tightening with concurrent writes by @wkalt in #8347
feat(mem_wal): validate maintained indexes against the writer's rules by @hamersaw in #8360
fix(datafusion): use caller session for filter planning by @app/lance-gatefixer in #8407
feat(format): add IndexMetadata.covering_fields for covering indexes by @vivek-bharathan in #8535

Critical Fixes ‼️

fix(encoding): bounds-check the length prefix in VariableFullZipDecoder::unzip by @professor-moody in #8138
fix(encoding): reject out-of-bounds variable-width offsets on decode by @Xuanwo in #8144

New Features 🎉

feat(java): expose ArrowArrayStream export on LanceScanner by @sezruby in #7259
feat(scanner): add external row-address mask prefilter by @JulianYG in #7288
feat(cache): support register and switch pluggable cache backend by @yanghua in #7683
feat: per value support for fixed-length packed structs by @morales-t-netflix in #7714
feat(index): add FTS document granularity by @Xuanwo in #7788
feat(python): expose file size from LanceFileWriter by @FANNG1 in #7876
perf(dataset): spill large inline transaction out of manifest by @zhangyue19921010 in #7881
feat(index): add zone map support for all data types by @westonpace in #8017
feat: support shared session for fragment API by @zhangyue19921010 in #8034
feat(index): stop HNSW greedy descent before level 0 by @u70b3 in #8035
feat(java): add OpenTelemetry metrics bridge by @yanghua in #8064
feat(java): expose fragment statics through java blocking dataset by @zhangyue19921010 in #8072
feat(sql): add blob handling to query builder by @dentiny in #8087
feat(java): expose index build progress callbacks by @hfutatzhanghb in #8090
feat(fts): add composable scorer core for multimatch by @BubbleCal in #8092
feat(fts): compose boolean and phrase scorers by @BubbleCal in #8093
feat(fts): compose boosted compound scorers by @BubbleCal in #8094
feat(object-store): add explicit IRSA and ECS credential storage options by @westonpace in #8103
feat(python): expose max_source_fragments in compact_files by @zhangyue19921010 in #8116
feat(fts): export CompoundQueryExec and support an injected base scorer by @LuQQiu in #8131
feat(goosefs): implement safe commit with ConditionalPutCommitHandler by @XuQianJin-Stars in #8134
feat(mem_wal): expose frozen memtable and backpressure stats by @hamersaw in #8241
feat(rowids): validate stable row ids in Dataset::validate by @wjones127 in #8258
feat(mem-wal): record index catch-up positions and withdraw them on index change by @xuanyu-z in #8263
feat(java): add scanner performance and memory options by @everySympathy in #8288
feat(dataset): per-fragment column writes that survive compaction by @wkalt in #8313
feat(python): expose BlobFile.read_ranges by @keunhong in #8319
feat(mem_wal): support delete against non-nullable base columns by @hamersaw in #8352
feat(java): expose index segment sizes by @majin1102 in #8355
feat(python): add RowIdSequence for constructing row id metadata by @wjones127 in #8356
feat(fts): show tokenized queries in analyze plan by @BubbleCal in #8414
feat(python): expose FTS query tokenization by @BubbleCal in #8415
feat(merge_insert): add write_mode to select how merged rows are written by @LuciferYang in #8423
feat(python): expose LanceFragment.validate by @keunhong in #8428
feat(java): support cache backend register and switch by @yanghua in #8446
feat(python): expose updated_fragment_offsets in the Update operation binding by @pengw0048 in #8447
feat(java): expose manifest location metadata by @everySympathy in #8450
feat(java): expose manifest writer version by @everySympathy in #8451
feat(java): add efficient dataset version count by @everySympathy in #8453
feat(fts): expose pure SHOULD MAXSCORE metrics by @BubbleCal in #8475
feat(mem-wal): derive index catch-up from the version a commit read by @xuanyu-z in #8481
feat: add Dataset::migrate_to_stable_row_ids migration method by @westonpace in #8521
feat(dataset): add lightweight dataset version references by @dentiny in #8523
feat(deps): prune unused dependencies by @dentiny in #8524
feat(python): expose frozen memtable metrics in memtable_stats by @hamersaw in #8526
feat(compaction): support excluding fragments from planning by @everySympathy in #8532
feat: report the row ids deleted between two versions by @wkalt in #8589
feat(mem_wal): observe flush latency through the metrics facade by @hamersaw in #8625
feat(sql): expose batch size to sql builder by @dentiny in #8648
feat(fts): add cross-column scorer composition by @BubbleCal in #8685

Bug Fixes 🐛

fix(java): expose updatedFragmentOffsets on Update operation for RewriteColumns by @jerryjch in #6748
fix(index): drop stale per-segment rows in vector search after in-place column update by @wombatu-kun in #7371
fix: fall back to AVX2 f16 kernels on AVX-512 CPUs by @paramt in #7428
fix: validate merge schema field bindings by @wkalt in #7703
fix: recognize own commit on conflict and avoid deleting committed artifacts by @jackye1995 in #7722
fix(bloomfilter): bound read/write to prevent BinaryArray i32 offset overflow by @zhangyue19921010 in #7753
fix(ngram): recheck contains() with sub-trigram needle instead of dropping rows by @Ar-maan05 in #7845
fix(index): don't report a ZoneMap value range that an overlay may have escaped by @wjones127 in #7924
fix(datagen): preserve timestamp time zones by @xixiGoodLuck in #7946
fix(index): align scalar index field type support by @ddupg in #7966
fix(fts): re-evaluate overlay-stale rows without segment scans by @Xuanwo in #7975
fix(lance-io): request vended credentials explicitly in namespace provider by @rpgreen in #7978
fix(io): publish object store metrics for local reads and writes by @wjones127 in #7994
fix(jni): align CommitBuilder.storageFormat() parsing with LanceFileVersion::from_str by @ivscheianu in #8063
fix(core): key row id sequence cache on fragment contents by @wjones127 in #8078
fix(index): keep address-domain indices off replaced fragments by @wjones127 in #8084
fix(index): use zone map index for IS NOT NULL predicates by @westonpace in #8088
fix(datagen): make default temporal ranges deterministic by @westonpace in #8121
fix(index): compare Hamming IVF centroids during merge by @jackye1995 in #8132
fix: preserve row address filters with stable row ids by @app/lance-gatekeeper in #8145
fix(index): resolve nested vector index field paths by @justinrmiller in #8148
fix: preserve schema metadata precedence on merge by @app/lance-gatekeeper in #8160
fix(python): preserve commit handler exceptions by @app/lance-gatekeeper in #8162
fix: type scoring columns in projection expressions by @app/lance-gatekeeper in #8163
fix(io): return an error for short reads by @app/lance-gatekeeper in #8164
fix(index): preserve row addresses in JSON index training by @app/lance-gatekeeper in #8165
fix(index): replace same-name index segments across types by @app/lance-gatekeeper in #8166
fix(index): count HNSW entry point at every level by @BubbleCal in #8167
fix: reject missing merge insert update addresses by @app/lance-gatekeeper in #8168
fix: bound commit conflict retry backoff by @app/lance-gatekeeper in #8169
fix: dedupe unmatched merge insert source rows by @app/lance-gatekeeper in #8171
fix(encoding): reject invalid miniblock row counts by @app/lance-gatekeeper in #8173
fix(storage): preserve multipart part identity across retries by @app/lance-gatekeeper in #8174
fix(merge_insert): deduplicate indexed candidates by @app/lance-gatekeeper in #8176
fix(ci): treat missing merge_insert base tag as not generated by @Xuanwo in #8177
fix: preserve large unsigned SQL filter literals by @app/lance-gatekeeper in #8178
fix: avoid Greek stemmer UTF-8 panic by @app/lance-gatekeeper in #8183
fix(java): type scanner operation failures by @app/lance-gatekeeper in #8184
fix(fts): align list post-filter with indexed semantics by @Ecthlion in #8185
fix(index): push down deeply nested scalar filters by @app/lance-gatekeeper in #8186
fix: respect external base layout in tracked files by @app/lance-gatekeeper in #8187
fix: align byte-sized batches across data files by @app/lance-gatekeeper in #8189
fix(ci): enlarge mem pool only around merge_insert index build by @Xuanwo in #8191
fix(index): initialize indices on nested fields by @app/lance-gatekeeper in #8192
fix: deduplicate MemWAL IN point keys by @app/lance-gatekeeper in #8197
fix: return error for divergent MemWAL schemas by @app/lance-gatekeeper in #8198
fix: prevent update false negatives for address-domain indices by @app/lance-gatekeeper in #8203
fix: retain empty vector index definitions in retain_relevant_indices by @wkalt in #8208
fix(fts): cover all segments for fragment scans by @ddupg in #8211
fix: tolerate concurrent manifest cleanup by @app/lance-gatefixer in #8213
fix(index): accept "RQ" when parsing QuantizationType by @jackylee-ch in #8214
fix: avoid v2.2 panic for full dictionaries by @app/lance-gatefixer in #8219
fix(encoding): reject mismatched dictionary index widths by @app/lance-gatefixer in #8220
fix(index): remap bloom filter search results by @app/lance-gatefixer in #8223
fix: validate GBO descriptors before metadata I/O by @app/lance-gatefixer in #8227
fix(docs): stop make help from running the website build by @jackylee-ch in #8230
fix: restore main CI after exact-format reader refactor by @Xuanwo in #8231
fix(namespace): self-heal DirectoryNamespace read manifest cache by @rpgreen in #8233
fix(table): reject corrupt row ID metadata by @app/lance-gatefixer in #8254
fix: compute row offsets for unsorted addresses by @app/lance-gatefixer in #8255
fix(lance): propagate deletion vector read errors by @app/lance-gatefixer in #8256
fix: include highest fragment in legacy prefilter masks by @app/lance-gatefixer in #8257
fix(rowids): validate serialized row ID metadata by @app/lance-gatefixer in #8259
fix: remove duplicate LanceFileVersion import in lance-file tests by

Note truncated.

Open source →

10.0.07 Aug 2026

What's Changed

Breaking Changes 🛠

perf(compaction)!: skip building row-address maps when index remapping is not needed by @everySympathy in https://github.com/lance-format/lance/pull/7778
perf(cache)!: use fixed-size cache keys by @Ali2Arslan in https://github.com/lance-format/lance/pull/7878
fix(blob)!: preserve null selections across blob APIs by @Xuanwo in https://github.com/lance-format/lance/pull/7903

Critical Fixes ‼️

fix(exec): preserve order when coalescing a sorted multi-partition plan by @wjones127 in https://github.com/lance-format/lance/pull/7868
fix(encoding): bounds-check the length prefix in VariableFullZipDecoder::unzip by @professor-moody in https://github.com/lance-format/lance/pull/8138
fix(encoding): reject out-of-bounds variable-width offsets on decode by @Xuanwo in https://github.com/lance-format/lance/pull/8144

Both fixes were backported to release/v10.0 in #8146. The first removes an unchecked length-prefix read from release builds; the second rejects corrupt offsets before constructing Arrow arrays.

New Features 🎉

feat(index): support segmented ngram index by @zhangyue19921010 in https://github.com/lance-format/lance/pull/7244
feat: accept TableProvider write inputs for merge_insert and insert by @wjones127 in https://github.com/lance-format/lance/pull/7368
feat(python): add Pydantic model instance auto-conversion and from_pydantic_model() classmethod by @vinaysurtani in https://github.com/lance-format/lance/pull/7383
feat(index): write zone map seeds into data file footers during append by @westonpace in https://github.com/lance-format/lance/pull/7427
feat(python): expose DataOverlay commit operation by @wjones127 in https://github.com/lance-format/lance/pull/7540
feat: support cross-store (cross-account) deep_clone by @sezruby in https://github.com/lance-format/lance/pull/7545
feat(index): mask data overlay files in scalar and vector index queries by @wjones127 in https://github.com/lance-format/lance/pull/7549
feat: support structured_query in the directory namespace query_table by @hamersaw in https://github.com/lance-format/lance/pull/7592
feat: support merge_insert when source omits blob columns by @nyl3532016 in https://github.com/lance-format/lance/pull/7615
feat: add code analyzer for FTS by @Xuanwo in https://github.com/lance-format/lance/pull/7681
feat(compaction): compact fragments over an overlay-count limit by @wjones127 in https://github.com/lance-format/lance/pull/7772
feat(fts): record scorer_build_ms for the exec-local BM25 scorer fallback by @LuQQiu in https://github.com/lance-format/lance/pull/7807
feat(fts): read inverted index params without opening the segment by @LuQQiu in https://github.com/lance-format/lance/pull/7816
feat: expose cached file metadata APIs on FileFragment by @jiaqizho in https://github.com/lance-format/lance/pull/7820
feat(index): per-query index cache hit/miss stats by @yanghua in https://github.com/lance-format/lance/pull/7862
feat: add batch blob range reads by @Xuanwo in https://github.com/lance-format/lance/pull/7864
feat: introduce exact file format identity and persisted codecs by @Xuanwo in https://github.com/lance-format/lance/pull/7879
feat(python): expose LABEL_LIST segment index build by @ddupg in https://github.com/lance-format/lance/pull/7884
feat(java): expose session metadata cache stats to java api by @zhangyue19921010 in https://github.com/lance-format/lance/pull/7885
feat(format): read and write sparse structural pages by @Xuanwo in https://github.com/lance-format/lance/pull/7889
feat(mem_wal): drive the WAL append on a flush-interval ticker by @hamersaw in https://github.com/lance-format/lance/pull/7894
feat(index): support segmented bloom filters by @jackye1995 in https://github.com/lance-format/lance/pull/7925
feat(index): use ACORN-1 traversal for prefiltered HNSW search by @leohoare in https://github.com/lance-format/lance/pull/7927
feat: support segmented RTree indexes by @jackye1995 in https://github.com/lance-format/lance/pull/7932
feat(namespace): idempotent create_table_version with strict CAS and dotfile-safe reserved marker by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/7940
feat(fts): add native exact segment selection by @Xuanwo in https://github.com/lance-format/lance/pull/7976

Bug Fixes 🐛

fix(index): normalize same-column scalar index predicates by @zouhuajian in https://github.com/lance-format/lance/pull/6782
fix: avoid panic due to empty IO buffers on tokio runtime shutdown by @ilya-zlobintsev in https://github.com/lance-format/lance/pull/7478
fix: don't quote column names in list_indices by @dantasse in https://github.com/lance-format/lance/pull/7503
fix(commit): verify finalized external manifests by @BubbleCal in https://github.com/lance-format/lance/pull/7578
fix: skip empty string docs in FTS stats by @Ecthlion in https://github.com/lance-format/lance/pull/7699
fix: keep ids and addresses aligned in filter_deleted_ids under stabl… by @wkalt in https://github.com/lance-format/lance/pull/7704
fix(mem-wal): propagate final flush failures from ShardWriter::close by @u70b3 in https://github.com/lance-format/lance/pull/7769
fix: bracket match query display by @jackye1995 in https://github.com/lance-format/lance/pull/7796
fix(sql): reject invalid LIKE/ILIKE escape strings by @wjones127 in https://github.com/lance-format/lance/pull/7810
fix(table): route tos:// scheme to ConditionalPutCommitHandler by @zhangyue19921010 in https://github.com/lance-format/lance/pull/7824
fix(python): allow blob writer cleanup across threads by @justinrmiller in https://github.com/lance-format/lance/pull/7827
fix(index): reuse cached FTS document lengths by @Xuanwo in https://github.com/lance-format/lance/pull/7830
fix(index): sort JSON-path values once after extraction, not the raw column by @wjones127 in https://github.com/lance-format/lance/pull/7835
fix: make FTS metadata loading retry-safe by @Xuanwo in https://github.com/lance-format/lance/pull/7838
fix: reenable flaky tests on test_torch_kmeans.py by @cswpy in https://github.com/lance-format/lance/pull/7849
fix(fts): restore format version environment fallback by @Xuanwo in https://github.com/lance-format/lance/pull/7855
fix(core): validate CPU/IO env vars with a diagnosable error by @LuciferYang in https://github.com/lance-format/lance/pull/7856
fix: dispatch concurrent blob range reads by @Xuanwo in https://github.com/lance-format/lance/pull/7861
fix(fts): use v3 as the capability gate by @Xuanwo in https://github.com/lance-format/lance/pull/7866
fix(index): return error instead of panicking when hierarchical k-means can't reach target_k by @wjones127 in https://github.com/lance-format/lance/pull/7869
fix: trim fragment-reuse index for non-covering indexes by @xuanyu-z in https://github.com/lance-format/lance/pull/7870
fix(mem_wal): harden shard open and fail reads on a poisoned writer by @hamersaw in https://github.com/lance-format/lance/pull/7872
fix(fts): preserve parameter compatibility by @Xuanwo in https://github.com/lance-format/lance/pull/7874
fix(core): bound SlotBackoff retry sleeps and fix slot overflow by @LuciferYang in https://github.com/lance-format/lance/pull/7883
fix(mem_wal): read-your-writes via split index-apply and dual visibility cursors by @hamersaw in https://github.com/lance-format/lance/pull/7888
fix(mem-wal): propagate task shutdown failures by @u70b3 in https://github.com/lance-format/lance/pull/7915
fix(mem-wal): refill scans after shadow filtering by @u70b3 in https://github.com/lance-format/lance/pull/7917
fix(index): correct offset→row-id translation for overlay masking under deletions by @wjones127 in https://github.com/lance-format/lance/pull/7918
fix(index): keep overlays masked when updating a non-indexed column by @wjones127 in https://github.com/lance-format/lance/pull/7926
fix(namespace): propagate dir-namespace storage errors instead of mapping to TableNotFound by @brendanclement in https://github.com/lance-format/lance/pull/7931
fix(index): rebind vector readers to current object store by @u70b3 in https://github.com/lance-format/lance/pull/7944
fix(blob): unwrap Option in merge_insert blob tests after take_blobs null API by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/7963
fix: persist finalized external manifest metadata by @Xuanwo in https://github.com/lance-format/lance/pull/7964
fix(blob): preserve null and empty values during compaction by @Xuanwo in https://github.com/lance-format/lance/pull/7965
fix(ci): unblock cargo publish and dry-run packaging in CI by @wjones127 in https://github.com/lance-format/lance/pull/7989
fix: preserve blob selection cardinality by @Xuanwo in https://github.com/lance-format/lance/pull/8003
fix(index): prune stale index coverage during rebase by @ddupg in https://github.com/lance-format/lance/pull/8011
fix(index): make vector append segment-set-native by @Xuanwo in https://github.com/lance-format/lance/pull/8047
fix(arrow): preserve all-null struct validity during merge by @mmatczuk in https://github.com/lance-format/lance/pull/8049
fix(io): stabilize wrapper identity in object store cache by @Xuanwo in https://github.com/lance-format/lance/pull/8068
fix(blob): preserve null and empty values across reads and compaction by @Xuanwo in https://github.com/lance-format/lance/pull/8070
fix(fts): propagate nested multimatch limits by @BubbleCal in https://github.com/lance-format/lance/pull/8073

Documentation 📚

docs: implement the Lance Docs design as the mkdocs site theme by @Xuanwo in https://github.com/lance-format/lance/pull/7821
docs: make the docs site theme usable on mobile and narrow screens by @Xuanwo in https://github.com/lance-format/lance/pull/7853
docs: fix nested agent guide loading and trim root AGENTS.md by @Xuanwo in https://github.com/lance-format/lance/pull/7992
docs: include lance-context in the full website build by @beinan in https://github.com/lance-format/lance/pull/8014
docs: define legacy compatibility boundaries by @Xuanwo in https://github.com/lance-format/lance/pull/8039

Performance Improvements 🚀

perf(encoding): cache complex-all-null levels in RLE run form and drain run-by-run by @Ali2Arslan in https://github.com/lance-format/lance/pull/7564
perf(index): norm-addend cache and slim top-k heap for FTS scoring by @BubbleCal in https://github.com/lance-format/lance/pull/7629
perf(encoding): compact mini-block chunk index for cached page state by @Ali2Arslan in https://github.com/lance-format/lance/pull/7651
perf(index): list index segment files concurrently when committing segments by @xloya in https://github.com/lance-format/lance/pull/7657
perf: plan take operations through FilteredReadExec's range-read path by @LuQQiu in https://github.com/lance-format/lance/pull/7672
perf(encoding): avoid inline bitpacking input copy by @u70b3 in https://github.com/lance-format/lance/pull/7696
perf(encoding): auto-select sparse structural pages by @Xuanwo in https://github.com/lance-format/lance/pull/7756
perf(filtered-read): reuse plan-time fragment metadata for the read stream by @LuQQiu in https://github.com/lance-format/lance/pull/7792
perf(dataset): read transactions by version without populating session caches by @xuanyu-z in https://github.com/lance-format/lance/pull/7817
perf(fts): resolve deferred row_ids via the cached whole ROW_ID column by @LuQQiu in https://github.com/lance-format/lance/pull/7897
perf(fts): search partitions in pipelined chunks by @LuQQiu in https://github.com/lance-format/lance/pull/7950
perf(cache): opt-in quick_cache backend for contention-free reads by @LuQQiu in https://github.com/lance-format/lance/pull/7953
perf: improve performance of CandidateBin.split_for_size by @fangbo in https://github.com/lance-format/lance/pull/8000
perf(cache): use the quick_cache backend for the session metadata cache by @LuQQiu in https://github.com/lance-format/lance/pull/8013
perf: avoid cloning every fragment descriptor in take and FTS planning by @LuQQiu in https://github.com/lance-format/lance/pull/8102
perf(fts): pre-load doc lengths in parallel on cold deferred search path by @westonpace in https://github.com/lance-format/lance/pull/8119, backported in https://github.com/lance-format/lance/pull/8146

Other Changes

refactor(fts): separate document identity from row addresses by @Xuanwo in https://github.com/lance-format/lance/pull/7863
refactor(mem-wal): rename flushed MemTable/generation to SSTable by @jackye1995 in https://github.com/lance-format/lance/pull/7943
refactor: rename MemWAL compaction progress by @jackye1995 in https://github.com/lance-format/lance/pull/7957
refactor(index): remove dead segment commit branch by @ddupg in https://github.com/lance-format/lance/pull/7971
refactor: make v1 the canonical legacy file owner by @Xuanwo in https://github.com/lance-format/lance/pull/8020
refactor: make encoding mechanisms version-free by @Xuanwo in https://github.com/lance-format/lance/pull/8021
refactor: add exact current-format writers by @Xuanwo in https://github.com/lance-format/lance/pull/8022
refactor(encoding): pass mini-block compression context by @Xuanwo in https://github.com/lance-format/lance/pull/8038

Full Changelog: https://github.com/lance-format/lance/compare/release-root/10.0.0-beta.N...v10.0.0

Open source →

9.0.024 Jul 2026Deprecated

fix(ci): replace deprecated array.shape assignment for NumPy 2.5 by @zhangyue19921010 in https://github.com/lance-format/lance/pull/7384

What's Changed

Breaking Changes 🛠

refactor!: rename FMIndexIndexDetails to FMIndexDetails by @westonpace in https://github.com/lance-format/lance/pull/7397
feat(fts)!: add configurable posting block size by @BubbleCal in https://github.com/lance-format/lance/pull/7466
feat(fts)!: make v2 the default index format by @BubbleCal in https://github.com/lance-format/lance/pull/7512

New Features 🎉

feat(namespace-dir): add alter table column operations (add/alter/drop) by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/6273
feat: compile-time feature-gated Rust backtrace for JNI error diagnostics by @summaryzb in https://github.com/lance-format/lance/pull/6365
feat(lance-linalg): runtime SIMD dispatch for pre-Haswell x86_64 from-source builds by @tobocop2 in https://github.com/lance-format/lance/pull/6630
feat(namespace-dir): implement update_table and delete_from_table by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/6923
feat(namespace-dir): implement alter_transaction for DirectoryNamespace by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/6974
feat(mem_wal): support prefiltered LSM vector and FTS search by @touch-of-grey in https://github.com/lance-format/lance/pull/7138
feat(index): support distributed LabelList scalar index builds by @jackye1995 in https://github.com/lance-format/lance/pull/7223
feat(compaction): introduce RowAddrRemap structure to avoid remap OOM caused by HashMap by @zhangyue19921010 in https://github.com/lance-format/lance/pull/7237
feat: support cleanup explain for python and java by @yanghua in https://github.com/lance-format/lance/pull/7248
feat(schema_evolution): allow Dict <-> value-type casts in alter_columns by @valkum in https://github.com/lance-format/lance/pull/7289
feat: add SpillStore trait with local-disk implementation by @wjones127 in https://github.com/lance-format/lance/pull/7311
feat(blob-v2): configure packed blob file max size with field metadata by @jo-migo in https://github.com/lance-format/lance/pull/7322
feat(index): support Utf8View prefixes in SargableQueryParser by @wombatu-kun in https://github.com/lance-format/lance/pull/7351
feat: support COUNT(*) pushdown on stable row id datasets by @wkalt in https://github.com/lance-format/lance/pull/7360
feat(mem-wal): add ShardWriter::abort + Sealed manifest fence for drop-table by @hamersaw in https://github.com/lance-format/lance/pull/7361
feat(mem-wal): add ShardWriter::put_no_wait by @hamersaw in https://github.com/lance-format/lance/pull/7362
feat: add RLE v2 run length widths by @Xuanwo in https://github.com/lance-format/lance/pull/7376
feat: support hamming clustering by @brendanclement in https://github.com/lance-format/lance/pull/7379
feat(file): v2 writer/reader support columns of unequal length by @wjones127 in https://github.com/lance-format/lance/pull/7406
feat(mem_wal): row-level delete via tombstone rows by @hamersaw in https://github.com/lance-format/lance/pull/7417
feat(python): add list_with_delimiter, delete_file, and read_range to LanceFileSession by @jmhsieh in https://github.com/lance-format/lance/pull/7426
feat(python): expose fragment-reuse remap and delete-by-offset by @wkalt in https://github.com/lance-format/lance/pull/7438
feat(index): read a column's min/max from ZoneMap without a scan by @Ali2Arslan in https://github.com/lance-format/lance/pull/7463
feat: add ICU split tokenizer variant by @Xuanwo in https://github.com/lance-format/lance/pull/7474
feat(mem_wal): tombstone-preserving point lookup by @hamersaw in https://github.com/lance-format/lance/pull/7482
feat(mem_wal): non-blocking ShardWriter::delete_no_wait by @hamersaw in https://github.com/lance-format/lance/pull/7483
feat(bitpacking): add owned bitpacking codecs by @BubbleCal in https://github.com/lance-format/lance/pull/7496
feat(io): publish object store metrics via the metrics crate by @wjones127 in https://github.com/lance-format/lance/pull/7533
feat: data overlay files — model, feature flag, and write/commit path by @wjones127 in https://github.com/lance-format/lance/pull/7535
feat: resolve data overlay files on the take (and scan) read path by @wjones127 in https://github.com/lance-format/lance/pull/7536
feat(python): expose Lance metrics via OpenTelemetry by @wjones127 in https://github.com/lance-format/lance/pull/7537
feat(python): expose DataOverlay commit operation by @wjones127 in https://github.com/lance-format/lance/pull/7540
feat: add blob write preparation API by @Xuanwo in https://github.com/lance-format/lance/pull/7558
feat: support update with blob encoding column by @nyl3532016 in https://github.com/lance-format/lance/pull/7579
feat(fts): impact skip data for posting lists by @BubbleCal in https://github.com/lance-format/lance/pull/7602
feat: support per-base storage options with base_<id>.<key> prefix by @jackye1995 in https://github.com/lance-format/lance/pull/7608
feat: support multi-base tables in merge insert with target base routing by @jackye1995 in https://github.com/lance-format/lance/pull/7610
feat(python): expose MemWAL shard delete by @u70b3 in https://github.com/lance-format/lance/pull/7649
feat: compress RLE child buffers by @Xuanwo in https://github.com/lance-format/lance/pull/7663
feat: support list blob scans by @Xuanwo in https://github.com/lance-format/lance/pull/7664
feat: support segment selection in pylance prewarm by @jackye1995 in https://github.com/lance-format/lance/pull/7677
feat: add code analyzer for FTS by @Xuanwo in https://github.com/lance-format/lance/pull/7681
feat: support nested field FTS by @Xuanwo in https://github.com/lance-format/lance/pull/7686
feat(java): expose MemWAL shard delete by @hamersaw in https://github.com/lance-format/lance/pull/7688
feat(java): forward table properties to declareTable on namespace create by @puchengy in https://github.com/lance-format/lance/pull/7711
feat(mem_wal): thread store_params + session through WAL write/read paths by @hamersaw in https://github.com/lance-format/lance/pull/7735
feat(python): add bulk packed blob writer API by @justinrmiller in https://github.com/lance-format/lance/pull/7743
feat(credentials): vend AWS credentials via AssumeRoleWithWebIdentity to avoid role chaining by @rpgreen in https://github.com/lance-format/lance/pull/7757
feat: support multi-segment indices in hamming clustering by @jackye1995 in https://github.com/lance-format/lance/pull/7758
feat: support wider hamming hashes by @jackye1995 in https://github.com/lance-format/lance/pull/7767
feat(compaction): compact fragments over an overlay-count limit by @wjones127 in https://github.com/lance-format/lance/pull/7772
feat(fts): record scorer_build_ms for the exec-local BM25 scorer fallback by @LuQQiu in https://github.com/lance-format/lance/pull/7807
feat(fts): read inverted index params without opening the segment by @LuQQiu in https://github.com/lance-format/lance/pull/7816
feat: expose cached file metadata APIs on FileFragment by @jiaqizho in https://github.com/lance-format/lance/pull/7820
feat: add batch blob range reads by @Xuanwo in https://github.com/lance-format/lance/pull/7864

Bug Fixes 🐛

fix(python): fix tuple bug in _coerce_query_vector condition check by @hushengquan in https://github.com/lance-format/lance/pull/6555
fix: map object_store::Error::NotFound to Error::NotFound instead of Error::IO by @LuciferYang in https://github.com/lance-format/lance/pull/6569
fix: shift offsets after trimming values in merge_with_schema by @kaan-simbe in https://github.com/lance-format/lance/pull/6581
fix(transaction): reduce redundant code for new_version calculations by @yangshangqing95 in https://github.com/lance-format/lance/pull/6589
fix(index): keep HNSW IVF scratch dir alive during partition writing by @geserdugarov in https://github.com/lance-format/lance/pull/6980
fix: support manifests >5 GB via size-aware copy by @lixmgl in https://github.com/lance-format/lance/pull/7047
fix: route JSON index queries to the correct sub-parser by path by @ztorchan in https://github.com/lance-format/lance/pull/7072
fix(index): cap IVF train prefetch memory for nullable columns by @geserdugarov in https://github.com/lance-format/lance/pull/7143
fix: evaluate all list-element docs in FTS prefilter walk-the-allowlist branch by @Ar-maan05 in https://github.com/lance-format/lance/pull/7246
fix: apply per-segment filters and frag-reuse remap in BTree segment merge by @zhangyue19921010 in https://github.com/lance-format/lance/pull/7320
fix(index): drop stale scalar index entries after stable-row-id update by @wkalt in https://github.com/lance-format/lance/pull/7359
fix: reject DataReplacement racing concurrent Update/Delete/Merge by @wkalt in https://github.com/lance-format/lance/pull/7373
fix(ci): replace deprecated array.shape assignment for NumPy 2.5 by @zhangyue19921010 in https://github.com/lance-format/lance/pull/7384
fix(fts): enforce required terms for and queries by @BubbleCal in https://github.com/lance-format/lance/pull/7385
fix(index): use range block max for fts conjunction by @BubbleCal in https://github.com/lance-format/lance/pull/7387
fix(merge_insert): keep nested-field index correct on stable row id update by @jackye1995 in https://github.com/lance-format/lance/pull/7410
fix(update): keep nested-field index correct when updating a struct column by @jackye1995 in https://github.com/lance-format/lance/pull/7412
fix(io): clamp ObjectStore::io_parallelism() to at least 1 by @LuciferYang in https://github.com/lance-format/lance/pull/7414
fix(namespace): allow create_branch bootstrap on managed tables by @geruh in https://github.com/lance-format/lance/pull/7415
fix(fts): use async send in FTS index builder to prevent thread-pool … by @a-agmon in https://github.com/lance-format/lance/pull/7423
fix(rowids): union overlapping segments in RowAddrTreeMap::from by @wkalt in https://github.com/lance-format/lance/pull/7429
fix: relax DataReplacement conflict with concurrent Delete/Update by @brendanclement in https://github.com/lance-format/lance/pull/7433
fix(python): validate delete_rows offsets and add missing type stubs by @wkalt in https://github.com/lance-format/lance/pull/7448
fix(index): give each FTS partition a distinct scheduler base priority by @LuQQiu in https://github.com/lance-format/lance/pull/7449
fix(encoding): honor zstd compression scheme and level on large per-value buffers by @claydugo in https://github.com/lance-format/lance/pull/7460
fix(index): record IVF_HNSW index file sizes after writing footer by @Ali2Arslan in https://github.com/lance-format/lance/pull/7461
fix(core): allow all-null Map columns in schema evolution by @Ali2Arslan in https://github.com/lance-format/lance/pull/7462
fix(compaction): reject defer_index_remap with stable row IDs by @zhangyue19921010 in https://github.com/lance-format/lance/pull/7468
fix: convert Arrow JSON to Lance JSON in single-fragment create path by @xloya in https://github.com/lance-format/lance/pull/7469
fix: return Arrow JSON from FragmentSession.take by @xloya in https://github.com/lance-format/lance/pull/7470
fix: return Arrow JSON from FileFragment.take by @xloya in https://github.com/lance-format/lance/pull/7471
fix: convert Arrow JSON when updating columns via merge/update path by @xloya in https://github.com/lance-format/lance/pull/7472
fix: convert Arrow JSON in merge-insert full-fragment-rewrite fast path by @xloya in https://github.com/lance-format/lance/pull/7473
fix(index): preserve schema metadata when re-serializing loaded HNSW by @yanghua in https://github.com/lance-format/lance/pull/7476
fix(rowids): tolerate sparse overlapping chunks in the stable row id index by @wkalt in https://github.com/lance-format/lance/pull/7480
fix: account for SQ offset in dot distance by @Xuanwo in https://github.com/lance-format/lance/pull/7481
fix(merge-insert): apply Delete/Fail on the indexed-scan path by @hamersaw in https://github.com/lance-format/lance/pull/7484
fix(mem-wal): apply cross-generation block-list to in-memory scan arms by @hamersaw in https://github.com/lance-format/lance/pull/7489
fix: invalidate indices for fields rewritten by Merge by @wkalt in https://github.com/lance-format/lance/pull/7491
fix(index): implement with_io_priority for FailNewFileStore test store by @LuciferYang in https://github.com/lance-format/lance/pull/7495
fix(arrow): preserve inner nulls in convert_to_floating_point by @LuciferYang in https://github.com/lance-format/lance/pull/7498
fix: improve FM index query performance by @jackye1995 in https://github.com/lance-format/lance/pull/7507
fix: avoid stack overflow on deep logical filters by @Xuanwo in https://github.com/lance-format/lance/pull/7510
fix(compaction): exclude system indices from compaction binning by @xuanyu-z in https://github.com/lance-format/lance/pull/7516
fix(index): accept large_string (LargeUtf8) for BTREE and ZONEMAP scalar indices by @FANNG1 in https://github.com/lance-format/lance/pull/7525
fix(index): work around rustc nightly ICE in NGramIndexBuilder::stream_spill_reader by @westonpace in https://github.com/lance-format/lance/pull/7534
fix: recover from stale cached manifest size on read by @wkalt in https://github.com/lance-format/lance/pull/7542
fix(lance-encoding): rebase nested-list offsets correctly across pages by @yesunbmh in https://github.com/lance-format/lance/pull/7546
fix(mem-wal): fence writer on WAL persistence failure by @hamersaw in https://github.com/lance-format/lance/pull/7547
fix(mem_wal): use slice-aware size estimate for memtable flush threshold by @hamersaw in https://github.com/lance-format/lance/pull/7563
fix(index): translate address-domain scalar index results to row ids under stable row ids by @charleshuang119 in https://github.com/lance-format/lance/pull/7565
fix(lance-io): include goosefs feature in DEFAULT_CLOUD_BLOCK_SIZE cfg gate by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/7570
fix: preserve structural page load errors by @ddupg in https://github.com/lance-format/lance/pull/7571
fix(cache): count key footprint in MokaCacheBackend eviction weight by @Ali2Arslan in https://github.com/lance-format/lance/pull/7573
fix(index): exclude shared metadata cache from LanceIndexStore deep size by @Ali2Arslan in https://github.com/lance-format/lance/pull/7574
fix(commit): verify finalized external manifests by @BubbleCal in https://github.com/lance-format/lance/pull/7578
fix(python): preserve PQ num_bits in model training by @ddupg in https://github.com/lance-format/lance/pull/7583
fix: respect PQ num_bits for numpy codebooks by @ddupg in https://github.com/lance-format/lance/pull/7586
fix(python): avoid CPU bf16 compile flake by @ddupg in https://github.com/lance-format/lance/pull/7587
fix: avoid scheduler deadlock on same-priority chunks by @Xuanwo in https://github.com/lance-format/lance/pull/7588
fix(fsst): correct decoder output-buffer size contract to 8x by @LuciferYang in https://github.com/lance-format/lance/pull/7589
fix: late-materialize blob columns read as binary by @wkalt in https://github.com/lance-format/lance/pull/7593
fix(fts): split tail-partition merge by the worker memory budget by @BubbleCal in https://github.com/lance-format/lance/pull/7601
fix: chunk ngram posting list writes by byte size to avoid i32 offset overflow by @lancedb-robot in https://github.com/lance-format/lance/pull/7607
fix: preserve DataFile base_id in DataReplacement commits by @jackye1995 in https://github.com/lance-format/lance/pull/7609
fix(goosefs): decouple OpenDAL root from URL path to avoid ObjectStore cache-key collision by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/7612
fix: treat blob descriptors as opaque projections by @Xuanwo in https://github.com/lance-format/lance/pull/7618
fix(fts): use complete English stop-word list for ICU tokenizer by @LuQQiu in https://github.com/lance-format/lance/pull/7621
fix: relax test_query_delta_indices assertion for approximate index by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/7622
fix(scanner): honor batch_readahead to bound v2 scan decode concurrency by @zhangyue19921010 in https://github.com/lance-format/lance/pull/7632
fix(fts): enforce fuzzy max_expansions globally across index partitions by @BubbleCal in https://github.com/lance-format/lance/pull/7634
fix(encoding): select RLE widths by encoded size by @Xuanwo in https://github.com/lance-format/lance/pull/7636
fix(lance-io): drop reservation when future is cancelled by @valkum in https://github.com/lance-format/lance/pull/7638
fix(deps): bump crossbeam-epoch to 0.9.20 for RUSTSEC-2026-0204 by @wjones127 in https://github.com/lance-format/lance/pull/7644
fix: return error instead of panicking on values too wide for miniblock by @zhangyue19921010 in https://github.com/lance-format/lance/pull/7650
fix: build list FTS indexes as row documents by @Xuanwo in https://github.com/lance-format/lance/pull/7656
fix(index): describe bitmap-less system indices in describe_indices by @hamersaw in https://github.com/lance-format/lance/pull/7667
fix: solve hang in train_streaming_coreset_ivf_model by @cmccabe in https://github.com/lance-format/lance/pull/7676
fix(index): batch IVF streaming partition search off the CPU pool by @a-agmon in https://github.com/lance-format/lance/pull/7680
fix(lance-core): avoid aliasing &mut Runtime in global_cpu_runtime by @LuciferYang in https://github.com/lance-format/lance/pull/7682
fix: avoid creating directory namespace manifest on read by @Xuanwo in https://github.com/lance-format/lance/pull/7687
fix(python): ignore PyTorch 3.14 script_method warning by @Xuanwo in https://github.com/lance-format/lance/pull/7689
fix: preserve case in generated field path expressions by @skyshineb in https://github.com/lance-format/lance/pull/7698
fix: skip empty string docs in FTS stats by @Ecthlion in https://github.com/lance-format/lance/pull/7699
fix(encoding): tolerate legacy RLE chunks that overflow the declared value count by @jackye1995 in https://github.com/lance-format/lance/pull/7708
fix: round-trip empty blob values in the 2.1+ structural encoding by @wkalt in https://github.com/lance-format/lance/pull/7717
fix(encoding): reject stalled RLE miniblock encoding by @Ali2Arslan in https://github.com/lance-format/lance/pull/7729
fix(encoding): decode empty inline-bitpacked blocks by @Xuanwo in https://github.com/lance-format/lance/pull/7752
fix: reconstruct protobuf schema fields in linear time by @Xuanwo in https://github.com/lance-format/lance/pull/7766
fix: train IVF indexes on fragment subsets by @jackye1995 in https://github.com/lance-format/lance/pull/7768
fix(mem-wal): propagate final flush failures from ShardWriter::close by @u70b3 in https://github.com/lance-format/lance/pull/7769
perf: skip fragment-reuse index for compactions with no indexed data by @xuanyu-z in https://github.com/lance-format/lance/pull/7774
fix: bracket match query display by @jackye1995 in https://github.com/lance-format/lance/pull/7796
fix(dataset): reject all system column names on write by @LuciferYang in https://github.com/lance-format/lance/pull/7797
fix(ci): pin goosefs-sdk to 0.1.5 by @zhangyue19921010 in https://github.com/lance-format/lance/pull/7798
fix(table): route tos:// scheme to ConditionalPutCommitHandler by @zhangyue19921010 in https://github.com/lance-format/lance/pull/7824
fix(python): allow blob writer cleanup across threads by @justinrmiller in https://github.com/lance-format/lance/pull/7827
fix(index): reuse cached FTS document lengths by @Xuanwo in https://github.com/lance-format/lance/pull/7830
fix: make FTS metadata loading retry-safe by @Xuanwo in https://github.com/lance-format/lance/pull/7838
fix(fts): restore format version environment fallback by @Xuanwo in https://github.com/lance-format/lance/pull/7855
fix: dispatch concurrent blob range reads by @Xuanwo in https://github.com/lance-format/lance/pull/7861
fix(fts): use v3 as the capability gate by @Xuanwo in https://github.com/lance-format/lance/pull/7866
fix(index): return error instead of panicking when hierarchical k-means can't reach target_k by @wjones127 in https://github.com/lance-format/lance/pull/7869
fix: trim fragment-reuse index for non-covering indexes by @xuanyu-z in https://github.com/lance-format/lance/pull/7870
fix(fts): preserve parameter compatibility by @Xuanwo in https://github.com/lance-format/lance/pull/7874

Documentation 📚

docs: add cleanup and auto cleanup documentation by @westonpace in https://github.com/lance-format/lance/pull/6546
docs: add fragment sizing guidance to performance guide by @hamersaw in https://github.com/lance-format/lance/pull/6606
docs(index): document public API of LogicalScalarIndex by @westonpace in https://github.com/lance-format/lance/pull/6804
docs(python): correct scalar index type count by @hfutatzhanghb in https://github.com/lance-format/lance/pull/7353
docs: specify data overlay files for the table format by @wjones127 in https://github.com/lance-format/lance/pull/7381
docs: clarify local LTO guidance by @Xuanwo in https://github.com/lance-format/lance/pull/7400
docs: clarify Lance format stability by @Xuanwo in https://github.com/lance-format/lance/pull/7475
docs: correct file-format version matrix for 2.2 stable / 2.3 unstable by @LuciferYang in https://github.com/lance-format/lance/pull/7479
docs: improve phrasing and update format spec image by @prrao87 in https://github.com/lance-format/lance/pull/7505
docs(arrow): add SAFETY comments to lance-arrow unsafe blocks by @LuciferYang in https://github.com/lance-format/lance/pull/7511
docs: add FTS v2 migration note by @BubbleCal in https://github.com/lance-format/lance/pull/7522
docs: document read_blobs for blob payload reads by @Xuanwo in https://github.com/lance-format/lance/pull/7530
docs: add blob v2 overview images by @Xuanwo in https://github.com/lance-format/lance/pull/7581
docs(python): document beta release install index by @Xuanwo in https://github.com/lance-format/lance/pull/7582
docs(core): document spawn_cpu limitations and audit call sites by @wjones127 in https://github.com/lance-format/lance/pull/7643
docs: update MemWAL format spec by @jackye1995 in https://github.com/lance-format/lance/pull/7655
docs: define file format stability contract by @Xuanwo in https://github.com/lance-format/lance/pull/7779
docs: implement the Lance Docs design as the mkdocs site theme by @Xuanwo in https://github.com/lance-format/lance/pull/7821
docs: make the docs site theme usable on mobile and narrow screens by @Xuanwo in https://github.com/lance-format/lance/pull/7853

Performance Improvements 🚀

perf(knn): reduce memory for batch flat vector search by @LeoReeYang in https://github.com/lance-format/lance/pull/6950
perf(index): reduce TwoFileShuffler peak memory via interleave sort by @wjones127 in https://github.com/lance-format/lance/pull/7295
perf(index): add exact null-row bitmap to zone map and bloom filter by @westonpace in https://github.com/lance-format/lance/pull/7372
perf: speed up cold read up to 8x by lazily load column metadata by @Xuanwo in https://github.com/lance-format/lance/pull/7375
perf(fts): prune low-scoring conjunction candidates by @BubbleCal in https://github.com/lance-format/lance/pull/7386
perf: speed up ICU FTS index builds by 11% by @Xuanwo in https://github.com/lance-format/lance/pull/7393
perf(index): improve FTS search metadata caching by @Xuanwo in https://github.com/lance-format/lance/pull/7398
perf(fts): open fts segments as scalar indices by @BubbleCal in https://github.com/lance-format/lance/pull/7408
perf(fts): remove OR hot path bookkeeping by @BubbleCal in https://github.com/lance-format/lance/pull/7416
perf(index): parallelize FMIndex partition builds by @jackye1995 in https://github.com/lance-format/lance/pull/7422
perf(fts): use block max for or tail bounds by @BubbleCal in https://github.com/lance-format/lance/pull/7435
perf(fts): prewarm larger chunks concurrently by @BubbleCal in https://github.com/lance-format/lance/pull/7436
perf(index): coalesce concurrent scalar-index opens (single-flight) by @Ali2Arslan in https://github.com/lance-format/lance/pull/7464
perf: merge half-open range queries on the same BTree index by @xloya in https://github.com/lance-format/lance/pull/7477
perf(arrow): avoid per-element allocation in BFloat16Array::from by @LuciferYang in https://github.com/lance-format/lance/pull/7500
perf(index): skip FM-Index rebuild when merging a single fully-live segment by @jackye1995 in https://github.com/lance-format/lance/pull/7569
perf(dataset): reuse session-cached manifest when opening a dataset by @zhangyue19921010 in https://github.com/lance-format/lance/pull/7576
perf(filtered-read): lower hot-path read/decode spans to debug by @LuQQiu in https://github.com/lance-format/lance/pull/7590
perf(fts): cache the num_tokens-only DocSet on LazyDocSet by @BubbleCal in https://github.com/lance-format/lance/pull/7600
perf(fts): bulk MAXSCORE search path for top-k disjunctions by @BubbleCal in https://github.com/lance-format/lance/pull/7603
perf(arrow): zero-copy BFloat16Array::from(Vec<bf16>) via Buffer::from_vec by @LuciferYang in https://github.com/lance-format/lance/pull/7614
perf(index): bulk conjunction path for FTS AND and phrase queries by @BubbleCal in https://github.com/lance-format/lance/pull/7624
perf(index): norm-addend cache and slim top-k heap for FTS scoring by @BubbleCal in https://github.com/lance-format/lance/pull/7629
perf(dataset): reuse session-cached manifest on checkout by @zhangyue19921010 in https://github.com/lance-format/lance/pull/7661
perf: plan take operations through FilteredReadExec's range-read path by @LuQQiu in https://github.com/lance-format/lance/pull/7672
perf: increase default memory pool to 150MB with 40MB sort spill reservation by @westonpace in https://github.com/lance-format/lance/pull/7675
perf(index): pack prewarmed FTS posting groups by @BubbleCal in https://github.com/lance-format/lance/pull/7720
perf(filtered-read): consolidate take-shaped masked reads off the consumer by @LuQQiu in https://github.com/lance-format/lance/pull/7783
perf: support indexed metadata for structural projections by @Xuanwo in https://github.com/lance-format/lance/pull/7790
perf(dataset): read transactions by version without populating session caches by @xuanyu-z in https://github.com/lance-format/lance/pull/7817

Other Changes

refactor(index): rely on total ordering for nan zonemap max by @HaochengLIU in https://github.com/lance-format/lance/pull/7049
refactor: remove as_vector_index from the Index trait by @westonpace in https://github.com/lance-format/lance/pull/7392
refactor(index): introduce RowIdRemapper trait to decouple ScalarIndexPlugin from FragReuseIndex by @westonpace in https://github.com/lance-format/lance/pull/7394
refactor(index): extract BasicTrainer trait from ScalarIndexPlugin by @westonpace in https://github.com/lance-format/lance/pull/7395
chore(deps): bump cmov from 0.5.3 to 0.5.4 in /test_data/fri_straddle_pre_6610/datagen by @app/dependabot in https://github.com/lance-format/lance/pull/7594
refactor(python): move tensorflow integration out of tree by @Xuanwo in https://github.com/lance-format/lance/pull/7685
refactor(index): introduce lance-index-core crate by @westonpace in https://github.com/lance-format/lance/pull/7713
refactor(java): rename WriteDatasetBuilder.tableProperties to properties by @puchengy in https://github.com/lance-format/lance/pull/7715
refactor(encoding): clarify mini-block repdef budget by @Xuanwo in https://github.com/lance-format/lance/pull/7751

Full Changelog: https://github.com/lance-format/lance/compare/release-root/9.0.0-beta.N...v9.0.0

Open source →

8.0.01 Jul 2026

What's Changed

Breaking Changes 🛠

feat!: migrate bitmap to index segment based by @Xuanwo in https://github.com/lance-format/lance/pull/6869
fix(python)!: derive index type from details instead of opening the index by @wjones127 in https://github.com/lance-format/lance/pull/6903
refactor!: remove index segment builder by @Xuanwo in https://github.com/lance-format/lance/pull/6997
refactor(index)!: move distributed BTree build to segmented index framework by @zhangyue19921010 in https://github.com/lance-format/lance/pull/7013
feat!: return write summaries from file writers by @Xuanwo in https://github.com/lance-format/lance/pull/7096
perf!: avoid listing index files after writes by @Xuanwo in https://github.com/lance-format/lance/pull/7129
fix(dataset)!: fail-fast casting for columns with attached indices by @WenDing-Y in https://github.com/lance-format/lance/pull/7158
feat(vector)!: add approx mode for RaBitQ search by @BubbleCal in https://github.com/lance-format/lance/pull/7179
perf(vector)!: add dedicated SIMD kernels for RaBitQ ex-code reranking by @BubbleCal in https://github.com/lance-format/lance/pull/7205
refactor!: rename FMIndexIndexDetails to FMIndexDetails by @westonpace in https://github.com/lance-format/lance/pull/7397

Critical Fixes ‼️

fix: merge_insert silently drops matches when a leading payload column is all-null by @Ar-maan05 in https://github.com/lance-format/lance/pull/7251

New Features 🎉

feat: expose tracked_files and all_files on LanceDataset by @wjones127 in https://github.com/lance-format/lance/pull/6011
feat: add commit timeout to CommitBuilder by @wjones127 in https://github.com/lance-format/lance/pull/6773
feat: add segmented BTree index merge_segments support by @zhangyue19921010 in https://github.com/lance-format/lance/pull/6889
feat(index): add streaming ivf kmeans training by @BubbleCal in https://github.com/lance-format/lance/pull/6913
feat: use indexes to accelerate filtered count_rows by @westonpace in https://github.com/lance-format/lance/pull/6916
feat(java): allow schema override for fragment writes by @beinan in https://github.com/lance-format/lance/pull/6919
feat: make ICU the default FTS tokenizer by @Xuanwo in https://github.com/lance-format/lance/pull/6968
feat: support Utf8View and BinaryView in encoding and filter coercion by @xuanyu-z in https://github.com/lance-format/lance/pull/6985
feat(python): add shared RaBitQ rotation for distributed IVF_RQ builds by @gstamatakis95 in https://github.com/lance-format/lance/pull/7014
feat(blobv2): support all BlobKind types in blob v2 compact_files by @yyzhao2025 in https://github.com/lance-format/lance/pull/7017
feat: add TOS object store support via OpenDAL by @ddupg in https://github.com/lance-format/lance/pull/7019
feat(index): implement FM-Index scalar index for exact substring search by @beinan in https://github.com/lance-format/lance/pull/7026
feat(python): expose external blob mode and outside-base option for fragments by @plotor in https://github.com/lance-format/lance/pull/7028
feat(lance-io): add GooseFS object store provider by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/7034
feat(index): support multi-bit IVF_RQ storage by @BubbleCal in https://github.com/lance-format/lance/pull/7038
feat: expose getters for ScalarIndexExec by @LuQQiu in https://github.com/lance-format/lance/pull/7039
feat: expose methods and getters of scalar index for possible distributed execution by @LuQQiu in https://github.com/lance-format/lance/pull/7045
feat(mem_wal): configurable HNSW build params for MemTable writers by @touch-of-grey in https://github.com/lance-format/lance/pull/7054
feat: dedup FTS results across LSM tiers in LsmFtsSearchPlanner by @hamersaw in https://github.com/lance-format/lance/pull/7066
feat(index): support raw-query ivf rq search by @BubbleCal in https://github.com/lance-format/lance/pull/7078
feat: add EnforceDistribution to physical optimizer by @wjones127 in https://github.com/lance-format/lance/pull/7086
feat(java): add missing scanner and merge insert params to align with Python/Rust by @WenDing-Y in https://github.com/lance-format/lance/pull/7100
feat: populate enriched IndexContent fields in dir namespace ListTableIndices by @wjones127 in https://github.com/lance-format/lance/pull/7109
feat(index): support configurable multi-segment FM-Index builds by @beinan in https://github.com/lance-format/lance/pull/7123
feat: support merging zonemap index segments by @Xuanwo in https://github.com/lance-format/lance/pull/7128
feat(index): accelerate regex and infix LIKE with the ngram index by @wombatu-kun in https://github.com/lance-format/lance/pull/7139
feat(rust): add cleanup explain API by @yanghua in https://github.com/lance-format/lance/pull/7147
feat: stabilize cache codec with a versioned envelope by @wjones127 in https://github.com/lance-format/lance/pull/7163
feat(lance-select): expose selected rows accessor on NullableRowAddrSet by @LuQQiu in https://github.com/lance-format/lance/pull/7164
feat: branch-aware table version ops in directory and rest namespaces by @brendanclement in https://github.com/lance-format/lance/pull/7166
feat(python): expose segment FTS build through create_index_uncommitted by @ddupg in https://github.com/lance-format/lance/pull/7170
feat(python): expose zonemap segment builds by @everySympathy in https://github.com/lance-format/lance/pull/7177
feat(dir-catalog): add reader/writer feature flags to __manifest by @jackye1995 in https://github.com/lance-format/lance/pull/7191
feat(index): expose per-query I/O metrics on ANN operators by @wombatu-kun in https://github.com/lance-format/lance/pull/7204
feat(mem-wal): snapshot-consistent as-of cut for fresh-tier membership by @hamersaw in https://github.com/lance-format/lance/pull/7215
feat: expose io_buffer_size in CompactionOptions by @aimanmalib in https://github.com/lance-format/lance/pull/7226
fix(python): expose stable row id property in stub by @BubbleCal in https://github.com/lance-format/lance/pull/7249
feat: bump lance-namespace-reqwest-client to 0.8.6 (source_task_size) by @justinrmiller in https://github.com/lance-format/lance/pull/7254
feat: configure blob inline threshold per column by @Xuanwo in https://github.com/lance-format/lance/pull/7269
feat(mem_wal): warm flushed generations into shared caches before query by @hamersaw in https://github.com/lance-format/lance/pull/7284
feat(java): expose RTree scalar index type to Java by @zhangyue19921010 in https://github.com/lance-format/lance/pull/7291
feat: expose session cache key inventory by @jackye1995 in https://github.com/lance-format/lance/pull/7298
feat: support mixed-language FTS stop words by @Xuanwo in https://github.com/lance-format/lance/pull/7324
feat(scalar): expose LogicalScalarIndex::try_new and load_named_scalar_segments by @LuQQiu in https://github.com/lance-format/lance/pull/7339
feat: allow tuning miniblock value chunks to 32k by @Xuanwo in https://github.com/lance-format/lance/pull/7356

Bug Fixes 🐛

fix(encoding): plan sparse structural miniblock pages by @Xuanwo in https://github.com/lance-format/lance/pull/6787
fix(java): resolve JNI classloader bug on dispatcher thread in Spark by @sezruby in https://github.com/lance-format/lance/pull/6946
fix: preserve zero-length buffers in binary copy compaction by @zhangyue19921010 in https://github.com/lance-format/lance/pull/6992
fix(storage): retry throttled fts metadata listing by @BubbleCal in https://github.com/lance-format/lance/pull/6994
fix: support multivector IVF centroids in segment builds by @ddupg in https://github.com/lance-format/lance/pull/6995
fix(python): validate BFloat16.from_bytes length by @ddupg in https://github.com/lance-format/lance/pull/6998
fix(test): tolerate boundary-tie in IVF distance_range assertions by @xuanyu-z in https://github.com/lance-format/lance/pull/6999
fix(datafusion): coerce filter literals for dictionary-encoded columns by @valkum in https://github.com/lance-format/lance/pull/7003
fix: stream object copies larger than cloud's CopyObject limit by @vivek-bharathan in https://github.com/lance-format/lance/pull/7004
fix: restore simple FTS tokenizer default by @Xuanwo in https://github.com/lance-format/lance/pull/7006
fix: expose Hugging Face download mode by @Xuanwo in https://github.com/lance-format/lance/pull/7022
fix(python): clamp target partition sizing by @ddupg in https://github.com/lance-format/lance/pull/7036
fix(fts): handle empty query tokens in flat full-text search by @vivek-bharathan in https://github.com/lance-format/lance/pull/7046
fix(lance-index): fix some flaky tests by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/7052
fix: specify roaring's patch version by @HuaHuaY in https://github.com/lance-format/lance/pull/7056
fix(filtered-read): record IO metrics even when filter matches no rows by @westonpace in https://github.com/lance-format/lance/pull/7057
fix: handle nested JSON conversion recursively by @Xuanwo in https://github.com/lance-format/lance/pull/7060
fix: retry S3 multipart request timeouts by @Xuanwo in https://github.com/lance-format/lance/pull/7061
fix(fts): size cached posting lists by referenced slice by @vivek-bharathan in https://github.com/lance-format/lance/pull/7068
fix: cap exec-node parallelism to DataFusion target_partitions by @wjones127 in https://github.com/lance-format/lance/pull/7087
fix(mem-wal): fence predecessor with a WAL sentinel on claim by @hamersaw in https://github.com/lance-format/lance/pull/7110
fix(fts): reset TokenSet next_id and total_length after remap by @vivek-bharathan in https://github.com/lance-format/lance/pull/7115
fix(linalg): reduce cosine bench TOTAL to avoid FixedSizeBinaryArray i32 overflow by @westonpace in https://github.com/lance-format/lance/pull/7116
fix: compile AVX-512 dist table for target CPU by @Xuanwo in https://github.com/lance-format/lance/pull/7121
fix(namespace): tolerate reqwest client model additions by @Xuanwo in https://github.com/lance-format/lance/pull/7124
fix(python): avoid pandas requirement for add_columns UDFs by @prrao87 in https://github.com/lance-format/lance/pull/7131
fix(mem-wal): surface fenced flush to durability waiters instead of hanging by @hamersaw in https://github.com/lance-format/lance/pull/7132
fix: remove empty index segment when committing real segments by @wkalt in https://github.com/lance-format/lance/pull/7141
fix(dataset): resolve Blob v2 external URIs and robustly clean failed writes in add_columns by @yyzhao2025 in https://github.com/lance-format/lance/pull/7152
fix(encoding): improve dict sampling for sorted strings by @Xuanwo in https://github.com/lance-format/lance/pull/7159
fix(index): support HNSW segment merge by @Xuanwo in https://github.com/lance-format/lance/pull/7178
fix: search all unindexed fragments in FTS flat match by @wkalt in https://github.com/lance-format/lance/pull/7188
fix(fts): recompute next_id when loading FST token set by @wkalt in https://github.com/lance-format/lance/pull/7200
fix(python): python binding of Compaction.commit by @gstamatakis95 in https://github.com/lance-format/lance/pull/7210
fix(namespace): tolerate new IndexContent fields from reqwest client by @wkalt in https://github.com/lance-format/lance/pull/7212
fix(rust): fixed IVF_RQ indexes ignoring the frag-reuse index by @gstamatakis95 in https://github.com/lance-format/lance/pull/7217
fix: branch support on object stores, python branch ops, s3/ddb integration tests by @brendanclement in https://github.com/lance-format/lance/pull/7219
fix: record TakeExec output and I/O metrics by @skyshineb in https://github.com/lance-format/lance/pull/7228
fix: handle empty batch from deletion in add_columns_from_stream by @zhangyang0418 in https://github.com/lance-format/lance/pull/7233
perf(dir-catalog): avoid unchanged manifest reloads by @jackye1995 in https://github.com/lance-format/lance/pull/7234
fix: return error instead of panicking on zero-dimension fixed-size-list columns by @DanielMao1 in https://github.com/lance-format/lance/pull/7247
fix: avoid versioned describe table for namespace opens by @FANNG1 in https://github.com/lance-format/lance/pull/7250
fix(mem_wal): push LIMIT/OFFSET down to fresh-tier scan sources by @hamersaw in https://github.com/lance-format/lance/pull/7256
fix: make FTS finalization idempotent by @jackye1995 in https://github.com/lance-format/lance/pull/7272
fix(index): share FTS position stream offsets by @BubbleCal in https://github.com/lance-format/lance/pull/7275
fix: reject nested legacy blobs in v2.2 by @Xuanwo in https://github.com/lance-format/lance/pull/7278
fix: support blob v2 in nested structs by @Xuanwo in https://github.com/lance-format/lance/pull/7281
fix: remap index data when its fragment bitmap was already coverage-remapped by @xuanyu-z in https://github.com/lance-format/lance/pull/7286
fix(index): refresh PQ storage row ids after fragment-reuse remap by @xuanyu-z in https://github.com/lance-format/lance/pull/7315
fix(fts): keep inverted index correct under deletions in the fragment-reuse window by @xuanyu-z in https://github.com/lance-format/lance/pull/7325
fix: stream FM index partition builds by @jackye1995 in https://github.com/lance-format/lance/pull/7354

Documentation 📚

docs: fix incorrect method call in create_scalar_index example by @wayneadams in https://github.com/lance-format/lance/pull/5399
docs: add new maintainers and update affiliations by @prrao87 in https://github.com/lance-format/lance/pull/6990
docs: update Yang Jie ecosystem roles by @LuciferYang in https://github.com/lance-format/lance/pull/7055
docs: fix broken pre-release install commands by @prrao87 in https://github.com/lance-format/lance/pull/7088
docs: correct alter_columns nullable docstring by @Alowator in https://github.com/lance-format/lance/pull/7095
docs: update ecosystem role for maintainer by @prrao87 in https://github.com/lance-format/lance/pull/7102
docs: clarify distributed vector model scopes by @Xuanwo in https://github.com/lance-format/lance/pull/7148
docs(object_store): add Tencent Cloud COS and GooseFS configuration by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/7151

Performance Improvements 🚀

perf: make HNSW cheaper to load by @wombatu-kun in https://github.com/lance-format/lance/pull/6798
perf(merge_insert): probe every indexed key column for composite joins by @ytyky in https://github.com/lance-format/lance/pull/6878
perf(index): reduce describe indices fragment scans by @hfutatzhanghb in https://github.com/lance-format/lance/pull/6958
perf(index): reduce HNSW build distance overhead by @BubbleCal in https://github.com/lance-format/lance/pull/6978
perf(search): reuse IVF_RQ query scratch by @BubbleCal in https://github.com/lance-format/lance/pull/6982
perf(fts): defer DocSet load until wand actually needs it by @wkalt in https://github.com/lance-format/lance/pull/6983
perf(mem_wal): match hnswlib throughput via runtime AVX-512 f32 distance by @touch-of-grey in https://github.com/lance-format/lance/pull/7009
perf(mem_wal): faster MemTable point lookups by @touch-of-grey in https://github.com/lance-format/lance/pull/7011
perf(index): cache wand head posting doc id by @BubbleCal in https://github.com/lance-format/lance/pull/7023
perf(mem_wal): close the FTS MemTable performance gap with Lucene by @touch-of-grey in https://github.com/lance-format/lance/pull/7029
perf(file): serve global-buffer reads from the captured tail bytes by @wjones127 in https://github.com/lance-format/lance/pull/7042
perf(fts): group small posting lists into shared cache entries by @wjones127 in https://github.com/lance-format/lance/pull/7043
perf(fts): share a top-k threshold across partitions during WAND by @wkalt in https://github.com/lance-format/lance/pull/7062
perf(index): avoid HEAD call when opening vector indexes by @gstamatakis95 in https://github.com/lance-format/lance/pull/7064
perf(index): fix O(N log N) warm-cache regression in BitmapIndex by @westonpace in https://github.com/lance-format/lance/pull/7079
perf: block-max pruning on the disjunctive WAND path by @wkalt in https://github.com/lance-format/lance/pull/7089
perf(pq): wrap l2_targets in Arc to eliminate per-partition clones by @yuju-huang in https://github.com/lance-format/lance/pull/7093
perf: reuse WAND lead buffer capacity in push_back_leads by @wkalt in https://github.com/lance-format/lance/pull/7153
perf(index): cache parsed btree lookup state by @LuQQiu in https://github.com/lance-format/lance/pull/7161
perf(dir-catalog): rewrite manifest mutations with copy-on-write by @jackye1995 in https://github.com/lance-format/lance/pull/7176
perf(index): query BTree lookup batch directly by @wjones127 in https://github.com/lance-format/lance/pull/7186
perf(fts): stream prewarm posting lists in chunks by @wkalt in https://github.com/lance-format/lance/pull/7208
perf(vector): vectorize RaBitQ dist table quantization by @BubbleCal in https://github.com/lance-format/lance/pull/7241
perf(vector): vectorize RaBitQ top-k lower-bound pruning scan by @BubbleCal in https://github.com/lance-format/lance/pull/7243
perf(mem_wal): parallelize fresh-tier source planning and execution by @hamersaw in https://github.com/lance-format/lance/pull/7257
perf(index): set ivf_rq target partition size to 4096 by @BubbleCal in https://github.com/lance-format/lance/pull/7273
perf(fts): push prefilter through scalar index on flat FTS by @LuQQiu in https://github.com/lance-format/lance/pull/7283
perf(scalar): compile IsIn predicate once across BTree pages by @JulianYG in https://github.com/lance-format/lance/pull/7287
perf: batch fragment-reuse index remap into a single rebuild + commit by @xuanyu-z in https://github.com/lance-format/lance/pull/7317
perf(fts): defer term frequency collection by @BubbleCal in https://github.com/lance-format/lance/pull/7357
perf(index): reuse WAND doc info by @BubbleCal in https://github.com/lance-format/lance/pull/7358

Other Changes

refactor: replace deepsize crate with custom DeepSizeOf trait for Arrow-aware memory accounting by @wjones127 in https://github.com/lance-format/lance/pull/6229
refactor: move from 3-variant index search result to single generic variant by @westonpace in https://github.com/lance-format/lance/pull/6902
refactor(io/exec): address #6799 review (drop helper, inline RAII timer) by @brendanclement in https://github.com/lance-format/lance/pull/6907
refactor: clean up lance-index/lance-table boundary by @wjones127 in https://github.com/lance-format/lance/pull/6988
refactor(index): scaffold ivfrq split code layout by @BubbleCal in https://github.com/lance-format/lance/pull/7021
fix: advance kmeans redo random init rng by @ddupg in https://github.com/lance-format/lance/pull/7074
refactor!(dir-catalog): remove table_version_storage_enabled and __manifest version storage by @jackye1995 in https://github.com/lance-format/lance/pull/7222

Full Changelog: https://github.com/lance-format/lance/compare/release-root/8.0.0-beta.N...v8.0.0

Open source →

7.0.027 May 2026

What's Changed

Breaking Changes 🛠

feat!: make dataset object store access base-aware by @jackye1995 in https://github.com/lance-format/lance/pull/6647
perf!: run scheduler initialize eagerly in async read_tasks by @westonpace in https://github.com/lance-format/lance/pull/6710
fix!: disable auto-cleanup by default by @touch-of-grey in https://github.com/lance-format/lance/pull/6755

New Features 🎉

feat: implement vector index details by @wjones127 in https://github.com/lance-format/lance/pull/6099
feat(lsm): expose lsm api to python by @zhangyue19921010 in https://github.com/lance-format/lance/pull/6259
feat: add branch/tag metadata maps and tag timestamps by @majin1102 in https://github.com/lance-format/lance/pull/6364
feat: route partial-schema merge_insert through the v2 write path by @wombatu-kun in https://github.com/lance-format/lance/pull/6472
feat: route find-or-create merge_insert through the v2 write path by @wombatu-kun in https://github.com/lance-format/lance/pull/6473
feat: expose base scoped store bindings to java by @zhangyue19921010 in https://github.com/lance-format/lance/pull/6548
feat: add unenforced_clustering_key to format spec by @beinan in https://github.com/lance-format/lance/pull/6552
feat(spec): formally split catalog, namespace, table and index specs by @jackye1995 in https://github.com/lance-format/lance/pull/6566
feat: support distributed bitmap index build by @zhangyue19921010 in https://github.com/lance-format/lance/pull/6598
feat: support segmented btree indices by @beinan in https://github.com/lance-format/lance/pull/6605
feat: expose FTS exec internals to enable distributed planning by @vivek-bharathan in https://github.com/lance-format/lance/pull/6648
feat: opt-in env var to drop centroids from vector index stats by @westonpace in https://github.com/lance-format/lance/pull/6654
feat: add write ahead log appender and tailer primitives by @touch-of-grey in https://github.com/lance-format/lance/pull/6669
feat: unify WalAppender into ShardWriter via enable_memtable mode by @touch-of-grey in https://github.com/lance-format/lance/pull/6675
feat: show scalar index type in ScalarIndexQuery plan output by @wkalt in https://github.com/lance-format/lance/pull/6677
feat(namespace): add backfill and refresh operations by @jackye1995 in https://github.com/lance-format/lance/pull/6678
feat(fts): implement CacheCodec for posting lists and positions by @wkalt in https://github.com/lance-format/lance/pull/6691
feat(java): expose scan execution stats by @fangbo in https://github.com/lance-format/lance/pull/6712
feat(mem_wal): flush active memtable on ShardWriter::close by @touch-of-grey in https://github.com/lance-format/lance/pull/6717
feat(index): merge incremental inverted index segments by @BubbleCal in https://github.com/lance-format/lance/pull/6737
feat: allow LSM scanner to read fresh tier without a base table by @hamersaw in https://github.com/lance-format/lance/pull/6749
feat: add manifest version hint for fast latest-version lookup by @touch-of-grey in https://github.com/lance-format/lance/pull/6752
feat(io): add shared-memory:// scheme for cross-component in-memory state by @hamersaw in https://github.com/lance-format/lance/pull/6753
feat(mem_wal): add ShardWriter manual compaction APIs by @hamersaw in https://github.com/lance-format/lance/pull/6766
feat(java): add Dataset.takeRows() for physical row ID access by @alex766 in https://github.com/lance-format/lance/pull/6772
feat: expose uncommitted delete transactions by @ragnorc in https://github.com/lance-format/lance/pull/6781
feat: support FTS index segment merging by @Xuanwo in https://github.com/lance-format/lance/pull/6790
feat(python): support nested blob fields in to_pandas by @geruh in https://github.com/lance-format/lance/pull/6791
feat(index): serializable cache for the BTree scalar index by @wjones127 in https://github.com/lance-format/lance/pull/6793
feat: add Lance-native MemWAL HNSW for shard writer by @touch-of-grey in https://github.com/lance-format/lance/pull/6795
feat(io): make shared-memory:// provider available outside tests by @hamersaw in https://github.com/lance-format/lance/pull/6805
fix: reject changing the unenforced primary key once set by @touch-of-grey in https://github.com/lance-format/lance/pull/6810
feat: support updating the unenforced clustering key via UpdateConfig by @touch-of-grey in https://github.com/lance-format/lance/pull/6812
feat: builder-style MemWAL initialization API by @touch-of-grey in https://github.com/lance-format/lance/pull/6815
feat(mem_wal): cache opened L0 flushed-generation datasets by @hamersaw in https://github.com/lance-format/lance/pull/6816
feat(java): add Java bindings for MemWAL APIs by @touch-of-grey in https://github.com/lance-format/lance/pull/6833
feat(rust): support update by _row_id by @yanghua in https://github.com/lance-format/lance/pull/6837
feat: expose granular trace event targets by @beinan in https://github.com/lance-format/lance/pull/6853
feat: add MemWAL sharding evaluator by @jackye1995 in https://github.com/lance-format/lance/pull/6854
feat: expose multi-base config to Python and Java write_fragments API by @zhangyue19921010 in https://github.com/lance-format/lance/pull/6855
feat(index): serializable cache for Bitmap and LabelList scalar indices by @wjones127 in https://github.com/lance-format/lance/pull/6874
feat: create materialized view API by @rpgreen in https://github.com/lance-format/lance/pull/6891

Bug Fixes 🐛

fix: set scope for junit-jupiter dependency to test by @yuqi1129 in https://github.com/lance-format/lance/pull/5576
fix: branch_identfier unstable for legacy branches by @majin1102 in https://github.com/lance-format/lance/pull/6390
fix: serialize version metadata through JNI and correct row-ID lookup by @ivscheianu in https://github.com/lance-format/lance/pull/6465
fix(blobV2): blob.from_uri optional validation by @zhangyue19921010 in https://github.com/lance-format/lance/pull/6558
fix: upgrade ethnum 1.5.2 to 1.5.3 and unpin nightly toolchain by @LuciferYang in https://github.com/lance-format/lance/pull/6578
fix: drop stale has_all_zeros guard so block bitpack engages on rep/def by @pengw0048 in https://github.com/lance-format/lance/pull/6629
fix(json): fix json object value fetch by integer by @dentiny in https://github.com/lance-format/lance/pull/6632
fix(java-jni): make dispatcher thread a daemon to allow jvm to exit by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/6633
fix(core): map DataFusion SchemaError to InvalidInput by @jja725 in https://github.com/lance-format/lance/pull/6639
fix: refresh last_updated metadata on Operation::Merge for rewritten fragments by @jerryjch in https://github.com/lance-format/lance/pull/6640
fix: make vector distance schema nullable by @BubbleCal in https://github.com/lance-format/lance/pull/6649
fix: propagate update_columns offsets and partial last_updated for RewriteColumns by @jerryjch in https://github.com/lance-format/lance/pull/6650
fix: json extract using wildcard by @zhangyue19921010 in https://github.com/lance-format/lance/pull/6651
fix(optimize): fix stale OCC read_version in distributed compaction to prevent row resurrection by @xiaguanglei in https://github.com/lance-format/lance/pull/6653
fix: add more info when describe indices error by @wojiaodoubao in https://github.com/lance-format/lance/pull/6665
fix: detect table metadata key conflicts in concurrent UpdateConfig by @jackye1995 in https://github.com/lance-format/lance/pull/6667
fix: always do wal tailer cursor update by @touch-of-grey in https://github.com/lance-format/lance/pull/6673
fix: add equals/hashCode to FullTextQuery concrete subclasses by @ivscheianu in https://github.com/lance-format/lance/pull/6674
fix(core): stop scalar-index training scans from over-projecting Map siblings by @kushudai in https://github.com/lance-format/lance/pull/6679
fix: swap dataset.rs warning argument order by @wkalt in https://github.com/lance-format/lance/pull/6683
fix(python): add backfill/refresh wrappers to namespace classes by @jackye1995 in https://github.com/lance-format/lance/pull/6684
fix: resolve system index types in describe_indices by @wkalt in https://github.com/lance-format/lance/pull/6685
fix: avoid nested tokio runtime panic in AWS credential vendor by @jackye1995 in https://github.com/lance-format/lance/pull/6689
fix: simplify column-map expressions before physical planning by @westonpace in https://github.com/lance-format/lance/pull/6698
fix(java): honor user-specified distance type in VectorTrainer by @jiaoew1991 in https://github.com/lance-format/lance/pull/6704
fix: sync unenforced_primary_key_position when field metadata updates by @touch-of-grey in https://github.com/lance-format/lance/pull/6706
fix: do no work when optimize_indices is called but no new data has arrived by @westonpace in https://github.com/lance-format/lance/pull/6711
fix(mem_wal): keep dispatcher and waiters alive on flush handler errors by @touch-of-grey in https://github.com/lance-format/lance/pull/6715
fix(encoding): support sparse boolean lists in full-zip encoding by @shenganzhang in https://github.com/lance-format/lance/pull/6723
fix(index): log vector training sampling progress by @hfutatzhanghb in https://github.com/lance-format/lance/pull/6724
fix: scalar index config plugin lookup to be case-insensitive by @zhangyue19921010 in https://github.com/lance-format/lance/pull/6739
fix(rust): avoid pushdown limit/offset for stable row id by @yanghua in https://github.com/lance-format/lance/pull/6740
fix: use physical scan stream for update by @wojiaodoubao in https://github.com/lance-format/lance/pull/6741
fix: use multipart-aware put for transaction file writes by @jackye1995 in https://github.com/lance-format/lance/pull/6750
fix(mem_wal): advance visibility cursor on WAL durability, not indexing by @hamersaw in https://github.com/lance-format/lance/pull/6754
fix: include view tag in FieldDataCacheKey by @wombatu-kun in https://github.com/lance-format/lance/pull/6758
fix(mem_wal): return _distance from LSM vector search across mixed sources by @hamersaw in https://github.com/lance-format/lance/pull/6761
fix(mem_wal): stop WAL replay from re-loading already-compacted entries by @hamersaw in https://github.com/lance-format/lance/pull/6767
fix: complex all-null list struct decoding by @Xuanwo in https://github.com/lance-format/lance/pull/6771
fix: set _row_created_at_version to new version for MERGE INTO INSERT rows by @jerryjch in https://github.com/lance-format/lance/pull/6774
docs: change a typo of WAL entry filename suffix to '.arrow' by @jiengup in https://github.com/lance-format/lance/pull/6786
fix(index): fix range query bound inclusive-ness by @dentiny in https://github.com/lance-format/lance/pull/6796
fix: stop double-counting child CPU in node-with-children Exec plans by @brendanclement in https://github.com/lance-format/lance/pull/6799
fix: decode constant list struct children by @Xuanwo in https://github.com/lance-format/lance/pull/6801
fix(mem_wal): keep sealed memtables queryable until flush commits by @hamersaw in https://github.com/lance-format/lance/pull/6814
fix(encoding): fail cleanly when blob data fails to load by @nsLance in https://github.com/lance-format/lance/pull/6817
fix: make HNSW graph build deterministic to stabilize test_ann_prefilter by @wombatu-kun in https://github.com/lance-format/lance/pull/6818
fix(python): pass batch size through fragment pandas export by @BubbleCal in https://github.com/lance-format/lance/pull/6829
fix(mem_wal): allow append-only tables without primary keys by @touch-of-grey in https://github.com/lance-format/lance/pull/6848
fix(mem_wal): dedupe duplicate primary keys in LSM point lookup by @touch-of-grey in https://github.com/lance-format/lance/pull/6880
fix(mem_wal): exact PK dedup for LSM vector search by @touch-of-grey in https://github.com/lance-format/lance/pull/6881

Documentation 📚

docs: update dataset source in llm_dataset_creation.md by @lhoestq in https://github.com/lance-format/lance/pull/5827
docs(mem_wal): warn about stalled-writer race when WAL GC is added by @hamersaw in https://github.com/lance-format/lance/pull/6699
docs: remove FROM community clause from DuckDB lance install command by @SiddiqueAhmad in https://github.com/lance-format/lance/pull/6727
docs: correct the suffix of wal file and unify the term "shard_id" by @jiengup in https://github.com/lance-format/lance/pull/6807
docs: document PR publishing requirements by @Xuanwo in https://github.com/lance-format/lance/pull/6870

Performance Improvements 🚀

perf: avoid materializing RoaringBitmap::full() in fragment allow-list by @wkalt in https://github.com/lance-format/lance/pull/6664
perf: reduce memory use when splitting IVF partitions by @wjones127 in https://github.com/lance-format/lance/pull/6687
perf(index): use SIMD-dispatched l2_u8 in SQDistCalculator by @martji in https://github.com/lance-format/lance/pull/6692
perf: use hnsw for memtable vector index by @touch-of-grey in https://github.com/lance-format/lance/pull/6701
perf: reduce IO requests from loading bitmap index by @wjones127 in https://github.com/lance-format/lance/pull/6703
perf: revert inline scheduling by @westonpace in https://github.com/lance-format/lance/pull/6709
perf: fix O(N·K) slow row-id lookup on stable-row-id datasets by @hamersaw in https://github.com/lance-format/lance/pull/6716

Other Changes

refactor(mem_wal): redesign FTS mem index for single-writer multi-reader by @touch-of-grey in https://github.com/lance-format/lance/pull/6726
refactor: rename ShardSpec to ShardingSpec by @touch-of-grey in https://github.com/lance-format/lance/pull/6813

Full Changelog: https://github.com/lance-format/lance/compare/release-root/7.0.0-beta.N...v7.0.0

Open source →

6.0.011 May 2026

What's Changed

Breaking Changes 🛠

refactor!: vendor the tokenizer stack into lance by @Xuanwo in https://github.com/lance-format/lance/pull/6512
perf!: run scheduler initialize eagerly in async read_tasks by @westonpace in https://github.com/lance-format/lance/pull/6710

New Features 🎉

feat: support segmented inverted index build and search by @Xuanwo in https://github.com/lance-format/lance/pull/6305
feat(dictionary-namespace): support table related operation by @zhangyue19921010 in https://github.com/lance-format/lance/pull/6308
feat: clean up transaction files on failed commits by @wjones127 in https://github.com/lance-format/lance/pull/6319
feat: add planned blob reads with source-level coalescing by @Xuanwo in https://github.com/lance-format/lance/pull/6352
refactor: use exact base-scoped store bindings by @Xuanwo in https://github.com/lance-format/lance/pull/6422
feat: wire batch_size_bytes to Python and public Rust API by @westonpace in https://github.com/lance-format/lance/pull/6428
feat(vector): add partition search parallelism by @BubbleCal in https://github.com/lance-format/lance/pull/6475
feat(index): support float16 and float64 in IVF_FLAT by @BubbleCal in https://github.com/lance-format/lance/pull/6476
feat: batch chopping fallback for filtered read by @westonpace in https://github.com/lance-format/lance/pull/6482
feat(java): add Dataset.sample() API by @beinan in https://github.com/lance-format/lance/pull/6500
feat: add ANN proto codecs and extract table_identifier module by @LuQQiu in https://github.com/lance-format/lance/pull/6503
feat: add configurable blob v2 pack file size by @hamersaw in https://github.com/lance-format/lance/pull/6508
feat: expose has_stable_row_ids property on LanceDataset by @pengw0048 in https://github.com/lance-format/lance/pull/6531
feat: expose base scoped store bindings to python by @zhangyue19921010 in https://github.com/lance-format/lance/pull/6547
feat: support zonemap index segments by @beinan in https://github.com/lance-format/lance/pull/6593
feat: update lance-namespace to 0.7.2 and align namespace declared table lifecycle by @jackye1995 in https://github.com/lance-format/lance/pull/6608
feat: generalize dynamic object store credentials by @jackye1995 in https://github.com/lance-format/lance/pull/6609
feat: add ANNIvfPartitionExecProto by @LuQQiu in https://github.com/lance-format/lance/pull/6612
feat: add prefilter_type to ANNIvfSubIndexExecProto by @LuQQiu in https://github.com/lance-format/lance/pull/6613
feat: replace Azure SDK and google-cloud-auth with direct reqwest for credential vending by @jackye1995 in https://github.com/lance-format/lance/pull/6617
feat(io): bypass backpressure for io_buffer_size=0 and 2.0 indirect I/O by @westonpace in https://github.com/lance-format/lance/pull/6627

Bug Fixes 🐛

fix: warn and clamp LANCE_INITIAL_UPLOAD_SIZE instead of panicking by @LuciferYang in https://github.com/lance-format/lance/pull/6389
fix: keep delete-by-source fast path with scalar indexes by @Xuanwo in https://github.com/lance-format/lance/pull/6435
fix: include column_metadatas and column_infos in CachedFileMetadata::DeepSizeOf by @jiaoew1991 in https://github.com/lance-format/lance/pull/6480
fix(index): preserve fts prewarm position codec by @BubbleCal in https://github.com/lance-format/lance/pull/6485
fix: handle FlatBin quantization in optimize_vector_indices_v2 by @jackye1995 in https://github.com/lance-format/lance/pull/6488
fix: use logical OR instead of bitwise OR in conflict resolver by @dentiny in https://github.com/lance-format/lance/pull/6492
fix: add dir_listing_to_manifest_migration_enabled flag to avoid extra object store calls by @jackye1995 in https://github.com/lance-format/lance/pull/6507
fix: prevent arithmetic overflow in U64Segment encoding selection for sparse/extreme row id ranges by @ivscheianu in https://github.com/lance-format/lance/pull/6516
fix: bump jieba-rs to 0.9.0 to fix build-no-lock CI by @westonpace in https://github.com/lance-format/lance/pull/6518
fix: blob projection schema compatibility by @Xuanwo in https://github.com/lance-format/lance/pull/6521
fix(namespace): serialize manifest mutations by @Xuanwo in https://github.com/lance-format/lance/pull/6525
fix: missing bumpversion entry for lance-tokenizer by @Xuanwo in https://github.com/lance-format/lance/pull/6526
fix: scale default memory pool size by partition count by @westonpace in https://github.com/lance-format/lance/pull/6562
fix: apply fragment bitmap allow-list to index search results by @westonpace in https://github.com/lance-format/lance/pull/6563
fix: hard cap batch size in merge_insert to prevent sort failures by @westonpace in https://github.com/lance-format/lance/pull/6564
fix: index type try_from miss RTree and BLOOMFILTER by @wojiaodoubao in https://github.com/lance-format/lance/pull/6568
fix(namespace): align error handling with namespace spec by @jackye1995 in https://github.com/lance-format/lance/pull/6575
fix: reject Rewrite vs CreateIndex when FRI groups straddle bitmap by @wjones127 in https://github.com/lance-format/lance/pull/6610
fix(json): detect float64-stored numbers in json type extraction by @dentiny in https://github.com/lance-format/lance/pull/6622
fix: respect LANCE_DEFAULT_IO_BUFFER_SIZE if it has been set by @westonpace in https://github.com/lance-format/lance/pull/6636
fix: make vector distance schema nullable by @BubbleCal in https://github.com/lance-format/lance/pull/6649

Documentation 📚

docs: tighten python environment workflow guidance by @Xuanwo in https://github.com/lance-format/lance/pull/6520
docs: fix broken intra-doc link in DatasetPreFilter by @LuciferYang in https://github.com/lance-format/lance/pull/6579
docs: correct repetition level example in encoding docs by @BubbleCal in https://github.com/lance-format/lance/pull/6585

Performance Improvements 🚀

perf: intern DataFile fields/column_indices to reduce manifest memory by @beinan in https://github.com/lance-format/lance/pull/6477
perf: intern RowDatasetVersionMeta inline bytes to reduce manifest memory by @beinan in https://github.com/lance-format/lance/pull/6499
perf: add SIMD-accelerated u8 dot product for SQ distance by @justinrmiller in https://github.com/lance-format/lance/pull/6506
perf: add SIMD kernels for bf16 distance functions by @justinrmiller in https://github.com/lance-format/lance/pull/6510
perf: submit I/O requests eagerly in FullZipScheduler by @hushengquan in https://github.com/lance-format/lance/pull/6513
perf: add SIMD-accelerated u8 L2 and cosine distance kernels by @justinrmiller in https://github.com/lance-format/lance/pull/6517
perf: speed up RaBitQ 4-bit LUT distance on ARM by 16x by @justinrmiller in https://github.com/lance-format/lance/pull/6537
perf: add explicit SIMD types and distance kernels for f64 by @justinrmiller in https://github.com/lance-format/lance/pull/6540
perf: don't spawn the scheduling on a separate thread for small reads by @westonpace in https://github.com/lance-format/lance/pull/6637
perf: avoid materializing RoaringBitmap::full() in fragment allow-list by @wkalt in https://github.com/lance-format/lance/pull/6664
perf: revert inline scheduling by @westonpace in https://github.com/lance-format/lance/pull/6709

Full Changelog: https://github.com/lance-format/lance/compare/release-root/6.0.0-beta.N...v6.0.0

Open source →

4.0.030 Mar 2026

What's Changed

Breaking Changes 🛠

feat!: upgrade DataFusion dependency to 52.1.0 by @wjones127 in https://github.com/lance-format/lance/pull/6015
refactor!: refactor java access to file format version by @jackye1995 in https://github.com/lance-format/lance/pull/6053
refactor!: remove create_empty_table usage by @jackye1995 in https://github.com/lance-format/lance/pull/6087
fix!: bump IVF_RQ version for compatibility check by @BubbleCal in https://github.com/lance-format/lance/pull/6097
perf(inverted)!: reduce fts indexing time and memory by @BubbleCal in https://github.com/lance-format/lance/pull/6174
feat: add index segment commit API by @Xuanwo in https://github.com/lance-format/lance/pull/6209
refactor!: remove staging from distributed vector indexing by @Xuanwo in https://github.com/lance-format/lance/pull/6269

New Features 🎉

feat: compress complex all null by @yingjianwu98 in https://github.com/lance-format/lance/pull/4990
feat: expose use_scalar_index param in Java scanner by @xloya in https://github.com/lance-format/lance/pull/5487
feat: add file list with sizes to IndexMetadata by @wjones127 in https://github.com/lance-format/lance/pull/5497
feat(compaction): add Python config for defer_index_remap by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5691
feat(core): add Levenshtein-based suggestions to not-found errors in schema by @HemantSudarshan in https://github.com/lance-format/lance/pull/5976
feat: add URI-based commit support to Java SDK by @hamersaw in https://github.com/lance-format/lance/pull/5978
fix: concurrent read and write to directory namespace by @jackye1995 in https://github.com/lance-format/lance/pull/5983
feat: add ability to pass custom headers to objectstore requests by @hamersaw in https://github.com/lance-format/lance/pull/5989
feat: add DeleteResult with num_deleted_rows by @wkalt in https://github.com/lance-format/lance/pull/6001
feat: introduce IncompatibleTransaction error by @wjones127 in https://github.com/lance-format/lance/pull/6003
feat(cleanup): add more metrics to RemovalStats by @zhangyue19921010 in https://github.com/lance-format/lance/pull/6025
feat(java): expose prefilter parameter to support vector search with fragments by @nyl3532016 in https://github.com/lance-format/lance/pull/6040
feat: surface ambiguous merge insert error as InvalidInput by @wjones127 in https://github.com/lance-format/lance/pull/6048
feat(blob): distribute blob sidecar keys with reversed binary ids by @Xuanwo in https://github.com/lance-format/lance/pull/6060
feat: handle JSONB literals in Lance SQL planner by @wkalt in https://github.com/lance-format/lance/pull/6061
feat(java): expose Dataset.dropIndex method to drop specific index by @fangbo in https://github.com/lance-format/lance/pull/6065
feat(blob): map external blob URIs to multi-base base ids by @Xuanwo in https://github.com/lance-format/lance/pull/6066
feat: add env toggle for repetition index cache on read by @Xuanwo in https://github.com/lance-format/lance/pull/6069
feat(compaction): single reserve_fragment_ids after rewriting files by @hamersaw in https://github.com/lance-format/lance/pull/6072
feat: expose compaction binary copy configuration through python and java SDKs by @hamersaw in https://github.com/lance-format/lance/pull/6074
feat(cleanup): support rate limiter for cleanup operation by @zhangyue19921010 in https://github.com/lance-format/lance/pull/6084
feat: mark 2.2 as stable and add 2.3 as the next file format version by @Xuanwo in https://github.com/lance-format/lance/pull/6088
feat: support prewarm for IVF-based ANN indices by @wjones127 in https://github.com/lance-format/lance/pull/6090
feat: add skip_transpose flag to vector index builders by @BubbleCal in https://github.com/lance-format/lance/pull/6114
feat: enable HNSW-accelerated partition assignment for fp16 vectors by @wkalt in https://github.com/lance-format/lance/pull/6119
feat: clearer progress reporting for IVF by @wkalt in https://github.com/lance-format/lance/pull/6126
feat: support vector indices in describe_indices filtering by @ndpvt-web in https://github.com/lance-format/lance/pull/6145
feat: reduce open file handles during IVF training by @westonpace in https://github.com/lance-format/lance/pull/6169
feat: add compaction options in manifest config by @hamersaw in https://github.com/lance-format/lance/pull/6170
feat: support atomic multi-table transactions via namespace manifest by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/6173
feat: add abfss:// scheme support for Azure ADLS Gen2 by @burlacio in https://github.com/lance-format/lance/pull/6192
feat: bounding source fragments for compaction execution by @hamersaw in https://github.com/lance-format/lance/pull/6232
fix: filter out detached versions when scanning manifests by @jackye1995 in https://github.com/lance-format/lance/pull/6245
feat: allow setting transaction properties in various operations by @jackye1995 in https://github.com/lance-format/lance/pull/6246
feat: add OpenDAL Azdls backend for abfss:// with use_opendal flag by @burlacio in https://github.com/lance-format/lance/pull/6256

Bug Fixes 🐛

fix(java): transaction fatal bug in java transaction api by @wojiaodoubao in https://github.com/lance-format/lance/pull/5824
fix: maintaining individual fragment operation when calling take_source by @hamersaw in https://github.com/lance-format/lance/pull/5844
fix(encoding): handle empty rows in variable packed struct decode by @Xuanwo in https://github.com/lance-format/lance/pull/5995
fix: various bugs to namespace access by @jackye1995 in https://github.com/lance-format/lance/pull/5996
fix: set namespace commit handler for LanceDataset.commit by @jackye1995 in https://github.com/lance-format/lance/pull/6002
fix: fast_search limits full text search to indexed fragments by @BubbleCal in https://github.com/lance-format/lance/pull/6006
fix: fast_search should ignore any unindexed data for vector search by @BubbleCal in https://github.com/lance-format/lance/pull/6007
fix: correctly calculate max visible level when a list has no def by @westonpace in https://github.com/lance-format/lance/pull/6008
perf: avoid oversized variable buffers in full-zip scan batches by @Xuanwo in https://github.com/lance-format/lance/pull/6013
fix: make overwrites retryable instead of compatible by @jackye1995 in https://github.com/lance-format/lance/pull/6014
fix(python): avoid interpreter shutdown panic in BackgroundExecutor by @Xuanwo in https://github.com/lance-format/lance/pull/6023
fix: filter stale row IDs in TakeExec for FTS/vector after delete by @wkalt in https://github.com/lance-format/lance/pull/6042
fix(btree): include null pages in non-IsNull queries for correct thre… by @wkalt in https://github.com/lance-format/lance/pull/6043
fix: handle list-level NULLs in NOT filters by @fenfeng9 in https://github.com/lance-format/lance/pull/6044
fix: allowing headers for static configuration to be consistent by @hamersaw in https://github.com/lance-format/lance/pull/6045
fix: bitmap iterator exhaustion in mask_to_offset_ranges by @wkalt in https://github.com/lance-format/lance/pull/6046
fix(build): add Android aarch64 support to lance-linalg by @dardourimohamed in https://github.com/lance-format/lance/pull/6057
fix: make blob v2 reads base-aware in multi-base datasets by @Xuanwo in https://github.com/lance-format/lance/pull/6064
fix(lance-linalg): fix missing return value in u8x16::bit_and for non-x86_64/aarch64 targets by @cheungxi in https://github.com/lance-format/lance/pull/6068
fix: resolve Python lint failure on main by @Xuanwo in https://github.com/lance-format/lance/pull/6073
fix: restore main CI by formatting take_blob imports by @Xuanwo in https://github.com/lance-format/lance/pull/6082
fix: incorrect deletion masking in DatasetPreFilter by @cijiugechu in https://github.com/lance-format/lance/pull/6083
fix: avoid thread pool contention between compression and write operations during FTS indexing by @BubbleCal in https://github.com/lance-format/lance/pull/6085
fix: compile error for err_express by @zhangyue19921010 in https://github.com/lance-format/lance/pull/6094
fix(python): crash when schema contains nested fixed_size_list or extension type by @erandagan in https://github.com/lance-format/lance/pull/6107
fix: dont sample if no vectors are needed by @westonpace in https://github.com/lance-format/lance/pull/6110
fix(index): preserve stable row-id entries during scalar index optimize by @acking-you in https://github.com/lance-format/lance/pull/6117
fix: disallow wrapping auto-detected fsst in other compression by @hamersaw in https://github.com/lance-format/lance/pull/6120
fix: pin substrait to 0.62.2 until DF supports 0.62.3 by @westonpace in https://github.com/lance-format/lance/pull/6121
fix: vector index type shown as unknown in describe_indices by @jackye1995 in https://github.com/lance-format/lance/pull/6122
fix: handle inverted index worker exits during dispatch by @BubbleCal in https://github.com/lance-format/lance/pull/6129
fix: add missing type hint for producer function by @Gallardot in https://github.com/lance-format/lance/pull/6133
fix: prevent duplicate manifest entries from concurrent table creation by @jmhsieh in https://github.com/lance-format/lance/pull/6143
fix: replace fetch_arrow_table with to_arrow_table by @BubbleCal in https://github.com/lance-format/lance/pull/6146
fix: preserve merge insert delete-by-source semantics by @Xuanwo in https://github.com/lance-format/lance/pull/6148
fix: handle DataType::Null in adjust_child_validity to prevent panic by @wjones127 in https://github.com/lance-format/lance/pull/6160
fix: persist frag reuse index external file on local filesystem by @wjones127 in https://github.com/lance-format/lance/pull/6163
fix: avoid empty range reads for zero-length blobs by @Xuanwo in https://github.com/lance-format/lance/pull/6168
fix: handle nullable validity layers without def levels by @Xuanwo in https://github.com/lance-format/lance/pull/6187
fix: like queries with a prefix should be accelerated by btree and zonemap by @jackye1995 in https://github.com/lance-format/lance/pull/6188
fix: use to_arrow_reader in benchmark datagen by @Xuanwo in https://github.com/lance-format/lance/pull/6190
fix: disallowing stale credentials from directory namespace by @hamersaw in https://github.com/lance-format/lance/pull/6194
fix: memory_limit and num_workers params are not passed to index worker by @BubbleCal in https://github.com/lance-format/lance/pull/6197
fix: preserve create index transaction semantics by @Xuanwo in https://github.com/lance-format/lance/pull/6204
fix: allow same field name with different type in dataset overwrites by @hamersaw in https://github.com/lance-format/lance/pull/6206
fix: prewarm all segments for named indices by @Xuanwo in https://github.com/lance-format/lance/pull/6211
fix: respect the old data filter on inverted index by @westonpace in https://github.com/lance-format/lance/pull/6216
fix: 2.1/2.2 panic when a list column had small values and many empty values by @westonpace in https://github.com/lance-format/lance/pull/6234
fix: resolve_latest_location converts errors to not_found unconditionally by @wkalt in https://github.com/lance-format/lance/pull/6248
fix: return errors for unsupported fixed-size-list child types by @myandpr in https://github.com/lance-format/lance/pull/6253
fix: adding namespace support to java SDK CommitBuilder from dataset by @hamersaw in https://github.com/lance-format/lance/pull/6257
fix: pass dataset_options to SafeLanceDataset in worker processes by @eddyxu in https://github.com/lance-format/lance/pull/6278

Documentation 📚

docs: fix incorrect URLs and cleanup by @prrao87 in https://github.com/lance-format/lance/pull/5317
docs: expand the FTS index doc explaining the training process and multiple partitions by @westonpace in https://github.com/lance-format/lance/pull/5988
docs: clarify v2.2 nested drop rollback risk by @Xuanwo in https://github.com/lance-format/lance/pull/5999
docs: require data_storage_version=2.2 in map type example by @Xuanwo in https://github.com/lance-format/lance/pull/6032
docs: update file versioning matrix for 2.2 rollout by @Xuanwo in https://github.com/lance-format/lance/pull/6033
docs: reorganize blob docs around blob v2 and clarify legacy compatibility by @Xuanwo in https://github.com/lance-format/lance/pull/6034
docs: align 2.2 encoding docs and nested add-column notes by @Xuanwo in https://github.com/lance-format/lance/pull/6038
docs: clarify how to generate TPCH benchmark dataset locally by @Xuanwo in https://github.com/lance-format/lance/pull/6063
docs: document vector index RAM (training) & storage requirements by @westonpace in https://github.com/lance-format/lance/pull/6108
docs: update index.md to fix indexes to indices for uniformity by @wombatu-kun in https://github.com/lance-format/lance/pull/6113
docs: document the rules for transaction conflicts by @westonpace in https://github.com/lance-format/lance/pull/6158
docs: add alicloud oss configuration by @FarmerChillax in https://github.com/lance-format/lance/pull/6167
docs: update the rules for data replacement conflicts to reflect reality by @westonpace in https://github.com/lance-format/lance/pull/6182
docs: add example to show how to index JSON column by @prrao87 in https://github.com/lance-format/lance/pull/6208
docs: remove legacy preview index note by @Xuanwo in https://github.com/lance-format/lance/pull/6218

Performance Improvements 🚀

perf: pre-transpose PQ codebook for SIMD-friendly L2 distance by @wkalt in https://github.com/lance-format/lance/pull/5923
perf: speed up format v2.2 scans by adding shortcut for full page by @Xuanwo in https://github.com/lance-format/lance/pull/5981
perf: speed up format 2.2 300% by spawning structural decode batch tasks by @Xuanwo in https://github.com/lance-format/lance/pull/5982
perf: reduce peak memory during cosine IVF-PQ index training by @wkalt in https://github.com/lance-format/lance/pull/6016
perf: fast rotation for RQ quantization by @BubbleCal in https://github.com/lance-format/lance/pull/6024
perf: avoid re-open shard indices and small reads by @BubbleCal in https://github.com/lance-format/lance/pull/6026
perf: disable auto FSST for binary fields by @Xuanwo in https://github.com/lance-format/lance/pull/6047
perf: speedup flat fts by @westonpace in https://github.com/lance-format/lance/pull/6054
perf: add dict-values compression controls with lz4 default by @Xuanwo in https://github.com/lance-format/lance/pull/6059
perf: avoid frequent allocating when computing residual vectors by @BubbleCal in https://github.com/lance-format/lance/pull/6062
perf: add take_blob benchmark with cache_repetition_index matrix by @Xuanwo in https://github.com/lance-format/lance/pull/6067
perf: parallelize FTS prewarming by @BubbleCal in https://github.com/lance-format/lance/pull/6144
perf: remove shard content key sorting from distributed merge by @Xuanwo in https://github.com/lance-format/lance/pull/6179
perf(inverted): reuse posting batch builder and merge tail partitions by @BubbleCal in https://github.com/lance-format/lance/pull/6191
perf: reuse distance calculator at selecting candidates by @BubbleCal in https://github.com/lance-format/lance/pull/6202
perf: new layout for positions and new algo for phrase query by @BubbleCal in https://github.com/lance-format/lance/pull/6203
perf: batched WAND and new WAND structure, ~50% faster by @BubbleCal in https://github.com/lance-format/lance/pull/6241

Other Changes

refactor: use dict entries and encoded size instead of cardinality for dict decision by @Xuanwo in https://github.com/lance-format/lance/pull/5891
refactor: upgrade to SNAFU 0.9 by @shepmaster in https://github.com/lance-format/lance/pull/6071
refactor: overhaul AGENTS.md with PR review insights by @Xuanwo in https://github.com/lance-format/lance/pull/6103
refactor: use the dataset file version to determine index file version by @westonpace in https://github.com/lance-format/lance/pull/6142
refactor: rename arrow-scalar to lance-arrow-scalar by @westonpace in https://github.com/lance-format/lance/pull/6199
refactor: distributed vector segment build by @Xuanwo in https://github.com/lance-format/lance/pull/6220

Full Changelog: https://github.com/lance-format/lance/compare/release-root/4.0.0-beta.N...v4.0.0

Open source →

3.0.013 Mar 2026Deprecated

refactor: deprecate list_indices and migrate tests to describe_indices by @fenfeng9 in https://github.com/lance-format/lance/pull/5945

What's Changed

Breaking Changes 🛠

feat!: support index progress reporting via callbacks by @wkalt in https://github.com/lance-format/lance/pull/5910
perf!: remove shuffle buffer by @wkalt in https://github.com/lance-format/lance/pull/5912
feat!: upgrade DataFusion dependency to 52.1.0 by @wjones127 in https://github.com/lance-format/lance/pull/6015
refactor!: refactor java access to file format version by @jackye1995 in https://github.com/lance-format/lance/pull/6053
refactor!: remove create_empty_table usage by @jackye1995 in https://github.com/lance-format/lance/pull/6087
fix!: bump IVF_RQ version for compatibility check by @BubbleCal in https://github.com/lance-format/lance/pull/6097

Critical Fixes ‼️

fix: deduplicate row addresses in take to prevent panic by @wjones127 in https://github.com/lance-format/lance/pull/5881
fix: fts flat search drops rows when avg_doc_length < 1.0 by @wjones127 in https://github.com/lance-format/lance/pull/5897
fix: invalidate index fragment bitmaps after data replacement and stale merge by @wjones127 in https://github.com/lance-format/lance/pull/5929

New Features 🎉

feat: add RLE support for block by @yingjianwu98 in https://github.com/lance-format/lance/pull/4937
feat: compress complex all null by @yingjianwu98 in https://github.com/lance-format/lance/pull/4990
feat: support cleanup across branches by @majin1102 in https://github.com/lance-format/lance/pull/5009
feat: dictionary index always32 bits by @yingjianwu98 in https://github.com/lance-format/lance/pull/5011
feat: abort dictionary encode if not useful by @yingjianwu98 in https://github.com/lance-format/lance/pull/5055
feat(cdf): cdf support upsert for views by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5369
feat(compaction): binary copy capability for compaction by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5434
feat: expose use_scalar_index param in Java scanner by @xloya in https://github.com/lance-format/lance/pull/5487
feat(python): expose search_filter in scanner by @wojiaodoubao in https://github.com/lance-format/lance/pull/5506
feat: add alter column nullable to non-nullable support by @Xuanwo in https://github.com/lance-format/lance/pull/5589
feat: evolute all_null_layout to constant layout by @Xuanwo in https://github.com/lance-format/lance/pull/5641
feat(java): support creating IVF_RQ index by @majin1102 in https://github.com/lance-format/lance/pull/5648
feat(java): support building vector index distributively by @majin1102 in https://github.com/lance-format/lance/pull/5664
feat(rust): add datafusion catalog_provider through namespace by @majin1102 in https://github.com/lance-format/lance/pull/5686
feat: support List and Struct type for KeyValue in inserted_rows.rs by @wojiaodoubao in https://github.com/lance-format/lance/pull/5713
feat: support tencent cos by @ztorchan in https://github.com/lance-format/lance/pull/5740
feat: add Lance-HF docs to lance.org/integrations/huggingface/ by @prrao87 in https://github.com/lance-format/lance/pull/5748
feat(python): support namespace for tensorflow by @yuqi1129 in https://github.com/lance-format/lance/pull/5750
feat: add range to External blob by @wojiaodoubao in https://github.com/lance-format/lance/pull/5765
feat(java): support json extraction by scanning by @majin1102 in https://github.com/lance-format/lance/pull/5770
feat: introduce RowIdSet and RowIdMask by @yanghua in https://github.com/lance-format/lance/pull/5771
feat: expose blob handling APIs to python by @Xuanwo in https://github.com/lance-format/lance/pull/5790
feat: add blob handling support for fragment by @Xuanwo in https://github.com/lance-format/lance/pull/5801
feat: add plan/execute separation to FilteredReadExec by @LuQQiu in https://github.com/lance-format/lance/pull/5843
feat: add LSM scanner with point lookup and vector search support by @touch-of-grey in https://github.com/lance-format/lance/pull/5850
feat: add rename table implementations to REST namespaces by @bryanck in https://github.com/lance-format/lance/pull/5874
feat(python): expose enable_stable_row_ids in commit() by @fecet in https://github.com/lance-format/lance/pull/5908
feat: support aggregate in scanner by @jackye1995 in https://github.com/lance-format/lance/pull/5911
feat: spill page metadata to disk during IVF shuffle by @wkalt in https://github.com/lance-format/lance/pull/5921
feat: add third party licenses lists by @jackye1995 in https://github.com/lance-format/lance/pull/5922
feat(java): support session by @jackye1995 in https://github.com/lance-format/lance/pull/5931
feat: make geodatafusion/geoarrow optional via geo feature flag by @apoc in https://github.com/lance-format/lance/pull/5934
perf: create local writer for efficient local writes by @wkalt in https://github.com/lance-format/lance/pull/5939
feat: add python and java binding for aggregate by @jackye1995 in https://github.com/lance-format/lance/pull/5951
feat: add proto serialization for FilteredReadExec by @LuQQiu in https://github.com/lance-format/lance/pull/5954
feat: create an arrow-scalar crate utilizing arrow-row and arrow-data by @westonpace in https://github.com/lance-format/lance/pull/5955
feat: add progress monitoring via callbacks for inverted indexes by @vivek-bharathan in https://github.com/lance-format/lance/pull/5958
feat: add size to object store tracing by @wjones127 in https://github.com/lance-format/lance/pull/5962
feat: update minimum supported rust version from 1.88 to 1.91 by @westonpace in https://github.com/lance-format/lance/pull/5964
feat: add Dataset::with_object_store for request-scoped store overrides by @wkalt in https://github.com/lance-format/lance/pull/5966
feat: support namespace as external manifest store by @jackye1995 in https://github.com/lance-format/lance/pull/5968
feat: serialize storage options in table identifier proto by @LuQQiu in https://github.com/lance-format/lance/pull/5973
feat(core): add Levenshtein-based suggestions to not-found errors in schema by @HemantSudarshan in https://github.com/lance-format/lance/pull/5976
feat: add URI-based commit support to Java SDK by @hamersaw in https://github.com/lance-format/lance/pull/5978
fix: concurrent read and write to directory namespace by @jackye1995 in https://github.com/lance-format/lance/pull/5983
feat: add ability to pass custom headers to objectstore requests by @hamersaw in https://github.com/lance-format/lance/pull/5989
feat: add DeleteResult with num_deleted_rows by @wkalt in https://github.com/lance-format/lance/pull/6001
feat: introduce IncompatibleTransaction error by @wjones127 in https://github.com/lance-format/lance/pull/6003
feat: surface ambiguous merge insert error as InvalidInput by @wjones127 in https://github.com/lance-format/lance/pull/6048
feat(blob): distribute blob sidecar keys with reversed binary ids by @Xuanwo in https://github.com/lance-format/lance/pull/6060
feat: handle JSONB literals in Lance SQL planner by @wkalt in https://github.com/lance-format/lance/pull/6061
feat(java): expose Dataset.dropIndex method to drop specific index by @fangbo in https://github.com/lance-format/lance/pull/6065
feat(blob): map external blob URIs to multi-base base ids by @Xuanwo in https://github.com/lance-format/lance/pull/6066
feat: add env toggle for repetition index cache on read by @Xuanwo in https://github.com/lance-format/lance/pull/6069
feat(compaction): single reserve_fragment_ids after rewriting files by @hamersaw in https://github.com/lance-format/lance/pull/6072
feat: expose compaction binary copy configuration through python and java SDKs by @hamersaw in https://github.com/lance-format/lance/pull/6074
feat: mark 2.2 as stable and add 2.3 as the next file format version by @Xuanwo in https://github.com/lance-format/lance/pull/6088
feat: support prewarm for IVF-based ANN indices by @wjones127 in https://github.com/lance-format/lance/pull/6090

Bug Fixes 🐛

fix: ensure blob encoding work when using file reader directly by @rahil-c in https://github.com/lance-format/lance/pull/5193
fix: support system columns in dataset.take* operations by @hamersaw in https://github.com/lance-format/lance/pull/5722
fix: skip missing indices in compaction rewrite by @AndreaBozzo in https://github.com/lance-format/lance/pull/5739
fix(lance-linalg): check fp16kernels feature before arch-specific code by @durch in https://github.com/lance-format/lance/pull/5747
refactor: align blob behavior that write via file format version, read via layout by @Xuanwo in https://github.com/lance-format/lance/pull/5752
fix: fix deletion when using file-object-store:// by @cmccabe in https://github.com/lance-format/lance/pull/5760
fix: remove unreasonable nullable check for data types in hash_joiner during merge operation by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5784
fix: allow unused_unsafe for __cpuid to support both stable and nightly by @jackye1995 in https://github.com/lance-format/lance/pull/5793
fix: set JUnit dependency as test scope by @bryanck in https://github.com/lance-format/lance/pull/5815
fix(java): transaction fatal bug in java transaction api by @wojiaodoubao in https://github.com/lance-format/lance/pull/5824
fix: fix remap so that it handles deletions correctly by @westonpace in https://github.com/lance-format/lance/pull/5828
fix: inconsistent transposed pq code and metadata when build ivf_pq index distributedly by @yanghua in https://github.com/lance-format/lance/pull/5834
fix: improve error messages in FixedSizeListArrayExt::convert_to_floating_point by @LuciferYang in https://github.com/lance-format/lance/pull/5836
fix(java): panic when reading CreateIndex transaction by @majin1102 in https://github.com/lance-format/lance/pull/5853
fix: fix mini-block dictionary bitpacking panic by @Xuanwo in https://github.com/lance-format/lance/pull/5860
fix: fix boolean inline constant decoding by @Xuanwo in https://github.com/lance-format/lance/pull/5862
fix: handle NULL elements in LABEL_LIST index results and explain_plan by @fenfeng9 in https://github.com/lance-format/lance/pull/5867
fix: open additional storage options provider related apis in lance dataset by @jackye1995 in https://github.com/lance-format/lance/pull/5869
fix: flaky test test_ann_prefilter for HNSW by @BubbleCal in https://github.com/lance-format/lance/pull/5870
fix(java): init allocator for new dataset when checkout branch/tag by @fangbo in https://github.com/lance-format/lance/pull/5876
fix: avoid panic when repdef serializes empty offsets by @fenfeng9 in https://github.com/lance-format/lance/pull/5890
fix(java): align version type from i32 to u64 by @majin1102 in https://github.com/lance-format/lance/pull/5892
fix: avoid bitmap range panic on inverted bounds by @fenfeng9 in https://github.com/lance-format/lance/pull/5893
fix: split index_statistics to reduce rustc query depth by @Xuanwo in https://github.com/lance-format/lance/pull/5894
fix: avoid panic on empty list LABEL_LIST filters by @fenfeng9 in https://github.com/lance-format/lance/pull/5914
fix: correct OR null semantics for nullable masks by @fenfeng9 in https://github.com/lance-format/lance/pull/5919
fix: respect fragment restrictions in vector and FTS searches when requested fragments by @yingjianwu98 in https://github.com/lance-format/lance/pull/5924
fix: don't drop field metadata on merge insert path by @westonpace in https://github.com/lance-format/lance/pull/5927
fix: apply SchemaAdapter in Updater by @westonpace in https://github.com/lance-format/lance/pull/5928
fix: correct OR null handling for BlockList|BlockList by @fenfeng9 in https://github.com/lance-format/lance/pull/5944
fix: remove unnecessary column projection for count aggregate by @jackye1995 in https://github.com/lance-format/lance/pull/5950
fix: respect requested indexed fragment in vector and FTS searches by @yingjianwu98 in https://github.com/lance-format/lance/pull/5953
fix: spawn part load in fts training by @westonpace in https://github.com/lance-format/lance/pull/5977
fix: rest namespace integration with table version apis by @jackye1995 in https://github.com/lance-format/lance/pull/5980
fix(encoding): handle empty rows in variable packed struct decode by @Xuanwo in https://github.com/lance-format/lance/pull/5995
fix: various bugs to namespace access by @jackye1995 in https://github.com/lance-format/lance/pull/5996
fix: set namespace commit handler for LanceDataset.commit by @jackye1995 in https://github.com/lance-format/lance/pull/6002
fix: fast_search limits full text search to indexed fragments by @BubbleCal in https://github.com/lance-format/lance/pull/6006
fix: fast_search should ignore any unindexed data for vector search by @BubbleCal in https://github.com/lance-format/lance/pull/6007
fix: correctly calculate max visible level when a list has no def by @westonpace in https://github.com/lance-format/lance/pull/6008
perf: avoid oversized variable buffers in full-zip scan batches by @Xuanwo in https://github.com/lance-format/lance/pull/6013
fix: make overwrites retryable instead of compatible by @jackye1995 in https://github.com/lance-format/lance/pull/6014
fix(python): avoid interpreter shutdown panic in BackgroundExecutor by @Xuanwo in https://github.com/lance-format/lance/pull/6023
fix(btree): include null pages in non-IsNull queries for correct thre… by @wkalt in https://github.com/lance-format/lance/pull/6043
fix: allowing headers for static configuration to be consistent by @hamersaw in https://github.com/lance-format/lance/pull/6045
fix(build): add Android aarch64 support to lance-linalg by @dardourimohamed in https://github.com/lance-format/lance/pull/6057
fix: make blob v2 reads base-aware in multi-base datasets by @Xuanwo in https://github.com/lance-format/lance/pull/6064
fix(lance-linalg): fix missing return value in u8x16::bit_and for non-x86_64/aarch64 targets by @cheungxi in https://github.com/lance-format/lance/pull/6068
fix: resolve Python lint failure on main by @Xuanwo in https://github.com/lance-format/lance/pull/6073
fix: restore main CI by formatting take_blob imports by @Xuanwo in https://github.com/lance-format/lance/pull/6082
fix: incorrect deletion masking in DatasetPreFilter by @cijiugechu in https://github.com/lance-format/lance/pull/6083
fix: avoid thread pool contention between compression and write operations during FTS indexing by @BubbleCal in https://github.com/lance-format/lance/pull/6085
fix: compile error for err_express by @zhangyue19921010 in https://github.com/lance-format/lance/pull/6094

Documentation 📚

docs: add docs for branch by @majin1102 in https://github.com/lance-format/lance/pull/5104
docs: fix issues in HF integration docs by @prrao87 in https://github.com/lance-format/lance/pull/5778
docs: fix MkDocs protobuf reference for ConstantLayout by @Xuanwo in https://github.com/lance-format/lance/pull/5833
docs(governance): introduce incubating subproject concept and update subproject list by @jackye1995 in https://github.com/lance-format/lance/pull/5847
docs: add lance skills as user guide by @Xuanwo in https://github.com/lance-format/lance/pull/5877
docs: add array type support by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/5884
docs: add FTS docs by @prrao87 in https://github.com/lance-format/lance/pull/5888
docs: add schema data types and field IDs documentation by @dik654 in https://github.com/lance-format/lance/pull/5925
docs: add lance-trino doc by @jackye1995 in https://github.com/lance-format/lance/pull/5943
docs: add new maintainers by @prrao87 in https://github.com/lance-format/lance/pull/5959
docs: expand the FTS index doc explaining the training process and multiple partitions by @westonpace in https://github.com/lance-format/lance/pull/5988
docs: clarify v2.2 nested drop rollback risk by @Xuanwo in https://github.com/lance-format/lance/pull/5999
docs: require data_storage_version=2.2 in map type example by @Xuanwo in https://github.com/lance-format/lance/pull/6032
docs: update file versioning matrix for 2.2 rollout by @Xuanwo in https://github.com/lance-format/lance/pull/6033
docs: reorganize blob docs around blob v2 and clarify legacy compatibility by @Xuanwo in https://github.com/lance-format/lance/pull/6034
docs: align 2.2 encoding docs and nested add-column notes by @Xuanwo in https://github.com/lance-format/lance/pull/6038
docs: clarify how to generate TPCH benchmark dataset locally by @Xuanwo in https://github.com/lance-format/lance/pull/6063

Performance Improvements 🚀

perf: add vector throughput benchmark by @westonpace in https://github.com/lance-format/lance/pull/5644
perf: add a full text search benchmark by @westonpace in https://github.com/lance-format/lance/pull/5665
perf: don't concat the batches for writing posting lists by @BubbleCal in https://github.com/lance-format/lance/pull/5769
perf: add a lightweight scheduler implementation by @westonpace in https://github.com/lance-format/lance/pull/5773
perf: use cpu pool to process all posting lists by @BubbleCal in https://github.com/lance-format/lance/pull/5780
perf: calculate cardinality lazily by @Xuanwo in https://github.com/lance-format/lance/pull/5783
perf: replace flatmap in build_distance_table by @wkalt in https://github.com/lance-format/lance/pull/5898
perf: change Dataset::sample to sort its random indices by @wkalt in https://github.com/lance-format/lance/pull/5915
perf: build fp16kernels with NEON support on iOS by @chyyran in https://github.com/lance-format/lance/pull/5917
perf: reduce peak memory in nullable training data sampling by @wkalt in https://github.com/lance-format/lance/pull/5935
perf: upgrade roaring to 0.11 and improve bitmap/range conversions by @LuQQiu in https://github.com/lance-format/lance/pull/5961
perf: speed up format v2.2 scans by adding shortcut for full page by @Xuanwo in https://github.com/lance-format/lance/pull/5981
perf: speed up format 2.2 300% by spawning structural decode batch tasks by @Xuanwo in https://github.com/lance-format/lance/pull/5982
perf: improve parallelism of data_stats by @wkalt in https://github.com/lance-format/lance/pull/5990
perf: reduce peak memory during cosine IVF-PQ index training by @wkalt in https://github.com/lance-format/lance/pull/6016
perf: fast rotation for RQ quantization by @BubbleCal in https://github.com/lance-format/lance/pull/6024
perf: avoid re-open shard indices and small reads by @BubbleCal in https://github.com/lance-format/lance/pull/6026
perf: disable auto FSST for binary fields by @Xuanwo in https://github.com/lance-format/lance/pull/6047
perf: speedup flat fts by @westonpace in https://github.com/lance-format/lance/pull/6054
perf: add dict-values compression controls with lz4 default by @Xuanwo in https://github.com/lance-format/lance/pull/6059
perf: avoid frequent allocating when computing residual vectors by @BubbleCal in https://github.com/lance-format/lance/pull/6062
perf: add take_blob benchmark with cache_repetition_index matrix by @Xuanwo in https://github.com/lance-format/lance/pull/6067

Other Changes

refactor: change reader's get_range result to be a static future by @westonpace in https://github.com/lance-format/lance/pull/5755
refactor(python): migrate torch.jit.script to torch.compile by @wjones127 in https://github.com/lance-format/lance/pull/5759
test: fix tests broken by pandas 3 release by @westonpace in https://github.com/lance-format/lance/pull/5786
refactor: use dict entries and encoded size instead of cardinality for dict decision by @Xuanwo in https://github.com/lance-format/lance/pull/5891
refactor: correct panic message typos in OrderableScalarValue::cmp by @fenfeng9 in https://github.com/lance-format/lance/pull/5913
refactor: deprecate list_indices and migrate tests to describe_indices by @fenfeng9 in https://github.com/lance-format/lance/pull/5945
refactor: upgrade to SNAFU 0.9 by @shepmaster in https://github.com/lance-format/lance/pull/6071

Full Changelog: https://github.com/lance-format/lance/compare/release-root/3.0.0-beta.N...v3.0.0

Open source →

2.0.05 Feb 2026

What's Changed

Breaking Changes 🛠

fix!: null handling when using NOT with scalar indices by @wjones127 in https://github.com/lance-format/lance/pull/5270
feat!: track cumulative wall time in analyze plan by @wkalt in https://github.com/lance-format/lance/pull/5505
fix!: check metric compatibility before using vector index by @wjones127 in https://github.com/lance-format/lance/pull/5609
feat!: define default index name and return IndexMetadata after building index by @wjones127 in https://github.com/lance-format/lance/pull/5645
feat!: make v2 manifest default by @wojiaodoubao in https://github.com/lance-format/lance/pull/5656
refactor!: introduce storage options accessor by @jackye1995 in https://github.com/lance-format/lance/pull/5728

New Features 🎉

feat: support using FTS as a filter in vector search by @wojiaodoubao in https://github.com/lance-format/lance/pull/4928
feat: support when_matched_delete in merge_insert by @jtuglu1 in https://github.com/lance-format/lance/pull/4939
feat: add support for large minichunk size (u32) in format v2.2 by @niyue in https://github.com/lance-format/lance/pull/4959
feat: support GEO RTree index by @ddupg in https://github.com/lance-format/lance/pull/5034
feat: support global tag retrieval and improve tag api by @majin1102 in https://github.com/lance-format/lance/pull/5088
feat: support create vector index distributedly by @chenghao-guo in https://github.com/lance-format/lance/pull/5117
feat: support add sub-column to struct col by @wojiaodoubao in https://github.com/lance-format/lance/pull/5126
feat: distributed range-based BTree index by @steFaiz in https://github.com/lance-format/lance/pull/5202
feat: strategized plan compaction by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5233
feat: dataset supports deep_clone by @majin1102 in https://github.com/lance-format/lance/pull/5250
feat: cleanup only scan managed files by @majin1102 in https://github.com/lance-format/lance/pull/5338
feat: support map data type in lance format version 2.2 by @xloya in https://github.com/lance-format/lance/pull/5349
feat: add RTree index spec in table format by @ddupg in https://github.com/lance-format/lance/pull/5360
feat(java): support row lineage and cdf apis by @yanghua in https://github.com/lance-format/lance/pull/5362
feat: disable default features on internal use by @valkum in https://github.com/lance-format/lance/pull/5372
feat(cdf): support set start/end timestamp in cdf by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5378
feat(blob_v2): add external blob support by @Xuanwo in https://github.com/lance-format/lance/pull/5385
feat(blob_v2): add dedicated blob support by @Xuanwo in https://github.com/lance-format/lance/pull/5406
feat: fallback to CPU if GPU accelerating is unavailable by @BubbleCal in https://github.com/lance-format/lance/pull/5407
feat(blob_v2): add packed blob support by @Xuanwo in https://github.com/lance-format/lance/pull/5413
feat: allow python tracing / logging to be independently configured by @westonpace in https://github.com/lance-format/lance/pull/5415
feat: add additional index APIs to support count rows split plan by @jackye1995 in https://github.com/lance-format/lance/pull/5447
feat(java): support multi-bases for writing database by @ddupg in https://github.com/lance-format/lance/pull/5450
feat(blob_v2): add BlobAray API for user input by @Xuanwo in https://github.com/lance-format/lance/pull/5451
feat: upgrade lance-namespace to 0.3.1 and add missing apis by @jackye1995 in https://github.com/lance-format/lance/pull/5457
feat(python): support cleanup_with_policy by @ddupg in https://github.com/lance-format/lance/pull/5458
feat: support dropping sub-column of list(struct) by @wojiaodoubao in https://github.com/lance-format/lance/pull/5469
feat(blob_v2): add GC support by @Xuanwo in https://github.com/lance-format/lance/pull/5473
feat: add py.typed marker file by @jonded94 in https://github.com/lance-format/lance/pull/5479
feat(python): expose the distance_range param in the Python scanner nearest config by @xloya in https://github.com/lance-format/lance/pull/5486
feat(java): simplify the use of optional in jni by @ddupg in https://github.com/lance-format/lance/pull/5488
feat(blob_v2): add Python API for Blob v2 by @Xuanwo in https://github.com/lance-format/lance/pull/5491
feat(python): add DatasetBasePath stub to improve IDE hints by @ddupg in https://github.com/lance-format/lance/pull/5503
feat(memtest): add macos support by @Xuanwo in https://github.com/lance-format/lance/pull/5510
feat(java): add full text search api by @wojiaodoubao in https://github.com/lance-format/lance/pull/5563
feat: support credentials vending in directory namespace by @jackye1995 in https://github.com/lance-format/lance/pull/5566
feat: upgrade lance-namespace to 0.4.0 by @jackye1995 in https://github.com/lance-format/lance/pull/5568
feat: add skip_merge for FTS index build by @BubbleCal in https://github.com/lance-format/lance/pull/5570
feat(java): add builder-style scalar index params by @wojiaodoubao in https://github.com/lance-format/lance/pull/5581
feat: optimize rle implementation by @Xuanwo in https://github.com/lance-format/lance/pull/5586
feat: support FixedSizeList<Struct> by @wkalt in https://github.com/lance-format/lance/pull/5593
feat: add dictionary encoding for 64bit types like int64/double by @Xuanwo in https://github.com/lance-format/lance/pull/5594
feat: support merge_insert with source dedupe on first seen value by @jackye1995 in https://github.com/lance-format/lance/pull/5603
feat: support truncate table api by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5604
feat: add Error::External variant for preserving user errors by @wjones127 in https://github.com/lance-format/lance/pull/5606
feat: upgrade lance-namespace to 0.4.5 by @jackye1995 in https://github.com/lance-format/lance/pull/5611
feat: refactor use of Error::io by @lichuang in https://github.com/lance-format/lance/pull/5612
feat(java): add detached flag to commitTransaction by @wojiaodoubao in https://github.com/lance-format/lance/pull/5626
feat: add parts_searched metrics for FTS by @BubbleCal in https://github.com/lance-format/lance/pull/5627
feat: improve the random access file benchmark by @westonpace in https://github.com/lance-format/lance/pull/5628
feat(oss): add sts token support for aliyun oss via storage_options by @hh23485 in https://github.com/lance-format/lance/pull/5632
feat: merge-insert with primary key dedupe by @jackye1995 in https://github.com/lance-format/lance/pull/5633
feat(java): expose index description and statistics by @majin1102 in https://github.com/lance-format/lance/pull/5655
feat: allow configure temp dir size for datafusion exec by @jackye1995 in https://github.com/lance-format/lance/pull/5659
feat(java): add support for optimizing indices by @majin1102 in https://github.com/lance-format/lance/pull/5663
feat: make on arg optional for merge insert api by @yanghua in https://github.com/lance-format/lance/pull/5667
feat: make OneShotPartitionStream pub by @timsaucer in https://github.com/lance-format/lance/pull/5672
feat: support array_contains in LabelList scalar index by @fenfeng9 in https://github.com/lance-format/lance/pull/5681
feat: add order to primary key by @touch-of-grey in https://github.com/lance-format/lance/pull/5683
feat: use independent region manifest for MemWAL by @touch-of-grey in https://github.com/lance-format/lance/pull/5689
feat: add stats() method to ObjectStoreRegistry by @wkalt in https://github.com/lance-format/lance/pull/5706
feat: support dynamic context for lance namespace by @jackye1995 in https://github.com/lance-format/lance/pull/5710
feat: make blob v2 dedicated threshold configurable by @yanghua in https://github.com/lance-format/lance/pull/5719
feat: cleanup partial idx files when merging distributed vector index by @yanghua in https://github.com/lance-format/lance/pull/5729
feat: expose blob handling APIs to python by @Xuanwo in https://github.com/lance-format/lance/pull/5790
feat: add blob handling support for fragment by @Xuanwo in https://github.com/lance-format/lance/pull/5801

Bug Fixes 🐛

fix: correct null_count aggregation in boolean statistics collection by @YinZheng-Sun in https://github.com/lance-format/lance/pull/4839
fix: remove logging for project_batch by @westonpace in https://github.com/lance-format/lance/pull/5267
fix: stop documenting FTS index type, standardize on INVERTED by @mackrorysd in https://github.com/lance-format/lance/pull/5315
fix: don't allow change blob version during update by @Xuanwo in https://github.com/lance-format/lance/pull/5386
fix: take_blobs_by_indices fails with stable row IDs on fragment 1+ by @jmhsieh in https://github.com/lance-format/lance/pull/5392
fix: respect index metric when user overrides by @BubbleCal in https://github.com/lance-format/lance/pull/5395
fix: remove expensive clone in bitmap search by @westonpace in https://github.com/lance-format/lance/pull/5409
fix: fix vector index prewarm index by @xloya in https://github.com/lance-format/lance/pull/5412
fix: panic unwrap on None in decoder.rs by @camilesing in https://github.com/lance-format/lance/pull/5424
fix: dir namespace cloud storage path removes one subdir level by @jackye1995 in https://github.com/lance-format/lance/pull/5464
fix: make column name lookups case-insensitive by @wjones127 in https://github.com/lance-format/lance/pull/5465
fix: ensure trailing slash is normalized in rest adapter by @jackye1995 in https://github.com/lance-format/lance/pull/5499
fix(java): support FixedSizeList for java LanceField by @fangbo in https://github.com/lance-format/lance/pull/5509
fix: head external manifest object happend 404 NotFound error by @hushengquan in https://github.com/lance-format/lance/pull/5512
fix: json's arrow extension metadata missing by @Xuanwo in https://github.com/lance-format/lance/pull/5527
fix: infer multivector sampling rows by @BubbleCal in https://github.com/lance-format/lance/pull/5534
fix: support ManifestNamingSchemeV2 with unordered object stores by @wjones127 in https://github.com/lance-format/lance/pull/5539
fix: merge_insert uses full schema path for reordered columns by @wjones127 in https://github.com/lance-format/lance/pull/5541
fix: allow storage options provider without expires_at_millis by @jackye1995 in https://github.com/lance-format/lance/pull/5542
fix(ci): use pull_request_target for fork PR reviews by @wjones127 in https://github.com/lance-format/lance/pull/5544
fix: restore decrease max_fragment_id in manifest by @majin1102 in https://github.com/lance-format/lance/pull/5554
fix: improve error handling for environment variable parsing by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/5560
fix: panic when lance.auto_cleanup.interval is set to 0 by @majin1102 in https://github.com/lance-format/lance/pull/5571
fix(python): correct type hint for to_tensor_fn parameter by @AndreaBozzo in https://github.com/lance-format/lance/pull/5577
fix: avoid panic while hitting non-null empty multi-vector by @Xuanwo in https://github.com/lance-format/lance/pull/5588
fix: filter garbage entries from null maps during encoding by @wkalt in https://github.com/lance-format/lance/pull/5591
fix: reduce verbosity of errors due to string conversion by @wjones127 in https://github.com/lance-format/lance/pull/5600
fix: remove imports that are not needed by @westonpace in https://github.com/lance-format/lance/pull/5651
fix: allow nearest applied in default_scan_options by @chenghao-guo in https://github.com/lance-format/lance/pull/5666
fix: trait Array has been sealed in arrow new version by @Xuanwo in https://github.com/lance-format/lance/pull/5690
fix: project_by_schema now reorders fields inside List<Struct> types by @wjones127 in https://github.com/lance-format/lance/pull/5703
fix: allocate too much memory for block max scores by @BubbleCal in https://github.com/lance-format/lance/pull/5718
docs: in dataset.rs, fix comment for get_fragments by @cmccabe in https://github.com/lance-format/lance/pull/5724
fix(python): close SQLite connections in BatchUDFCheckpoint by @wjones127 in https://github.com/lance-format/lance/pull/5733
fix: remove credential vending features from python and java bindings by @jackye1995 in https://github.com/lance-format/lance/pull/5737
fix: allow unused_unsafe for __cpuid to support both stable and nightly by @jackye1995 in https://github.com/lance-format/lance/pull/5793
fix: fix remap so that it handles deletions correctly by @westonpace in https://github.com/lance-format/lance/pull/5828

Documentation 📚

docs: fix Append call in distributed write guide by @rongou in https://github.com/lance-format/lance/pull/5439
docs: fix and improve the description about row id by @yanghua in https://github.com/lance-format/lance/pull/5463
docs: add specification for handling indices by @wjones127 in https://github.com/lance-format/lance/pull/5543
docs: fix duplicate words in comments and error messages by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/5548
docs: add research paper link to the landing page by @prrao87 in https://github.com/lance-format/lance/pull/5549
docs: auto-build refactored namespace integrations doc by @jackye1995 in https://github.com/lance-format/lance/pull/5562
docs: rename RowIdTreeMap to RowAddrTreeMap in rtree.md by @ddupg in https://github.com/lance-format/lance/pull/5564
docs: add docs for DuckDB extension by @prrao87 in https://github.com/lance-format/lance/pull/5578
docs: update Lance-DuckDB docs to latest version 0.4.1 by @prrao87 in https://github.com/lance-format/lance/pull/5613

Performance Improvements 🚀

perf: do not instrument self in multipart upload by @westonpace in https://github.com/lance-format/lance/pull/5416
perf: various btree performance improvements by @westonpace in https://github.com/lance-format/lance/pull/5446
perf: reuse session context by @wjones127 in https://github.com/lance-format/lance/pull/5462
perf: offload IVF partition build to CPU pool by @BubbleCal in https://github.com/lance-format/lance/pull/5551
perf: materialize the tokens after WAND done by @BubbleCal in https://github.com/lance-format/lance/pull/5572
perf: compute HNSW level counts after build by @BubbleCal in https://github.com/lance-format/lance/pull/5590
perf: improve SQ query speed by @BubbleCal in https://github.com/lance-format/lance/pull/5596
perf: reuse zstd compressors in encoding by @wkalt in https://github.com/lance-format/lance/pull/5598
perf: use binary search to skip documents by @BubbleCal in https://github.com/lance-format/lance/pull/5636
perf: improve FTS indexing perf and reduce memory footprint by @BubbleCal in https://github.com/lance-format/lance/pull/5650
perf: avoid copying tokens while merging by @BubbleCal in https://github.com/lance-format/lance/pull/5661
perf: tighten WAND block score upper bound by @BubbleCal in https://github.com/lance-format/lance/pull/5668
perf: cache global BM25 idf per query by @BubbleCal in https://github.com/lance-format/lance/pull/5727
perf: use LRU cache for session contexts in get_session_context by @wjones127 in https://github.com/lance-format/lance/pull/5736
perf: merge partitions in stream style by @BubbleCal in https://github.com/lance-format/lance/pull/5754

Other Changes

refactor: write bitmap index statistics in file instead by @Xuanwo in https://github.com/lance-format/lance/pull/5251
refactor: rename RowIdTreeMap to RowAddrTreeMap by @yanghua in https://github.com/lance-format/lance/pull/5266
refactor: rename RowIdMask to RowAddrMask by @yanghua in https://github.com/lance-format/lance/pull/5281
refactor: consolidate logic between zonemap and bloomfilter indexes by @fenfeng9 in https://github.com/lance-format/lance/pull/5374
refactor: split dataset tests in a tests mod by @Xuanwo in https://github.com/lance-format/lance/pull/5387
refactor: use the same path for dedicated and packed blob by @Xuanwo in https://github.com/lance-format/lance/pull/5449
refactor: add store_prefix to lance-io's ObjectStore by @cmccabe in https://github.com/lance-format/lance/pull/5468
refactor: expose take_blobs_by_addresses to python by @Xuanwo in https://github.com/lance-format/lance/pull/5474
refactor: support java 21, drop java 8 by @cmccabe in https://github.com/lance-format/lance/pull/5565
refactor: allow switching to bitpack inside RLE by @Xuanwo in https://github.com/lance-format/lance/pull/5595
refactor: introduce RowSetOps and refactor RowAddrTreeMap by @yanghua in https://github.com/lance-format/lance/pull/5624
refactor(python): migrate torch.jit.script to torch.compile by @wjones127 in https://github.com/lance-format/lance/pull/5759
test: fix tests broken by pandas 3 release by @westonpace in https://github.com/lance-format/lance/pull/5786

Full Changelog: https://github.com/lance-format/lance/compare/release-root/2.0.0-beta.N...v2.0.0

Open source →

2.0.0rc429 Jan 2026pre-release

What's Changed

Breaking Changes 🛠

fix!: null handling when using NOT with scalar indices by @wjones127 in https://github.com/lance-format/lance/pull/5270
feat!: track cumulative wall time in analyze plan by @wkalt in https://github.com/lance-format/lance/pull/5505
fix!: check metric compatibility before using vector index by @wjones127 in https://github.com/lance-format/lance/pull/5609
feat!: define default index name and return IndexMetadata after building index by @wjones127 in https://github.com/lance-format/lance/pull/5645
feat!: make v2 manifest default by @wojiaodoubao in https://github.com/lance-format/lance/pull/5656
refactor!: introduce storage options accessor by @jackye1995 in https://github.com/lance-format/lance/pull/5728

New Features 🎉

feat: support using FTS as a filter in vector search by @wojiaodoubao in https://github.com/lance-format/lance/pull/4928
feat: support when_matched_delete in merge_insert by @jtuglu1 in https://github.com/lance-format/lance/pull/4939
feat: add support for large minichunk size (u32) in format v2.2 by @niyue in https://github.com/lance-format/lance/pull/4959
feat: support GEO RTree index by @ddupg in https://github.com/lance-format/lance/pull/5034
feat: support global tag retrieval and improve tag api by @majin1102 in https://github.com/lance-format/lance/pull/5088
feat: support create vector index distributedly by @chenghao-guo in https://github.com/lance-format/lance/pull/5117
feat: support add sub-column to struct col by @wojiaodoubao in https://github.com/lance-format/lance/pull/5126
feat: distributed range-based BTree index by @steFaiz in https://github.com/lance-format/lance/pull/5202
feat: strategized plan compaction by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5233
feat: dataset supports deep_clone by @majin1102 in https://github.com/lance-format/lance/pull/5250
feat: cleanup only scan managed files by @majin1102 in https://github.com/lance-format/lance/pull/5338
feat: support map data type in lance format version 2.2 by @xloya in https://github.com/lance-format/lance/pull/5349
feat: add RTree index spec in table format by @ddupg in https://github.com/lance-format/lance/pull/5360
feat(java): support row lineage and cdf apis by @yanghua in https://github.com/lance-format/lance/pull/5362
feat: disable default features on internal use by @valkum in https://github.com/lance-format/lance/pull/5372
feat(cdf): support set start/end timestamp in cdf by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5378
feat(blob_v2): add external blob support by @Xuanwo in https://github.com/lance-format/lance/pull/5385
feat(blob_v2): add dedicated blob support by @Xuanwo in https://github.com/lance-format/lance/pull/5406
feat: fallback to CPU if GPU accelerating is unavailable by @BubbleCal in https://github.com/lance-format/lance/pull/5407
feat(blob_v2): add packed blob support by @Xuanwo in https://github.com/lance-format/lance/pull/5413
feat: allow python tracing / logging to be independently configured by @westonpace in https://github.com/lance-format/lance/pull/5415
feat: add additional index APIs to support count rows split plan by @jackye1995 in https://github.com/lance-format/lance/pull/5447
feat(java): support multi-bases for writing database by @ddupg in https://github.com/lance-format/lance/pull/5450
feat(blob_v2): add BlobAray API for user input by @Xuanwo in https://github.com/lance-format/lance/pull/5451
feat: upgrade lance-namespace to 0.3.1 and add missing apis by @jackye1995 in https://github.com/lance-format/lance/pull/5457
feat(python): support cleanup_with_policy by @ddupg in https://github.com/lance-format/lance/pull/5458
feat: support dropping sub-column of list(struct) by @wojiaodoubao in https://github.com/lance-format/lance/pull/5469
feat(blob_v2): add GC support by @Xuanwo in https://github.com/lance-format/lance/pull/5473
feat: add py.typed marker file by @jonded94 in https://github.com/lance-format/lance/pull/5479
feat(python): expose the distance_range param in the Python scanner nearest config by @xloya in https://github.com/lance-format/lance/pull/5486
feat(java): simplify the use of optional in jni by @ddupg in https://github.com/lance-format/lance/pull/5488
feat(blob_v2): add Python API for Blob v2 by @Xuanwo in https://github.com/lance-format/lance/pull/5491
feat(python): add DatasetBasePath stub to improve IDE hints by @ddupg in https://github.com/lance-format/lance/pull/5503
feat(memtest): add macos support by @Xuanwo in https://github.com/lance-format/lance/pull/5510
feat(java): add full text search api by @wojiaodoubao in https://github.com/lance-format/lance/pull/5563
feat: support credentials vending in directory namespace by @jackye1995 in https://github.com/lance-format/lance/pull/5566
feat: upgrade lance-namespace to 0.4.0 by @jackye1995 in https://github.com/lance-format/lance/pull/5568
feat: add skip_merge for FTS index build by @BubbleCal in https://github.com/lance-format/lance/pull/5570
feat(java): add builder-style scalar index params by @wojiaodoubao in https://github.com/lance-format/lance/pull/5581
feat: optimize rle implementation by @Xuanwo in https://github.com/lance-format/lance/pull/5586
feat: support FixedSizeList<Struct> by @wkalt in https://github.com/lance-format/lance/pull/5593
feat: add dictionary encoding for 64bit types like int64/double by @Xuanwo in https://github.com/lance-format/lance/pull/5594
feat: support merge_insert with source dedupe on first seen value by @jackye1995 in https://github.com/lance-format/lance/pull/5603
feat: support truncate table api by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5604
feat: add Error::External variant for preserving user errors by @wjones127 in https://github.com/lance-format/lance/pull/5606
feat: upgrade lance-namespace to 0.4.5 by @jackye1995 in https://github.com/lance-format/lance/pull/5611
feat: refactor use of Error::io by @lichuang in https://github.com/lance-format/lance/pull/5612
feat(java): add detached flag to commitTransaction by @wojiaodoubao in https://github.com/lance-format/lance/pull/5626
feat: add parts_searched metrics for FTS by @BubbleCal in https://github.com/lance-format/lance/pull/5627
feat: improve the random access file benchmark by @westonpace in https://github.com/lance-format/lance/pull/5628
feat(oss): add sts token support for aliyun oss via storage_options by @hh23485 in https://github.com/lance-format/lance/pull/5632
feat: merge-insert with primary key dedupe by @jackye1995 in https://github.com/lance-format/lance/pull/5633
feat(java): expose index description and statistics by @majin1102 in https://github.com/lance-format/lance/pull/5655
feat: allow configure temp dir size for datafusion exec by @jackye1995 in https://github.com/lance-format/lance/pull/5659
feat(java): add support for optimizing indices by @majin1102 in https://github.com/lance-format/lance/pull/5663
feat: make on arg optional for merge insert api by @yanghua in https://github.com/lance-format/lance/pull/5667
feat: make OneShotPartitionStream pub by @timsaucer in https://github.com/lance-format/lance/pull/5672
feat: support array_contains in LabelList scalar index by @fenfeng9 in https://github.com/lance-format/lance/pull/5681
feat: add order to primary key by @touch-of-grey in https://github.com/lance-format/lance/pull/5683
feat: use independent region manifest for MemWAL by @touch-of-grey in https://github.com/lance-format/lance/pull/5689
feat: add stats() method to ObjectStoreRegistry by @wkalt in https://github.com/lance-format/lance/pull/5706
feat: support dynamic context for lance namespace by @jackye1995 in https://github.com/lance-format/lance/pull/5710
feat: make blob v2 dedicated threshold configurable by @yanghua in https://github.com/lance-format/lance/pull/5719
feat: cleanup partial idx files when merging distributed vector index by @yanghua in https://github.com/lance-format/lance/pull/5729
feat: expose blob handling APIs to python by @Xuanwo in https://github.com/lance-format/lance/pull/5790
feat: add blob handling support for fragment by @Xuanwo in https://github.com/lance-format/lance/pull/5801

Bug Fixes 🐛

fix: correct null_count aggregation in boolean statistics collection by @YinZheng-Sun in https://github.com/lance-format/lance/pull/4839
fix: remove logging for project_batch by @westonpace in https://github.com/lance-format/lance/pull/5267
fix: stop documenting FTS index type, standardize on INVERTED by @mackrorysd in https://github.com/lance-format/lance/pull/5315
fix: don't allow change blob version during update by @Xuanwo in https://github.com/lance-format/lance/pull/5386
fix: take_blobs_by_indices fails with stable row IDs on fragment 1+ by @jmhsieh in https://github.com/lance-format/lance/pull/5392
fix: respect index metric when user overrides by @BubbleCal in https://github.com/lance-format/lance/pull/5395
fix: remove expensive clone in bitmap search by @westonpace in https://github.com/lance-format/lance/pull/5409
fix: fix vector index prewarm index by @xloya in https://github.com/lance-format/lance/pull/5412
fix: panic unwrap on None in decoder.rs by @camilesing in https://github.com/lance-format/lance/pull/5424
fix: dir namespace cloud storage path removes one subdir level by @jackye1995 in https://github.com/lance-format/lance/pull/5464
fix: make column name lookups case-insensitive by @wjones127 in https://github.com/lance-format/lance/pull/5465
fix: ensure trailing slash is normalized in rest adapter by @jackye1995 in https://github.com/lance-format/lance/pull/5499
fix(java): support FixedSizeList for java LanceField by @fangbo in https://github.com/lance-format/lance/pull/5509
fix: head external manifest object happend 404 NotFound error by @hushengquan in https://github.com/lance-format/lance/pull/5512
fix: json's arrow extension metadata missing by @Xuanwo in https://github.com/lance-format/lance/pull/5527
fix: infer multivector sampling rows by @BubbleCal in https://github.com/lance-format/lance/pull/5534
fix: support ManifestNamingSchemeV2 with unordered object stores by @wjones127 in https://github.com/lance-format/lance/pull/5539
fix: merge_insert uses full schema path for reordered columns by @wjones127 in https://github.com/lance-format/lance/pull/5541
fix: allow storage options provider without expires_at_millis by @jackye1995 in https://github.com/lance-format/lance/pull/5542
fix(ci): use pull_request_target for fork PR reviews by @wjones127 in https://github.com/lance-format/lance/pull/5544
fix: restore decrease max_fragment_id in manifest by @majin1102 in https://github.com/lance-format/lance/pull/5554
fix: improve error handling for environment variable parsing by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/5560
fix: panic when lance.auto_cleanup.interval is set to 0 by @majin1102 in https://github.com/lance-format/lance/pull/5571
fix(python): correct type hint for to_tensor_fn parameter by @AndreaBozzo in https://github.com/lance-format/lance/pull/5577
fix: avoid panic while hitting non-null empty multi-vector by @Xuanwo in https://github.com/lance-format/lance/pull/5588
fix: filter garbage entries from null maps during encoding by @wkalt in https://github.com/lance-format/lance/pull/5591
fix: reduce verbosity of errors due to string conversion by @wjones127 in https://github.com/lance-format/lance/pull/5600
fix: remove imports that are not needed by @westonpace in https://github.com/lance-format/lance/pull/5651
fix: allow nearest applied in default_scan_options by @chenghao-guo in https://github.com/lance-format/lance/pull/5666
fix: trait Array has been sealed in arrow new version by @Xuanwo in https://github.com/lance-format/lance/pull/5690
fix: project_by_schema now reorders fields inside List<Struct> types by @wjones127 in https://github.com/lance-format/lance/pull/5703
fix: allocate too much memory for block max scores by @BubbleCal in https://github.com/lance-format/lance/pull/5718
docs: in dataset.rs, fix comment for get_fragments by @cmccabe in https://github.com/lance-format/lance/pull/5724
fix(python): close SQLite connections in BatchUDFCheckpoint by @wjones127 in https://github.com/lance-format/lance/pull/5733
fix: remove credential vending features from python and java bindings by @jackye1995 in https://github.com/lance-format/lance/pull/5737
fix: allow unused_unsafe for __cpuid to support both stable and nightly by @jackye1995 in https://github.com/lance-format/lance/pull/5793
fix: fix remap so that it handles deletions correctly by @westonpace in https://github.com/lance-format/lance/pull/5828

Documentation 📚

docs: fix Append call in distributed write guide by @rongou in https://github.com/lance-format/lance/pull/5439
docs: fix and improve the description about row id by @yanghua in https://github.com/lance-format/lance/pull/5463
docs: add specification for handling indices by @wjones127 in https://github.com/lance-format/lance/pull/5543
docs: fix duplicate words in comments and error messages by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/5548
docs: add research paper link to the landing page by @prrao87 in https://github.com/lance-format/lance/pull/5549
docs: auto-build refactored namespace integrations doc by @jackye1995 in https://github.com/lance-format/lance/pull/5562
docs: rename RowIdTreeMap to RowAddrTreeMap in rtree.md by @ddupg in https://github.com/lance-format/lance/pull/5564
docs: add docs for DuckDB extension by @prrao87 in https://github.com/lance-format/lance/pull/5578
docs: update Lance-DuckDB docs to latest version 0.4.1 by @prrao87 in https://github.com/lance-format/lance/pull/5613

Performance Improvements 🚀

perf: do not instrument self in multipart upload by @westonpace in https://github.com/lance-format/lance/pull/5416
perf: various btree performance improvements by @westonpace in https://github.com/lance-format/lance/pull/5446
perf: reuse session context by @wjones127 in https://github.com/lance-format/lance/pull/5462
perf: offload IVF partition build to CPU pool by @BubbleCal in https://github.com/lance-format/lance/pull/5551
perf: materialize the tokens after WAND done by @BubbleCal in https://github.com/lance-format/lance/pull/5572
perf: compute HNSW level counts after build by @BubbleCal in https://github.com/lance-format/lance/pull/5590
perf: improve SQ query speed by @BubbleCal in https://github.com/lance-format/lance/pull/5596
perf: reuse zstd compressors in encoding by @wkalt in https://github.com/lance-format/lance/pull/5598
perf: use binary search to skip documents by @BubbleCal in https://github.com/lance-format/lance/pull/5636
perf: improve FTS indexing perf and reduce memory footprint by @BubbleCal in https://github.com/lance-format/lance/pull/5650
perf: avoid copying tokens while merging by @BubbleCal in https://github.com/lance-format/lance/pull/5661
perf: tighten WAND block score upper bound by @BubbleCal in https://github.com/lance-format/lance/pull/5668
perf: cache global BM25 idf per query by @BubbleCal in https://github.com/lance-format/lance/pull/5727
perf: use LRU cache for session contexts in get_session_context by @wjones127 in https://github.com/lance-format/lance/pull/5736
perf: merge partitions in stream style by @BubbleCal in https://github.com/lance-format/lance/pull/5754

Other Changes

refactor: write bitmap index statistics in file instead by @Xuanwo in https://github.com/lance-format/lance/pull/5251
refactor: rename RowIdTreeMap to RowAddrTreeMap by @yanghua in https://github.com/lance-format/lance/pull/5266
refactor: rename RowIdMask to RowAddrMask by @yanghua in https://github.com/lance-format/lance/pull/5281
refactor: consolidate logic between zonemap and bloomfilter indexes by @fenfeng9 in https://github.com/lance-format/lance/pull/5374
refactor: split dataset tests in a tests mod by @Xuanwo in https://github.com/lance-format/lance/pull/5387
refactor: use the same path for dedicated and packed blob by @Xuanwo in https://github.com/lance-format/lance/pull/5449
refactor: add store_prefix to lance-io's ObjectStore by @cmccabe in https://github.com/lance-format/lance/pull/5468
refactor: expose take_blobs_by_addresses to python by @Xuanwo in https://github.com/lance-format/lance/pull/5474
refactor: support java 21, drop java 8 by @cmccabe in https://github.com/lance-format/lance/pull/5565
refactor: allow switching to bitpack inside RLE by @Xuanwo in https://github.com/lance-format/lance/pull/5595
refactor: introduce RowSetOps and refactor RowAddrTreeMap by @yanghua in https://github.com/lance-format/lance/pull/5624
refactor(python): migrate torch.jit.script to torch.compile by @wjones127 in https://github.com/lance-format/lance/pull/5759
test: fix tests broken by pandas 3 release by @westonpace in https://github.com/lance-format/lance/pull/5786

Full Changelog: https://github.com/lance-format/lance/compare/release-root/2.0.0-beta.N...v2.0.0-rc.4

Open source →

2.0.0rc327 Jan 2026pre-release

What's Changed

Breaking Changes 🛠

fix!: null handling when using NOT with scalar indices by @wjones127 in https://github.com/lance-format/lance/pull/5270
feat!: track cumulative wall time in analyze plan by @wkalt in https://github.com/lance-format/lance/pull/5505
fix!: check metric compatibility before using vector index by @wjones127 in https://github.com/lance-format/lance/pull/5609
feat!: define default index name and return IndexMetadata after building index by @wjones127 in https://github.com/lance-format/lance/pull/5645
feat!: make v2 manifest default by @wojiaodoubao in https://github.com/lance-format/lance/pull/5656
refactor!: introduce storage options accessor by @jackye1995 in https://github.com/lance-format/lance/pull/5728

New Features 🎉

feat: support using FTS as a filter in vector search by @wojiaodoubao in https://github.com/lance-format/lance/pull/4928
feat: support when_matched_delete in merge_insert by @jtuglu1 in https://github.com/lance-format/lance/pull/4939
feat: add support for large minichunk size (u32) in format v2.2 by @niyue in https://github.com/lance-format/lance/pull/4959
feat: support GEO RTree index by @ddupg in https://github.com/lance-format/lance/pull/5034
feat: support global tag retrieval and improve tag api by @majin1102 in https://github.com/lance-format/lance/pull/5088
feat: support create vector index distributedly by @chenghao-guo in https://github.com/lance-format/lance/pull/5117
feat: support add sub-column to struct col by @wojiaodoubao in https://github.com/lance-format/lance/pull/5126
feat: distributed range-based BTree index by @steFaiz in https://github.com/lance-format/lance/pull/5202
feat: strategized plan compaction by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5233
feat: dataset supports deep_clone by @majin1102 in https://github.com/lance-format/lance/pull/5250
feat: cleanup only scan managed files by @majin1102 in https://github.com/lance-format/lance/pull/5338
feat: support map data type in lance format version 2.2 by @xloya in https://github.com/lance-format/lance/pull/5349
feat: add RTree index spec in table format by @ddupg in https://github.com/lance-format/lance/pull/5360
feat(java): support row lineage and cdf apis by @yanghua in https://github.com/lance-format/lance/pull/5362
feat: disable default features on internal use by @valkum in https://github.com/lance-format/lance/pull/5372
feat(cdf): support set start/end timestamp in cdf by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5378
feat(blob_v2): add external blob support by @Xuanwo in https://github.com/lance-format/lance/pull/5385
feat(blob_v2): add dedicated blob support by @Xuanwo in https://github.com/lance-format/lance/pull/5406
feat: fallback to CPU if GPU accelerating is unavailable by @BubbleCal in https://github.com/lance-format/lance/pull/5407
feat(blob_v2): add packed blob support by @Xuanwo in https://github.com/lance-format/lance/pull/5413
feat: allow python tracing / logging to be independently configured by @westonpace in https://github.com/lance-format/lance/pull/5415
feat: add additional index APIs to support count rows split plan by @jackye1995 in https://github.com/lance-format/lance/pull/5447
feat(java): support multi-bases for writing database by @ddupg in https://github.com/lance-format/lance/pull/5450
feat(blob_v2): add BlobAray API for user input by @Xuanwo in https://github.com/lance-format/lance/pull/5451
feat: upgrade lance-namespace to 0.3.1 and add missing apis by @jackye1995 in https://github.com/lance-format/lance/pull/5457
feat(python): support cleanup_with_policy by @ddupg in https://github.com/lance-format/lance/pull/5458
feat: support dropping sub-column of list(struct) by @wojiaodoubao in https://github.com/lance-format/lance/pull/5469
feat(blob_v2): add GC support by @Xuanwo in https://github.com/lance-format/lance/pull/5473
feat: add py.typed marker file by @jonded94 in https://github.com/lance-format/lance/pull/5479
feat(python): expose the distance_range param in the Python scanner nearest config by @xloya in https://github.com/lance-format/lance/pull/5486
feat(java): simplify the use of optional in jni by @ddupg in https://github.com/lance-format/lance/pull/5488
feat(blob_v2): add Python API for Blob v2 by @Xuanwo in https://github.com/lance-format/lance/pull/5491
feat(python): add DatasetBasePath stub to improve IDE hints by @ddupg in https://github.com/lance-format/lance/pull/5503
feat(memtest): add macos support by @Xuanwo in https://github.com/lance-format/lance/pull/5510
feat(java): add full text search api by @wojiaodoubao in https://github.com/lance-format/lance/pull/5563
feat: support credentials vending in directory namespace by @jackye1995 in https://github.com/lance-format/lance/pull/5566
feat: upgrade lance-namespace to 0.4.0 by @jackye1995 in https://github.com/lance-format/lance/pull/5568
feat: add skip_merge for FTS index build by @BubbleCal in https://github.com/lance-format/lance/pull/5570
feat(java): add builder-style scalar index params by @wojiaodoubao in https://github.com/lance-format/lance/pull/5581
feat: optimize rle implementation by @Xuanwo in https://github.com/lance-format/lance/pull/5586
feat: support FixedSizeList<Struct> by @wkalt in https://github.com/lance-format/lance/pull/5593
feat: add dictionary encoding for 64bit types like int64/double by @Xuanwo in https://github.com/lance-format/lance/pull/5594
feat: support merge_insert with source dedupe on first seen value by @jackye1995 in https://github.com/lance-format/lance/pull/5603
feat: support truncate table api by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5604
feat: add Error::External variant for preserving user errors by @wjones127 in https://github.com/lance-format/lance/pull/5606
feat: upgrade lance-namespace to 0.4.5 by @jackye1995 in https://github.com/lance-format/lance/pull/5611
feat: refactor use of Error::io by @lichuang in https://github.com/lance-format/lance/pull/5612
feat(java): add detached flag to commitTransaction by @wojiaodoubao in https://github.com/lance-format/lance/pull/5626
feat: add parts_searched metrics for FTS by @BubbleCal in https://github.com/lance-format/lance/pull/5627
feat: improve the random access file benchmark by @westonpace in https://github.com/lance-format/lance/pull/5628
feat(oss): add sts token support for aliyun oss via storage_options by @hh23485 in https://github.com/lance-format/lance/pull/5632
feat: merge-insert with primary key dedupe by @jackye1995 in https://github.com/lance-format/lance/pull/5633
feat(java): expose index description and statistics by @majin1102 in https://github.com/lance-format/lance/pull/5655
feat: allow configure temp dir size for datafusion exec by @jackye1995 in https://github.com/lance-format/lance/pull/5659
feat(java): add support for optimizing indices by @majin1102 in https://github.com/lance-format/lance/pull/5663
feat: make on arg optional for merge insert api by @yanghua in https://github.com/lance-format/lance/pull/5667
feat: make OneShotPartitionStream pub by @timsaucer in https://github.com/lance-format/lance/pull/5672
feat: support array_contains in LabelList scalar index by @fenfeng9 in https://github.com/lance-format/lance/pull/5681
feat: add order to primary key by @touch-of-grey in https://github.com/lance-format/lance/pull/5683
feat: use independent region manifest for MemWAL by @touch-of-grey in https://github.com/lance-format/lance/pull/5689
feat: add stats() method to ObjectStoreRegistry by @wkalt in https://github.com/lance-format/lance/pull/5706
feat: support dynamic context for lance namespace by @jackye1995 in https://github.com/lance-format/lance/pull/5710
feat: make blob v2 dedicated threshold configurable by @yanghua in https://github.com/lance-format/lance/pull/5719
feat: cleanup partial idx files when merging distributed vector index by @yanghua in https://github.com/lance-format/lance/pull/5729
feat: expose blob handling APIs to python by @Xuanwo in https://github.com/lance-format/lance/pull/5790
feat: add blob handling support for fragment by @Xuanwo in https://github.com/lance-format/lance/pull/5801

Bug Fixes 🐛

fix: correct null_count aggregation in boolean statistics collection by @YinZheng-Sun in https://github.com/lance-format/lance/pull/4839
fix: remove logging for project_batch by @westonpace in https://github.com/lance-format/lance/pull/5267
fix: stop documenting FTS index type, standardize on INVERTED by @mackrorysd in https://github.com/lance-format/lance/pull/5315
fix: don't allow change blob version during update by @Xuanwo in https://github.com/lance-format/lance/pull/5386
fix: take_blobs_by_indices fails with stable row IDs on fragment 1+ by @jmhsieh in https://github.com/lance-format/lance/pull/5392
fix: respect index metric when user overrides by @BubbleCal in https://github.com/lance-format/lance/pull/5395
fix: remove expensive clone in bitmap search by @westonpace in https://github.com/lance-format/lance/pull/5409
fix: fix vector index prewarm index by @xloya in https://github.com/lance-format/lance/pull/5412
fix: panic unwrap on None in decoder.rs by @camilesing in https://github.com/lance-format/lance/pull/5424
fix: dir namespace cloud storage path removes one subdir level by @jackye1995 in https://github.com/lance-format/lance/pull/5464
fix: make column name lookups case-insensitive by @wjones127 in https://github.com/lance-format/lance/pull/5465
fix: ensure trailing slash is normalized in rest adapter by @jackye1995 in https://github.com/lance-format/lance/pull/5499
fix(java): support FixedSizeList for java LanceField by @fangbo in https://github.com/lance-format/lance/pull/5509
fix: head external manifest object happend 404 NotFound error by @hushengquan in https://github.com/lance-format/lance/pull/5512
fix: json's arrow extension metadata missing by @Xuanwo in https://github.com/lance-format/lance/pull/5527
fix: infer multivector sampling rows by @BubbleCal in https://github.com/lance-format/lance/pull/5534
fix: support ManifestNamingSchemeV2 with unordered object stores by @wjones127 in https://github.com/lance-format/lance/pull/5539
fix: merge_insert uses full schema path for reordered columns by @wjones127 in https://github.com/lance-format/lance/pull/5541
fix: allow storage options provider without expires_at_millis by @jackye1995 in https://github.com/lance-format/lance/pull/5542
fix(ci): use pull_request_target for fork PR reviews by @wjones127 in https://github.com/lance-format/lance/pull/5544
fix: restore decrease max_fragment_id in manifest by @majin1102 in https://github.com/lance-format/lance/pull/5554
fix: improve error handling for environment variable parsing by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/5560
fix: panic when lance.auto_cleanup.interval is set to 0 by @majin1102 in https://github.com/lance-format/lance/pull/5571
fix(python): correct type hint for to_tensor_fn parameter by @AndreaBozzo in https://github.com/lance-format/lance/pull/5577
fix: avoid panic while hitting non-null empty multi-vector by @Xuanwo in https://github.com/lance-format/lance/pull/5588
fix: filter garbage entries from null maps during encoding by @wkalt in https://github.com/lance-format/lance/pull/5591
fix: reduce verbosity of errors due to string conversion by @wjones127 in https://github.com/lance-format/lance/pull/5600
fix: remove imports that are not needed by @westonpace in https://github.com/lance-format/lance/pull/5651
fix: allow nearest applied in default_scan_options by @chenghao-guo in https://github.com/lance-format/lance/pull/5666
fix: trait Array has been sealed in arrow new version by @Xuanwo in https://github.com/lance-format/lance/pull/5690
fix: project_by_schema now reorders fields inside List<Struct> types by @wjones127 in https://github.com/lance-format/lance/pull/5703
fix: allocate too much memory for block max scores by @BubbleCal in https://github.com/lance-format/lance/pull/5718
docs: in dataset.rs, fix comment for get_fragments by @cmccabe in https://github.com/lance-format/lance/pull/5724
fix(python): close SQLite connections in BatchUDFCheckpoint by @wjones127 in https://github.com/lance-format/lance/pull/5733
fix: remove credential vending features from python and java bindings by @jackye1995 in https://github.com/lance-format/lance/pull/5737
fix: allow unused_unsafe for __cpuid to support both stable and nightly by @jackye1995 in https://github.com/lance-format/lance/pull/5793

Documentation 📚

docs: fix Append call in distributed write guide by @rongou in https://github.com/lance-format/lance/pull/5439
docs: fix and improve the description about row id by @yanghua in https://github.com/lance-format/lance/pull/5463
docs: add specification for handling indices by @wjones127 in https://github.com/lance-format/lance/pull/5543
docs: fix duplicate words in comments and error messages by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/5548
docs: add research paper link to the landing page by @prrao87 in https://github.com/lance-format/lance/pull/5549
docs: auto-build refactored namespace integrations doc by @jackye1995 in https://github.com/lance-format/lance/pull/5562
docs: rename RowIdTreeMap to RowAddrTreeMap in rtree.md by @ddupg in https://github.com/lance-format/lance/pull/5564
docs: add docs for DuckDB extension by @prrao87 in https://github.com/lance-format/lance/pull/5578
docs: update Lance-DuckDB docs to latest version 0.4.1 by @prrao87 in https://github.com/lance-format/lance/pull/5613

Performance Improvements 🚀

perf: do not instrument self in multipart upload by @westonpace in https://github.com/lance-format/lance/pull/5416
perf: various btree performance improvements by @westonpace in https://github.com/lance-format/lance/pull/5446
perf: reuse session context by @wjones127 in https://github.com/lance-format/lance/pull/5462
perf: offload IVF partition build to CPU pool by @BubbleCal in https://github.com/lance-format/lance/pull/5551
perf: materialize the tokens after WAND done by @BubbleCal in https://github.com/lance-format/lance/pull/5572
perf: compute HNSW level counts after build by @BubbleCal in https://github.com/lance-format/lance/pull/5590
perf: improve SQ query speed by @BubbleCal in https://github.com/lance-format/lance/pull/5596
perf: reuse zstd compressors in encoding by @wkalt in https://github.com/lance-format/lance/pull/5598
perf: use binary search to skip documents by @BubbleCal in https://github.com/lance-format/lance/pull/5636
perf: improve FTS indexing perf and reduce memory footprint by @BubbleCal in https://github.com/lance-format/lance/pull/5650
perf: avoid copying tokens while merging by @BubbleCal in https://github.com/lance-format/lance/pull/5661
perf: tighten WAND block score upper bound by @BubbleCal in https://github.com/lance-format/lance/pull/5668
perf: cache global BM25 idf per query by @BubbleCal in https://github.com/lance-format/lance/pull/5727
perf: use LRU cache for session contexts in get_session_context by @wjones127 in https://github.com/lance-format/lance/pull/5736
perf: merge partitions in stream style by @BubbleCal in https://github.com/lance-format/lance/pull/5754

Other Changes

refactor: write bitmap index statistics in file instead by @Xuanwo in https://github.com/lance-format/lance/pull/5251
refactor: rename RowIdTreeMap to RowAddrTreeMap by @yanghua in https://github.com/lance-format/lance/pull/5266
refactor: rename RowIdMask to RowAddrMask by @yanghua in https://github.com/lance-format/lance/pull/5281
refactor: consolidate logic between zonemap and bloomfilter indexes by @fenfeng9 in https://github.com/lance-format/lance/pull/5374
refactor: split dataset tests in a tests mod by @Xuanwo in https://github.com/lance-format/lance/pull/5387
refactor: use the same path for dedicated and packed blob by @Xuanwo in https://github.com/lance-format/lance/pull/5449
refactor: add store_prefix to lance-io's ObjectStore by @cmccabe in https://github.com/lance-format/lance/pull/5468
refactor: expose take_blobs_by_addresses to python by @Xuanwo in https://github.com/lance-format/lance/pull/5474
refactor: support java 21, drop java 8 by @cmccabe in https://github.com/lance-format/lance/pull/5565
refactor: allow switching to bitpack inside RLE by @Xuanwo in https://github.com/lance-format/lance/pull/5595
refactor: introduce RowSetOps and refactor RowAddrTreeMap by @yanghua in https://github.com/lance-format/lance/pull/5624
refactor(python): migrate torch.jit.script to torch.compile by @wjones127 in https://github.com/lance-format/lance/pull/5759
test: fix tests broken by pandas 3 release by @westonpace in https://github.com/lance-format/lance/pull/5786

Full Changelog: https://github.com/lance-format/lance/compare/release-root/2.0.0-beta.N...v2.0.0-rc.3

Open source →

2.0.0rc223 Jan 2026pre-release

What's Changed

Breaking Changes 🛠

fix!: null handling when using NOT with scalar indices by @wjones127 in https://github.com/lance-format/lance/pull/5270
feat!: track cumulative wall time in analyze plan by @wkalt in https://github.com/lance-format/lance/pull/5505
fix!: check metric compatibility before using vector index by @wjones127 in https://github.com/lance-format/lance/pull/5609
feat!: define default index name and return IndexMetadata after building index by @wjones127 in https://github.com/lance-format/lance/pull/5645
feat!: make v2 manifest default by @wojiaodoubao in https://github.com/lance-format/lance/pull/5656
refactor!: introduce storage options accessor by @jackye1995 in https://github.com/lance-format/lance/pull/5728

New Features 🎉

feat: support using FTS as a filter in vector search by @wojiaodoubao in https://github.com/lance-format/lance/pull/4928
feat: support when_matched_delete in merge_insert by @jtuglu1 in https://github.com/lance-format/lance/pull/4939
feat: add support for large minichunk size (u32) in format v2.2 by @niyue in https://github.com/lance-format/lance/pull/4959
feat: support GEO RTree index by @ddupg in https://github.com/lance-format/lance/pull/5034
feat: support global tag retrieval and improve tag api by @majin1102 in https://github.com/lance-format/lance/pull/5088
feat: support create vector index distributedly by @chenghao-guo in https://github.com/lance-format/lance/pull/5117
feat: support add sub-column to struct col by @wojiaodoubao in https://github.com/lance-format/lance/pull/5126
feat: distributed range-based BTree index by @steFaiz in https://github.com/lance-format/lance/pull/5202
feat: strategized plan compaction by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5233
feat: dataset supports deep_clone by @majin1102 in https://github.com/lance-format/lance/pull/5250
feat: cleanup only scan managed files by @majin1102 in https://github.com/lance-format/lance/pull/5338
feat: support map data type in lance format version 2.2 by @xloya in https://github.com/lance-format/lance/pull/5349
feat: add RTree index spec in table format by @ddupg in https://github.com/lance-format/lance/pull/5360
feat(java): support row lineage and cdf apis by @yanghua in https://github.com/lance-format/lance/pull/5362
feat: disable default features on internal use by @valkum in https://github.com/lance-format/lance/pull/5372
feat(cdf): support set start/end timestamp in cdf by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5378
feat(blob_v2): add external blob support by @Xuanwo in https://github.com/lance-format/lance/pull/5385
feat(blob_v2): add dedicated blob support by @Xuanwo in https://github.com/lance-format/lance/pull/5406
feat: fallback to CPU if GPU accelerating is unavailable by @BubbleCal in https://github.com/lance-format/lance/pull/5407
feat(blob_v2): add packed blob support by @Xuanwo in https://github.com/lance-format/lance/pull/5413
feat: allow python tracing / logging to be independently configured by @westonpace in https://github.com/lance-format/lance/pull/5415
feat: add additional index APIs to support count rows split plan by @jackye1995 in https://github.com/lance-format/lance/pull/5447
feat(java): support multi-bases for writing database by @ddupg in https://github.com/lance-format/lance/pull/5450
feat(blob_v2): add BlobAray API for user input by @Xuanwo in https://github.com/lance-format/lance/pull/5451
feat: upgrade lance-namespace to 0.3.1 and add missing apis by @jackye1995 in https://github.com/lance-format/lance/pull/5457
feat(python): support cleanup_with_policy by @ddupg in https://github.com/lance-format/lance/pull/5458
feat: support dropping sub-column of list(struct) by @wojiaodoubao in https://github.com/lance-format/lance/pull/5469
feat(blob_v2): add GC support by @Xuanwo in https://github.com/lance-format/lance/pull/5473
feat: add py.typed marker file by @jonded94 in https://github.com/lance-format/lance/pull/5479
feat(python): expose the distance_range param in the Python scanner nearest config by @xloya in https://github.com/lance-format/lance/pull/5486
feat(java): simplify the use of optional in jni by @ddupg in https://github.com/lance-format/lance/pull/5488
feat(blob_v2): add Python API for Blob v2 by @Xuanwo in https://github.com/lance-format/lance/pull/5491
feat(python): add DatasetBasePath stub to improve IDE hints by @ddupg in https://github.com/lance-format/lance/pull/5503
feat(memtest): add macos support by @Xuanwo in https://github.com/lance-format/lance/pull/5510
feat(java): add full text search api by @wojiaodoubao in https://github.com/lance-format/lance/pull/5563
feat: support credentials vending in directory namespace by @jackye1995 in https://github.com/lance-format/lance/pull/5566
feat: upgrade lance-namespace to 0.4.0 by @jackye1995 in https://github.com/lance-format/lance/pull/5568
feat: add skip_merge for FTS index build by @BubbleCal in https://github.com/lance-format/lance/pull/5570
feat(java): add builder-style scalar index params by @wojiaodoubao in https://github.com/lance-format/lance/pull/5581
feat: optimize rle implementation by @Xuanwo in https://github.com/lance-format/lance/pull/5586
feat: support FixedSizeList<Struct> by @wkalt in https://github.com/lance-format/lance/pull/5593
feat: add dictionary encoding for 64bit types like int64/double by @Xuanwo in https://github.com/lance-format/lance/pull/5594
feat: support merge_insert with source dedupe on first seen value by @jackye1995 in https://github.com/lance-format/lance/pull/5603
feat: support truncate table api by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5604
feat: add Error::External variant for preserving user errors by @wjones127 in https://github.com/lance-format/lance/pull/5606
feat: upgrade lance-namespace to 0.4.5 by @jackye1995 in https://github.com/lance-format/lance/pull/5611
feat: refactor use of Error::io by @lichuang in https://github.com/lance-format/lance/pull/5612
feat(java): add detached flag to commitTransaction by @wojiaodoubao in https://github.com/lance-format/lance/pull/5626
feat: add parts_searched metrics for FTS by @BubbleCal in https://github.com/lance-format/lance/pull/5627
feat: improve the random access file benchmark by @westonpace in https://github.com/lance-format/lance/pull/5628
feat(oss): add sts token support for aliyun oss via storage_options by @hh23485 in https://github.com/lance-format/lance/pull/5632
feat: merge-insert with primary key dedupe by @jackye1995 in https://github.com/lance-format/lance/pull/5633
feat(java): expose index description and statistics by @majin1102 in https://github.com/lance-format/lance/pull/5655
feat: allow configure temp dir size for datafusion exec by @jackye1995 in https://github.com/lance-format/lance/pull/5659
feat(java): add support for optimizing indices by @majin1102 in https://github.com/lance-format/lance/pull/5663
feat: make on arg optional for merge insert api by @yanghua in https://github.com/lance-format/lance/pull/5667
feat: make OneShotPartitionStream pub by @timsaucer in https://github.com/lance-format/lance/pull/5672
feat: support array_contains in LabelList scalar index by @fenfeng9 in https://github.com/lance-format/lance/pull/5681
feat: add order to primary key by @touch-of-grey in https://github.com/lance-format/lance/pull/5683
feat: use independent region manifest for MemWAL by @touch-of-grey in https://github.com/lance-format/lance/pull/5689
feat: add stats() method to ObjectStoreRegistry by @wkalt in https://github.com/lance-format/lance/pull/5706
feat: support dynamic context for lance namespace by @jackye1995 in https://github.com/lance-format/lance/pull/5710
feat: make blob v2 dedicated threshold configurable by @yanghua in https://github.com/lance-format/lance/pull/5719
feat: cleanup partial idx files when merging distributed vector index by @yanghua in https://github.com/lance-format/lance/pull/5729
feat: expose blob handling APIs to python by @Xuanwo in https://github.com/lance-format/lance/pull/5790

Bug Fixes 🐛

fix: correct null_count aggregation in boolean statistics collection by @YinZheng-Sun in https://github.com/lance-format/lance/pull/4839
fix: remove logging for project_batch by @westonpace in https://github.com/lance-format/lance/pull/5267
fix: stop documenting FTS index type, standardize on INVERTED by @mackrorysd in https://github.com/lance-format/lance/pull/5315
fix: don't allow change blob version during update by @Xuanwo in https://github.com/lance-format/lance/pull/5386
fix: take_blobs_by_indices fails with stable row IDs on fragment 1+ by @jmhsieh in https://github.com/lance-format/lance/pull/5392
fix: respect index metric when user overrides by @BubbleCal in https://github.com/lance-format/lance/pull/5395
fix: remove expensive clone in bitmap search by @westonpace in https://github.com/lance-format/lance/pull/5409
fix: fix vector index prewarm index by @xloya in https://github.com/lance-format/lance/pull/5412
fix: panic unwrap on None in decoder.rs by @camilesing in https://github.com/lance-format/lance/pull/5424
fix: dir namespace cloud storage path removes one subdir level by @jackye1995 in https://github.com/lance-format/lance/pull/5464
fix: make column name lookups case-insensitive by @wjones127 in https://github.com/lance-format/lance/pull/5465
fix: ensure trailing slash is normalized in rest adapter by @jackye1995 in https://github.com/lance-format/lance/pull/5499
fix(java): support FixedSizeList for java LanceField by @fangbo in https://github.com/lance-format/lance/pull/5509
fix: head external manifest object happend 404 NotFound error by @hushengquan in https://github.com/lance-format/lance/pull/5512
fix: json's arrow extension metadata missing by @Xuanwo in https://github.com/lance-format/lance/pull/5527
fix: infer multivector sampling rows by @BubbleCal in https://github.com/lance-format/lance/pull/5534
fix: support ManifestNamingSchemeV2 with unordered object stores by @wjones127 in https://github.com/lance-format/lance/pull/5539
fix: merge_insert uses full schema path for reordered columns by @wjones127 in https://github.com/lance-format/lance/pull/5541
fix: allow storage options provider without expires_at_millis by @jackye1995 in https://github.com/lance-format/lance/pull/5542
fix(ci): use pull_request_target for fork PR reviews by @wjones127 in https://github.com/lance-format/lance/pull/5544
fix: restore decrease max_fragment_id in manifest by @majin1102 in https://github.com/lance-format/lance/pull/5554
fix: improve error handling for environment variable parsing by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/5560
fix: panic when lance.auto_cleanup.interval is set to 0 by @majin1102 in https://github.com/lance-format/lance/pull/5571
fix(python): correct type hint for to_tensor_fn parameter by @AndreaBozzo in https://github.com/lance-format/lance/pull/5577
fix: avoid panic while hitting non-null empty multi-vector by @Xuanwo in https://github.com/lance-format/lance/pull/5588
fix: filter garbage entries from null maps during encoding by @wkalt in https://github.com/lance-format/lance/pull/5591
fix: reduce verbosity of errors due to string conversion by @wjones127 in https://github.com/lance-format/lance/pull/5600
fix: remove imports that are not needed by @westonpace in https://github.com/lance-format/lance/pull/5651
fix: allow nearest applied in default_scan_options by @chenghao-guo in https://github.com/lance-format/lance/pull/5666
fix: trait Array has been sealed in arrow new version by @Xuanwo in https://github.com/lance-format/lance/pull/5690
fix: project_by_schema now reorders fields inside List<Struct> types by @wjones127 in https://github.com/lance-format/lance/pull/5703
fix: allocate too much memory for block max scores by @BubbleCal in https://github.com/lance-format/lance/pull/5718
docs: in dataset.rs, fix comment for get_fragments by @cmccabe in https://github.com/lance-format/lance/pull/5724
fix(python): close SQLite connections in BatchUDFCheckpoint by @wjones127 in https://github.com/lance-format/lance/pull/5733
fix: remove credential vending features from python and java bindings by @jackye1995 in https://github.com/lance-format/lance/pull/5737
fix: allow unused_unsafe for __cpuid to support both stable and nightly by @jackye1995 in https://github.com/lance-format/lance/pull/5793

Documentation 📚

docs: fix Append call in distributed write guide by @rongou in https://github.com/lance-format/lance/pull/5439
docs: fix and improve the description about row id by @yanghua in https://github.com/lance-format/lance/pull/5463
docs: add specification for handling indices by @wjones127 in https://github.com/lance-format/lance/pull/5543
docs: fix duplicate words in comments and error messages by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/5548
docs: add research paper link to the landing page by @prrao87 in https://github.com/lance-format/lance/pull/5549
docs: auto-build refactored namespace integrations doc by @jackye1995 in https://github.com/lance-format/lance/pull/5562
docs: rename RowIdTreeMap to RowAddrTreeMap in rtree.md by @ddupg in https://github.com/lance-format/lance/pull/5564
docs: add docs for DuckDB extension by @prrao87 in https://github.com/lance-format/lance/pull/5578
docs: update Lance-DuckDB docs to latest version 0.4.1 by @prrao87 in https://github.com/lance-format/lance/pull/5613

Performance Improvements 🚀

perf: do not instrument self in multipart upload by @westonpace in https://github.com/lance-format/lance/pull/5416
perf: various btree performance improvements by @westonpace in https://github.com/lance-format/lance/pull/5446
perf: reuse session context by @wjones127 in https://github.com/lance-format/lance/pull/5462
perf: offload IVF partition build to CPU pool by @BubbleCal in https://github.com/lance-format/lance/pull/5551
perf: materialize the tokens after WAND done by @BubbleCal in https://github.com/lance-format/lance/pull/5572
perf: compute HNSW level counts after build by @BubbleCal in https://github.com/lance-format/lance/pull/5590
perf: improve SQ query speed by @BubbleCal in https://github.com/lance-format/lance/pull/5596
perf: reuse zstd compressors in encoding by @wkalt in https://github.com/lance-format/lance/pull/5598
perf: use binary search to skip documents by @BubbleCal in https://github.com/lance-format/lance/pull/5636
perf: improve FTS indexing perf and reduce memory footprint by @BubbleCal in https://github.com/lance-format/lance/pull/5650
perf: avoid copying tokens while merging by @BubbleCal in https://github.com/lance-format/lance/pull/5661
perf: tighten WAND block score upper bound by @BubbleCal in https://github.com/lance-format/lance/pull/5668
perf: cache global BM25 idf per query by @BubbleCal in https://github.com/lance-format/lance/pull/5727
perf: use LRU cache for session contexts in get_session_context by @wjones127 in https://github.com/lance-format/lance/pull/5736
perf: merge partitions in stream style by @BubbleCal in https://github.com/lance-format/lance/pull/5754

Other Changes

refactor: write bitmap index statistics in file instead by @Xuanwo in https://github.com/lance-format/lance/pull/5251
refactor: rename RowIdTreeMap to RowAddrTreeMap by @yanghua in https://github.com/lance-format/lance/pull/5266
refactor: rename RowIdMask to RowAddrMask by @yanghua in https://github.com/lance-format/lance/pull/5281
refactor: consolidate logic between zonemap and bloomfilter indexes by @fenfeng9 in https://github.com/lance-format/lance/pull/5374
refactor: split dataset tests in a tests mod by @Xuanwo in https://github.com/lance-format/lance/pull/5387
refactor: use the same path for dedicated and packed blob by @Xuanwo in https://github.com/lance-format/lance/pull/5449
refactor: add store_prefix to lance-io's ObjectStore by @cmccabe in https://github.com/lance-format/lance/pull/5468
refactor: expose take_blobs_by_addresses to python by @Xuanwo in https://github.com/lance-format/lance/pull/5474
refactor: support java 21, drop java 8 by @cmccabe in https://github.com/lance-format/lance/pull/5565
refactor: allow switching to bitpack inside RLE by @Xuanwo in https://github.com/lance-format/lance/pull/5595
refactor: introduce RowSetOps and refactor RowAddrTreeMap by @yanghua in https://github.com/lance-format/lance/pull/5624
refactor(python): migrate torch.jit.script to torch.compile by @wjones127 in https://github.com/lance-format/lance/pull/5759
test: fix tests broken by pandas 3 release by @westonpace in https://github.com/lance-format/lance/pull/5786

Full Changelog: https://github.com/lance-format/lance/compare/release-root/2.0.0-beta.N...v2.0.0-rc.2

Open source →

2.0.0rc120 Jan 2026pre-release

What's Changed

Breaking Changes 🛠

fix!: null handling when using NOT with scalar indices by @wjones127 in https://github.com/lance-format/lance/pull/5270
feat!: track cumulative wall time in analyze plan by @wkalt in https://github.com/lance-format/lance/pull/5505
fix!: check metric compatibility before using vector index by @wjones127 in https://github.com/lance-format/lance/pull/5609
feat!: define default index name and return IndexMetadata after building index by @wjones127 in https://github.com/lance-format/lance/pull/5645
feat!: make v2 manifest default by @wojiaodoubao in https://github.com/lance-format/lance/pull/5656
refactor!: introduce storage options accessor by @jackye1995 in https://github.com/lance-format/lance/pull/5728

New Features 🎉

feat: support using FTS as a filter in vector search by @wojiaodoubao in https://github.com/lance-format/lance/pull/4928
feat: support when_matched_delete in merge_insert by @jtuglu1 in https://github.com/lance-format/lance/pull/4939
feat: add support for large minichunk size (u32) in format v2.2 by @niyue in https://github.com/lance-format/lance/pull/4959
feat: support GEO RTree index by @ddupg in https://github.com/lance-format/lance/pull/5034
feat: support global tag retrieval and improve tag api by @majin1102 in https://github.com/lance-format/lance/pull/5088
feat: support create vector index distributedly by @chenghao-guo in https://github.com/lance-format/lance/pull/5117
feat: support add sub-column to struct col by @wojiaodoubao in https://github.com/lance-format/lance/pull/5126
feat: distributed range-based BTree index by @steFaiz in https://github.com/lance-format/lance/pull/5202
feat: strategized plan compaction by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5233
feat: dataset supports deep_clone by @majin1102 in https://github.com/lance-format/lance/pull/5250
feat: cleanup only scan managed files by @majin1102 in https://github.com/lance-format/lance/pull/5338
feat: support map data type in lance format version 2.2 by @xloya in https://github.com/lance-format/lance/pull/5349
feat: add RTree index spec in table format by @ddupg in https://github.com/lance-format/lance/pull/5360
feat(java): support row lineage and cdf apis by @yanghua in https://github.com/lance-format/lance/pull/5362
feat: disable default features on internal use by @valkum in https://github.com/lance-format/lance/pull/5372
feat(cdf): support set start/end timestamp in cdf by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5378
feat(blob_v2): add external blob support by @Xuanwo in https://github.com/lance-format/lance/pull/5385
feat(blob_v2): add dedicated blob support by @Xuanwo in https://github.com/lance-format/lance/pull/5406
feat: fallback to CPU if GPU accelerating is unavailable by @BubbleCal in https://github.com/lance-format/lance/pull/5407
feat(blob_v2): add packed blob support by @Xuanwo in https://github.com/lance-format/lance/pull/5413
feat: allow python tracing / logging to be independently configured by @westonpace in https://github.com/lance-format/lance/pull/5415
feat: add additional index APIs to support count rows split plan by @jackye1995 in https://github.com/lance-format/lance/pull/5447
feat(java): support multi-bases for writing database by @ddupg in https://github.com/lance-format/lance/pull/5450
feat(blob_v2): add BlobAray API for user input by @Xuanwo in https://github.com/lance-format/lance/pull/5451
feat: upgrade lance-namespace to 0.3.1 and add missing apis by @jackye1995 in https://github.com/lance-format/lance/pull/5457
feat(python): support cleanup_with_policy by @ddupg in https://github.com/lance-format/lance/pull/5458
feat: support dropping sub-column of list(struct) by @wojiaodoubao in https://github.com/lance-format/lance/pull/5469
feat(blob_v2): add GC support by @Xuanwo in https://github.com/lance-format/lance/pull/5473
feat: add py.typed marker file by @jonded94 in https://github.com/lance-format/lance/pull/5479
feat(python): expose the distance_range param in the Python scanner nearest config by @xloya in https://github.com/lance-format/lance/pull/5486
feat(java): simplify the use of optional in jni by @ddupg in https://github.com/lance-format/lance/pull/5488
feat(blob_v2): add Python API for Blob v2 by @Xuanwo in https://github.com/lance-format/lance/pull/5491
feat(python): add DatasetBasePath stub to improve IDE hints by @ddupg in https://github.com/lance-format/lance/pull/5503
feat(memtest): add macos support by @Xuanwo in https://github.com/lance-format/lance/pull/5510
feat(java): add full text search api by @wojiaodoubao in https://github.com/lance-format/lance/pull/5563
feat: support credentials vending in directory namespace by @jackye1995 in https://github.com/lance-format/lance/pull/5566
feat: upgrade lance-namespace to 0.4.0 by @jackye1995 in https://github.com/lance-format/lance/pull/5568
feat: add skip_merge for FTS index build by @BubbleCal in https://github.com/lance-format/lance/pull/5570
feat(java): add builder-style scalar index params by @wojiaodoubao in https://github.com/lance-format/lance/pull/5581
feat: optimize rle implementation by @Xuanwo in https://github.com/lance-format/lance/pull/5586
feat: support FixedSizeList<Struct> by @wkalt in https://github.com/lance-format/lance/pull/5593
feat: add dictionary encoding for 64bit types like int64/double by @Xuanwo in https://github.com/lance-format/lance/pull/5594
feat: support merge_insert with source dedupe on first seen value by @jackye1995 in https://github.com/lance-format/lance/pull/5603
feat: support truncate table api by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5604
feat: add Error::External variant for preserving user errors by @wjones127 in https://github.com/lance-format/lance/pull/5606
feat: upgrade lance-namespace to 0.4.5 by @jackye1995 in https://github.com/lance-format/lance/pull/5611
feat: refactor use of Error::io by @lichuang in https://github.com/lance-format/lance/pull/5612
feat(java): add detached flag to commitTransaction by @wojiaodoubao in https://github.com/lance-format/lance/pull/5626
feat: add parts_searched metrics for FTS by @BubbleCal in https://github.com/lance-format/lance/pull/5627
feat: improve the random access file benchmark by @westonpace in https://github.com/lance-format/lance/pull/5628
feat(oss): add sts token support for aliyun oss via storage_options by @hh23485 in https://github.com/lance-format/lance/pull/5632
feat: merge-insert with primary key dedupe by @jackye1995 in https://github.com/lance-format/lance/pull/5633
feat(java): expose index description and statistics by @majin1102 in https://github.com/lance-format/lance/pull/5655
feat: allow configure temp dir size for datafusion exec by @jackye1995 in https://github.com/lance-format/lance/pull/5659
feat(java): add support for optimizing indices by @majin1102 in https://github.com/lance-format/lance/pull/5663
feat: make on arg optional for merge insert api by @yanghua in https://github.com/lance-format/lance/pull/5667
feat: make OneShotPartitionStream pub by @timsaucer in https://github.com/lance-format/lance/pull/5672
feat: support array_contains in LabelList scalar index by @fenfeng9 in https://github.com/lance-format/lance/pull/5681
feat: add order to primary key by @touch-of-grey in https://github.com/lance-format/lance/pull/5683
feat: use independent region manifest for MemWAL by @touch-of-grey in https://github.com/lance-format/lance/pull/5689
feat: add stats() method to ObjectStoreRegistry by @wkalt in https://github.com/lance-format/lance/pull/5706
feat: support dynamic context for lance namespace by @jackye1995 in https://github.com/lance-format/lance/pull/5710
feat: make blob v2 dedicated threshold configurable by @yanghua in https://github.com/lance-format/lance/pull/5719
feat: cleanup partial idx files when merging distributed vector index by @yanghua in https://github.com/lance-format/lance/pull/5729

Bug Fixes 🐛

fix: correct null_count aggregation in boolean statistics collection by @YinZheng-Sun in https://github.com/lance-format/lance/pull/4839
fix: remove logging for project_batch by @westonpace in https://github.com/lance-format/lance/pull/5267
fix: stop documenting FTS index type, standardize on INVERTED by @mackrorysd in https://github.com/lance-format/lance/pull/5315
fix: don't allow change blob version during update by @Xuanwo in https://github.com/lance-format/lance/pull/5386
fix: take_blobs_by_indices fails with stable row IDs on fragment 1+ by @jmhsieh in https://github.com/lance-format/lance/pull/5392
fix: respect index metric when user overrides by @BubbleCal in https://github.com/lance-format/lance/pull/5395
fix: remove expensive clone in bitmap search by @westonpace in https://github.com/lance-format/lance/pull/5409
fix: fix vector index prewarm index by @xloya in https://github.com/lance-format/lance/pull/5412
fix: panic unwrap on None in decoder.rs by @camilesing in https://github.com/lance-format/lance/pull/5424
fix: dir namespace cloud storage path removes one subdir level by @jackye1995 in https://github.com/lance-format/lance/pull/5464
fix: make column name lookups case-insensitive by @wjones127 in https://github.com/lance-format/lance/pull/5465
fix: ensure trailing slash is normalized in rest adapter by @jackye1995 in https://github.com/lance-format/lance/pull/5499
fix(java): support FixedSizeList for java LanceField by @fangbo in https://github.com/lance-format/lance/pull/5509
fix: head external manifest object happend 404 NotFound error by @hushengquan in https://github.com/lance-format/lance/pull/5512
fix: json's arrow extension metadata missing by @Xuanwo in https://github.com/lance-format/lance/pull/5527
fix: infer multivector sampling rows by @BubbleCal in https://github.com/lance-format/lance/pull/5534
fix: support ManifestNamingSchemeV2 with unordered object stores by @wjones127 in https://github.com/lance-format/lance/pull/5539
fix: merge_insert uses full schema path for reordered columns by @wjones127 in https://github.com/lance-format/lance/pull/5541
fix: allow storage options provider without expires_at_millis by @jackye1995 in https://github.com/lance-format/lance/pull/5542
fix(ci): use pull_request_target for fork PR reviews by @wjones127 in https://github.com/lance-format/lance/pull/5544
fix: restore decrease max_fragment_id in manifest by @majin1102 in https://github.com/lance-format/lance/pull/5554
fix: improve error handling for environment variable parsing by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/5560
fix: panic when lance.auto_cleanup.interval is set to 0 by @majin1102 in https://github.com/lance-format/lance/pull/5571
fix(python): correct type hint for to_tensor_fn parameter by @AndreaBozzo in https://github.com/lance-format/lance/pull/5577
fix: avoid panic while hitting non-null empty multi-vector by @Xuanwo in https://github.com/lance-format/lance/pull/5588
fix: filter garbage entries from null maps during encoding by @wkalt in https://github.com/lance-format/lance/pull/5591
fix: reduce verbosity of errors due to string conversion by @wjones127 in https://github.com/lance-format/lance/pull/5600
fix: remove imports that are not needed by @westonpace in https://github.com/lance-format/lance/pull/5651
fix: allow nearest applied in default_scan_options by @chenghao-guo in https://github.com/lance-format/lance/pull/5666
fix: trait Array has been sealed in arrow new version by @Xuanwo in https://github.com/lance-format/lance/pull/5690
fix: project_by_schema now reorders fields inside List<Struct> types by @wjones127 in https://github.com/lance-format/lance/pull/5703
fix: allocate too much memory for block max scores by @BubbleCal in https://github.com/lance-format/lance/pull/5718
docs: in dataset.rs, fix comment for get_fragments by @cmccabe in https://github.com/lance-format/lance/pull/5724
fix(python): close SQLite connections in BatchUDFCheckpoint by @wjones127 in https://github.com/lance-format/lance/pull/5733
fix: remove credential vending features from python and java bindings by @jackye1995 in https://github.com/lance-format/lance/pull/5737

Documentation 📚

docs: fix Append call in distributed write guide by @rongou in https://github.com/lance-format/lance/pull/5439
docs: fix and improve the description about row id by @yanghua in https://github.com/lance-format/lance/pull/5463
docs: add specification for handling indices by @wjones127 in https://github.com/lance-format/lance/pull/5543
docs: fix duplicate words in comments and error messages by @XuQianJin-Stars in https://github.com/lance-format/lance/pull/5548
docs: add research paper link to the landing page by @prrao87 in https://github.com/lance-format/lance/pull/5549
docs: auto-build refactored namespace integrations doc by @jackye1995 in https://github.com/lance-format/lance/pull/5562
docs: rename RowIdTreeMap to RowAddrTreeMap in rtree.md by @ddupg in https://github.com/lance-format/lance/pull/5564
docs: add docs for DuckDB extension by @prrao87 in https://github.com/lance-format/lance/pull/5578
docs: update Lance-DuckDB docs to latest version 0.4.1 by @prrao87 in https://github.com/lance-format/lance/pull/5613

Performance Improvements 🚀

perf: do not instrument self in multipart upload by @westonpace in https://github.com/lance-format/lance/pull/5416
perf: various btree performance improvements by @westonpace in https://github.com/lance-format/lance/pull/5446
perf: reuse session context by @wjones127 in https://github.com/lance-format/lance/pull/5462
perf: offload IVF partition build to CPU pool by @BubbleCal in https://github.com/lance-format/lance/pull/5551
perf: materialize the tokens after WAND done by @BubbleCal in https://github.com/lance-format/lance/pull/5572
perf: compute HNSW level counts after build by @BubbleCal in https://github.com/lance-format/lance/pull/5590
perf: improve SQ query speed by @BubbleCal in https://github.com/lance-format/lance/pull/5596
perf: reuse zstd compressors in encoding by @wkalt in https://github.com/lance-format/lance/pull/5598
perf: use binary search to skip documents by @BubbleCal in https://github.com/lance-format/lance/pull/5636
perf: improve FTS indexing perf and reduce memory footprint by @BubbleCal in https://github.com/lance-format/lance/pull/5650
perf: avoid copying tokens while merging by @BubbleCal in https://github.com/lance-format/lance/pull/5661
perf: tighten WAND block score upper bound by @BubbleCal in https://github.com/lance-format/lance/pull/5668
perf: cache global BM25 idf per query by @BubbleCal in https://github.com/lance-format/lance/pull/5727
perf: use LRU cache for session contexts in get_session_context by @wjones127 in https://github.com/lance-format/lance/pull/5736
perf: merge partitions in stream style by @BubbleCal in https://github.com/lance-format/lance/pull/5754

Other Changes

refactor: write bitmap index statistics in file instead by @Xuanwo in https://github.com/lance-format/lance/pull/5251
refactor: rename RowIdTreeMap to RowAddrTreeMap by @yanghua in https://github.com/lance-format/lance/pull/5266
refactor: rename RowIdMask to RowAddrMask by @yanghua in https://github.com/lance-format/lance/pull/5281
refactor: consolidate logic between zonemap and bloomfilter indexes by @fenfeng9 in https://github.com/lance-format/lance/pull/5374
refactor: split dataset tests in a tests mod by @Xuanwo in https://github.com/lance-format/lance/pull/5387
refactor: use the same path for dedicated and packed blob by @Xuanwo in https://github.com/lance-format/lance/pull/5449
refactor: add store_prefix to lance-io's ObjectStore by @cmccabe in https://github.com/lance-format/lance/pull/5468
refactor: expose take_blobs_by_addresses to python by @Xuanwo in https://github.com/lance-format/lance/pull/5474
refactor: support java 21, drop java 8 by @cmccabe in https://github.com/lance-format/lance/pull/5565
refactor: allow switching to bitpack inside RLE by @Xuanwo in https://github.com/lance-format/lance/pull/5595
refactor: introduce RowSetOps and refactor RowAddrTreeMap by @yanghua in https://github.com/lance-format/lance/pull/5624

Full Changelog: https://github.com/lance-format/lance/compare/release-root/2.0.0-beta.N...v2.0.0-rc.1

Open source →

1.0.012 Dec 2025Deprecated

refactor!: deprecate TFRecord support by @jackye1995 in https://github.com/lance-format/lance/pull/4593

What's Changed

Breaking Changes 🛠

perf!: dynamic pruning for vector search by @BubbleCal in https://github.com/lance-format/lance/pull/4773
feat!: remove unnecessary mut of dataset::sql by @ddupg in https://github.com/lance-format/lance/pull/5207
refactor!: move all previous code into previous mod by @Xuanwo in https://github.com/lance-format/lance/pull/5217
refactor!: deprecate TFRecord support by @jackye1995 in https://github.com/lance-format/lance/pull/4593
refactor!: use org.lance namespace for java package by @jackye1995 in https://github.com/lance-format/lance/pull/5339
refactor!: deprecate mac x86 support by @jackye1995 in https://github.com/lance-format/lance/pull/5391

Critical Fixes ‼️

fix: convert some panics into errors by @wjones127 in https://github.com/lance-format/lance/pull/5258

New Features 🎉

feat: provide inline_transaction model for IO optimizing by @majin1102 in https://github.com/lance-format/lance/pull/4774
feat(python): add support for HuggingFace IterableDataset by @changhiskhan in https://github.com/lance-format/lance/pull/2599
feat: add public accessors for count plan construction by @wkalt in https://github.com/lance-format/lance/pull/5103
feat: add adapter for REST namespace with manifest namespace backend by @jackye1995 in https://github.com/lance-format/lance/pull/4984
feat: add blob compaction support by @Xuanwo in https://github.com/lance-format/lance/pull/5189
feat: add inline optimization for dir namespace by @jackye1995 in https://github.com/lance-format/lance/pull/5244
feat: conflict resolution for DataReplacement by @wjones127 in https://github.com/lance-format/lance/pull/3631
feat: support namespace vended credentials for write by @jackye1995 in https://github.com/lance-format/lance/pull/5161
docs: correct the comment in util.py by @shiyajuan123 in https://github.com/lance-format/lance/pull/5252
feat: introduce community governance by @jackye1995 in https://github.com/lance-format/lance/pull/5262
feat: add describe_indices function by @westonpace in https://github.com/lance-format/lance/pull/5221
feat: add target_bases extension to python write_fragments API by @jaystarshot in https://github.com/lance-format/lance/pull/5234
feat: support credentials vending for file reader and session by @jackye1995 in https://github.com/lance-format/lance/pull/5256
feat: introduce blob arrow extension type by @Xuanwo in https://github.com/lance-format/lance/pull/5239
feat(java): add binding for rest and dir namespaces by @jackye1995 in https://github.com/lance-format/lance/pull/5292
feat: expose file upload and download in Lance file session by @jackye1995 in https://github.com/lance-format/lance/pull/5336
feat(java): support credential vending at write time by @jackye1995 in https://github.com/lance-format/lance/pull/5309
feat(ds-sql-api): support JSON bulit in functions in ds.sql API by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5350
feat: support GEO types by @ddupg in https://github.com/lance-format/lance/pull/4678
feat(python): expose DatasetDeltaBuilder and relevant apis by @yanghua in https://github.com/lance-format/lance/pull/5091
feat: return Unprocessable error while expected error happened by @Xuanwo in https://github.com/lance-format/lance/pull/5347
feat: add huggingface native support by @Xuanwo in https://github.com/lance-format/lance/pull/5353
feat: dynamically choose distance type by @BubbleCal in https://github.com/lance-format/lance/pull/5370
feat(java): support writing schema metadata through java LanceFileWriter API by @steFaiz in https://github.com/lance-format/lance/pull/5310

Bug Fixes 🐛

fix: ensure recheck for IsNotNull in bloom filter by @Xuanwo in https://github.com/lance-format/lance/pull/5192
fix: contributing URL gives 404 by @prrao87 in https://github.com/lance-format/lance/pull/5196
fix: merge struct array use wrong child values by @wojiaodoubao in https://github.com/lance-format/lance/pull/5106
fix: avoid unnecessary get_fragments calling during plan compaction by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5179
fix: split partition may be assigned to itself by @BubbleCal in https://github.com/lance-format/lance/pull/5190
fix: improve schema validation for nullability and subschemas by @fenfeng9 in https://github.com/lance-format/lance/pull/4994
fix: compile error in test_inline_transaction by @wojiaodoubao in https://github.com/lance-format/lance/pull/5206
docs: fix batch udf with checkpoint document error by @fangyinc in https://github.com/lance-format/lance/pull/5185
fix: memory-limited string BTREE index building by @wjones127 in https://github.com/lance-format/lance/pull/5175
fix: home page code snippets cuasing various problems by @jackye1995 in https://github.com/lance-format/lance/pull/5245
fix: panic if only one partition and split is triggered by @BubbleCal in https://github.com/lance-format/lance/pull/5241
fix: clearer error in dataset take by @wkalt in https://github.com/lance-format/lance/pull/5243
fix: docs and comment have broken links by @prrao87 in https://github.com/lance-format/lance/pull/5261
fix: handle logical rows deletion properly for zonemap and bloomfilter by @HaochengLIU in https://github.com/lance-format/lance/pull/5140
fix: blob version should be passed in Projection by @Xuanwo in https://github.com/lance-format/lance/pull/5295
docs: fix broken links and 404s by @prrao87 in https://github.com/lance-format/lance/pull/5284
fix: index overestimates the posting list size by @BubbleCal in https://github.com/lance-format/lance/pull/5327
fix: update CachedFileMetadata version API to V2.0 by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5330
fix: correctly handle OSS commit protocol to prevent data loss by @Pmathsun in https://github.com/lance-format/lance/pull/5332
fix: update btree index with its own zone size instead of DEFAULT_BTR… by @ztorchan in https://github.com/lance-format/lance/pull/5301
fix: join job may cause inconsistent delta indices by @BubbleCal in https://github.com/lance-format/lance/pull/5328
fix: panicException when calling compaction by @yanghua in https://github.com/lance-format/lance/pull/5282
fix: parallelize bitmap partition loading in IsIn expressions by @wkalt in https://github.com/lance-format/lance/pull/5355
fix: avx512 related symbol not found in mac x86 by @jackye1995 in https://github.com/lance-format/lance/pull/5379
fix: add graceful shutdown and start for rest namespace adapter by @jackye1995 in https://github.com/lance-format/lance/pull/5325
fix: respect index metric when user overrides by @BubbleCal in https://github.com/lance-format/lance/pull/5397
fix: critical fixes for 1.0.0-rc3 by @jackye1995 in https://github.com/lance-format/lance/pull/5421

Documentation 📚

docs: introduce lance as a lakehouse format by @jackye1995 in https://github.com/lance-format/lance/pull/5209
docs: update readme with latest lakehouse format info by @jackye1995 in https://github.com/lance-format/lance/pull/5216
docs: capitalization change by @timsaucer in https://github.com/lance-format/lance/pull/5269
docs: build project specific guidelines into web doc by @jackye1995 in https://github.com/lance-format/lance/pull/5324

Performance Improvements 🚀

perf: speed up filtered scan by up to 18.9× by moving the heavy CPU task out by @Xuanwo in https://github.com/lance-format/lance/pull/5165
perf: parallelize split job assigning by @BubbleCal in https://github.com/lance-format/lance/pull/5265
perf: use CPU pool to run WAND algo by @BubbleCal in https://github.com/lance-format/lance/pull/5363
perf: avoid allocating filtered nodes on HNSW search path by @BubbleCal in https://github.com/lance-format/lance/pull/5377

Other Changes

refactor: move blob version as a table level config by @Xuanwo in https://github.com/lance-format/lance/pull/5220
refactor: add helper functions to delta.rs tests by @yanghua in https://github.com/lance-format/lance/pull/5298
refactor: move LanceNamespace interface to pylance and java lance-core by @jackye1995 in https://github.com/lance-format/lance/pull/5345
refactor: allow datafiles to contain columns without field id by @Xuanwo in https://github.com/lance-format/lance/pull/5348
refactor: rename RowIdSelection to RowAddrSelection by @yanghua in https://github.com/lance-format/lance/pull/5263
refactor: separate out python and java LanceNamespace interface by @jackye1995 in https://github.com/lance-format/lance/pull/5364
refactor: align with blob v2 logical types change by @Xuanwo in https://github.com/lance-format/lance/pull/5375

New Contributors

@prrao87 made their first contribution in https://github.com/lance-format/lance/pull/5196
@fangyinc made their first contribution in https://github.com/lance-format/lance/pull/5185
@shiyajuan123 made their first contribution in https://github.com/lance-format/lance/pull/5252
@Pmathsun made their first contribution in https://github.com/lance-format/lance/pull/5332
@ztorchan made their first contribution in https://github.com/lance-format/lance/pull/5301
@fMeow made their first contribution in https://github.com/lance-format/lance/pull/5371

Full Changelog: https://github.com/lance-format/lance/compare/release-root/1.0.0-beta.N...v1.0.0

Open source →

1.0.0rc35 Dec 2025pre-releaseDeprecated

refactor!: deprecate TFRecord support by @jackye1995 in https://github.com/lance-format/lance/pull/4593

What's Changed

Breaking Changes 🛠

perf!: dynamic pruning for vector search by @BubbleCal in https://github.com/lance-format/lance/pull/4773
feat!: remove unnecessary mut of dataset::sql by @ddupg in https://github.com/lance-format/lance/pull/5207
refactor!: move all previous code into previous mod by @Xuanwo in https://github.com/lance-format/lance/pull/5217
refactor!: deprecate TFRecord support by @jackye1995 in https://github.com/lance-format/lance/pull/4593
refactor!: use org.lance namespace for java package by @jackye1995 in https://github.com/lance-format/lance/pull/5339
refactor!: deprecate mac x86 support by @jackye1995 in https://github.com/lance-format/lance/pull/5391

Critical Fixes ‼️

fix: convert some panics into errors by @wjones127 in https://github.com/lance-format/lance/pull/5258

New Features 🎉

feat: provide inline_transaction model for IO optimizing by @majin1102 in https://github.com/lance-format/lance/pull/4774
feat(python): add support for HuggingFace IterableDataset by @changhiskhan in https://github.com/lance-format/lance/pull/2599
feat: add public accessors for count plan construction by @wkalt in https://github.com/lance-format/lance/pull/5103
feat: add adapter for REST namespace with manifest namespace backend by @jackye1995 in https://github.com/lance-format/lance/pull/4984
feat: add blob compaction support by @Xuanwo in https://github.com/lance-format/lance/pull/5189
feat: add inline optimization for dir namespace by @jackye1995 in https://github.com/lance-format/lance/pull/5244
feat: conflict resolution for DataReplacement by @wjones127 in https://github.com/lance-format/lance/pull/3631
feat: support namespace vended credentials for write by @jackye1995 in https://github.com/lance-format/lance/pull/5161
docs: correct the comment in util.py by @shiyajuan123 in https://github.com/lance-format/lance/pull/5252
feat: introduce community governance by @jackye1995 in https://github.com/lance-format/lance/pull/5262
feat: add describe_indices function by @westonpace in https://github.com/lance-format/lance/pull/5221
feat: add target_bases extension to python write_fragments API by @jaystarshot in https://github.com/lance-format/lance/pull/5234
feat: support credentials vending for file reader and session by @jackye1995 in https://github.com/lance-format/lance/pull/5256
feat: introduce blob arrow extension type by @Xuanwo in https://github.com/lance-format/lance/pull/5239
feat(java): add binding for rest and dir namespaces by @jackye1995 in https://github.com/lance-format/lance/pull/5292
feat: expose file upload and download in Lance file session by @jackye1995 in https://github.com/lance-format/lance/pull/5336
feat(java): support credential vending at write time by @jackye1995 in https://github.com/lance-format/lance/pull/5309
feat(ds-sql-api): support JSON bulit in functions in ds.sql API by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5350
feat: support GEO types by @ddupg in https://github.com/lance-format/lance/pull/4678
feat(python): expose DatasetDeltaBuilder and relevant apis by @yanghua in https://github.com/lance-format/lance/pull/5091
feat: return Unprocessable error while expected error happened by @Xuanwo in https://github.com/lance-format/lance/pull/5347
feat: add huggingface native support by @Xuanwo in https://github.com/lance-format/lance/pull/5353
feat: dynamically choose distance type by @BubbleCal in https://github.com/lance-format/lance/pull/5370
feat(java): support writing schema metadata through java LanceFileWriter API by @steFaiz in https://github.com/lance-format/lance/pull/5310

Bug Fixes 🐛

fix: ensure recheck for IsNotNull in bloom filter by @Xuanwo in https://github.com/lance-format/lance/pull/5192
fix: contributing URL gives 404 by @prrao87 in https://github.com/lance-format/lance/pull/5196
fix: merge struct array use wrong child values by @wojiaodoubao in https://github.com/lance-format/lance/pull/5106
fix: avoid unnecessary get_fragments calling during plan compaction by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5179
fix: split partition may be assigned to itself by @BubbleCal in https://github.com/lance-format/lance/pull/5190
fix: improve schema validation for nullability and subschemas by @fenfeng9 in https://github.com/lance-format/lance/pull/4994
fix: compile error in test_inline_transaction by @wojiaodoubao in https://github.com/lance-format/lance/pull/5206
docs: fix batch udf with checkpoint document error by @fangyinc in https://github.com/lance-format/lance/pull/5185
fix: memory-limited string BTREE index building by @wjones127 in https://github.com/lance-format/lance/pull/5175
fix: home page code snippets cuasing various problems by @jackye1995 in https://github.com/lance-format/lance/pull/5245
fix: panic if only one partition and split is triggered by @BubbleCal in https://github.com/lance-format/lance/pull/5241
fix: clearer error in dataset take by @wkalt in https://github.com/lance-format/lance/pull/5243
fix: docs and comment have broken links by @prrao87 in https://github.com/lance-format/lance/pull/5261
fix: handle logical rows deletion properly for zonemap and bloomfilter by @HaochengLIU in https://github.com/lance-format/lance/pull/5140
fix: blob version should be passed in Projection by @Xuanwo in https://github.com/lance-format/lance/pull/5295
docs: fix broken links and 404s by @prrao87 in https://github.com/lance-format/lance/pull/5284
fix: index overestimates the posting list size by @BubbleCal in https://github.com/lance-format/lance/pull/5327
fix: update CachedFileMetadata version API to V2.0 by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5330
fix: correctly handle OSS commit protocol to prevent data loss by @Pmathsun in https://github.com/lance-format/lance/pull/5332
fix: update btree index with its own zone size instead of DEFAULT_BTR… by @ztorchan in https://github.com/lance-format/lance/pull/5301
fix: join job may cause inconsistent delta indices by @BubbleCal in https://github.com/lance-format/lance/pull/5328
fix: panicException when calling compaction by @yanghua in https://github.com/lance-format/lance/pull/5282
fix: parallelize bitmap partition loading in IsIn expressions by @wkalt in https://github.com/lance-format/lance/pull/5355
fix: avx512 related symbol not found in mac x86 by @jackye1995 in https://github.com/lance-format/lance/pull/5379
fix: add graceful shutdown and start for rest namespace adapter by @jackye1995 in https://github.com/lance-format/lance/pull/5325
fix: respect index metric when user overrides by @BubbleCal in https://github.com/lance-format/lance/pull/5397
fix: critical fixes for 1.0.0-rc3 by @jackye1995 in https://github.com/lance-format/lance/pull/5421

Documentation 📚

docs: introduce lance as a lakehouse format by @jackye1995 in https://github.com/lance-format/lance/pull/5209
docs: update readme with latest lakehouse format info by @jackye1995 in https://github.com/lance-format/lance/pull/5216
docs: capitalization change by @timsaucer in https://github.com/lance-format/lance/pull/5269
docs: build project specific guidelines into web doc by @jackye1995 in https://github.com/lance-format/lance/pull/5324

Performance Improvements 🚀

perf: speed up filtered scan by up to 18.9× by moving the heavy CPU task out by @Xuanwo in https://github.com/lance-format/lance/pull/5165
perf: parallelize split job assigning by @BubbleCal in https://github.com/lance-format/lance/pull/5265
perf: use CPU pool to run WAND algo by @BubbleCal in https://github.com/lance-format/lance/pull/5363
perf: avoid allocating filtered nodes on HNSW search path by @BubbleCal in https://github.com/lance-format/lance/pull/5377

Other Changes

refactor: move blob version as a table level config by @Xuanwo in https://github.com/lance-format/lance/pull/5220
refactor: add helper functions to delta.rs tests by @yanghua in https://github.com/lance-format/lance/pull/5298
refactor: move LanceNamespace interface to pylance and java lance-core by @jackye1995 in https://github.com/lance-format/lance/pull/5345
refactor: allow datafiles to contain columns without field id by @Xuanwo in https://github.com/lance-format/lance/pull/5348
refactor: rename RowIdSelection to RowAddrSelection by @yanghua in https://github.com/lance-format/lance/pull/5263
refactor: separate out python and java LanceNamespace interface by @jackye1995 in https://github.com/lance-format/lance/pull/5364
refactor: align with blob v2 logical types change by @Xuanwo in https://github.com/lance-format/lance/pull/5375

New Contributors

@prrao87 made their first contribution in https://github.com/lance-format/lance/pull/5196
@fenfeng9 made their first contribution in https://github.com/lance-format/lance/pull/4994
@fangyinc made their first contribution in https://github.com/lance-format/lance/pull/5185
@shiyajuan123 made their first contribution in https://github.com/lance-format/lance/pull/5252
@Pmathsun made their first contribution in https://github.com/lance-format/lance/pull/5332
@ztorchan made their first contribution in https://github.com/lance-format/lance/pull/5301
@fMeow made their first contribution in https://github.com/lance-format/lance/pull/5371

Full Changelog: https://github.com/lance-format/lance/compare/release-root/1.0.0-beta.N...v1.0.0-rc.3

Open source →

1.0.0rc23 Dec 2025pre-releaseDeprecated

refactor!: deprecate TFRecord support by @jackye1995 in https://github.com/lance-format/lance/pull/4593

What's Changed

Breaking Changes 🛠

perf!: dynamic pruning for vector search by @BubbleCal in https://github.com/lance-format/lance/pull/4773
feat!: remove unnecessary mut of dataset::sql by @ddupg in https://github.com/lance-format/lance/pull/5207
refactor!: move all previous code into previous mod by @Xuanwo in https://github.com/lance-format/lance/pull/5217
refactor!: deprecate TFRecord support by @jackye1995 in https://github.com/lance-format/lance/pull/4593
refactor!: use org.lance namespace for java package by @jackye1995 in https://github.com/lance-format/lance/pull/5339
refactor!: deprecate mac x86 support by @jackye1995 in https://github.com/lance-format/lance/pull/5391

Critical Fixes ‼️

fix: convert some panics into errors by @wjones127 in https://github.com/lance-format/lance/pull/5258

New Features 🎉

feat: provide inline_transaction model for IO optimizing by @majin1102 in https://github.com/lance-format/lance/pull/4774
feat(python): add support for HuggingFace IterableDataset by @changhiskhan in https://github.com/lance-format/lance/pull/2599
feat: add public accessors for count plan construction by @wkalt in https://github.com/lance-format/lance/pull/5103
feat: add adapter for REST namespace with manifest namespace backend by @jackye1995 in https://github.com/lance-format/lance/pull/4984
feat: add blob compaction support by @Xuanwo in https://github.com/lance-format/lance/pull/5189
feat: add inline optimization for dir namespace by @jackye1995 in https://github.com/lance-format/lance/pull/5244
feat: conflict resolution for DataReplacement by @wjones127 in https://github.com/lance-format/lance/pull/3631
feat: support namespace vended credentials for write by @jackye1995 in https://github.com/lance-format/lance/pull/5161
docs: correct the comment in util.py by @shiyajuan123 in https://github.com/lance-format/lance/pull/5252
feat: introduce community governance by @jackye1995 in https://github.com/lance-format/lance/pull/5262
feat: add describe_indices function by @westonpace in https://github.com/lance-format/lance/pull/5221
feat: add target_bases extension to python write_fragments API by @jaystarshot in https://github.com/lance-format/lance/pull/5234
feat: support credentials vending for file reader and session by @jackye1995 in https://github.com/lance-format/lance/pull/5256
feat: introduce blob arrow extension type by @Xuanwo in https://github.com/lance-format/lance/pull/5239
feat(java): add binding for rest and dir namespaces by @jackye1995 in https://github.com/lance-format/lance/pull/5292
feat: expose file upload and download in Lance file session by @jackye1995 in https://github.com/lance-format/lance/pull/5336
feat(java): support credential vending at write time by @jackye1995 in https://github.com/lance-format/lance/pull/5309
feat(ds-sql-api): support JSON bulit in functions in ds.sql API by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5350
feat: support GEO types by @ddupg in https://github.com/lance-format/lance/pull/4678
feat(python): expose DatasetDeltaBuilder and relevant apis by @yanghua in https://github.com/lance-format/lance/pull/5091
feat: return Unprocessable error while expected error happened by @Xuanwo in https://github.com/lance-format/lance/pull/5347
feat: add huggingface native support by @Xuanwo in https://github.com/lance-format/lance/pull/5353
feat: dynamically choose distance type by @BubbleCal in https://github.com/lance-format/lance/pull/5370
feat(java): support writing schema metadata through java LanceFileWriter API by @steFaiz in https://github.com/lance-format/lance/pull/5310

Bug Fixes 🐛

fix: ensure recheck for IsNotNull in bloom filter by @Xuanwo in https://github.com/lance-format/lance/pull/5192
fix: contributing URL gives 404 by @prrao87 in https://github.com/lance-format/lance/pull/5196
fix: merge struct array use wrong child values by @wojiaodoubao in https://github.com/lance-format/lance/pull/5106
fix: avoid unnecessary get_fragments calling during plan compaction by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5179
fix: split partition may be assigned to itself by @BubbleCal in https://github.com/lance-format/lance/pull/5190
fix: improve schema validation for nullability and subschemas by @fenfeng9 in https://github.com/lance-format/lance/pull/4994
fix: compile error in test_inline_transaction by @wojiaodoubao in https://github.com/lance-format/lance/pull/5206
docs: fix batch udf with checkpoint document error by @fangyinc in https://github.com/lance-format/lance/pull/5185
fix: memory-limited string BTREE index building by @wjones127 in https://github.com/lance-format/lance/pull/5175
fix: home page code snippets cuasing various problems by @jackye1995 in https://github.com/lance-format/lance/pull/5245
fix: panic if only one partition and split is triggered by @BubbleCal in https://github.com/lance-format/lance/pull/5241
fix: clearer error in dataset take by @wkalt in https://github.com/lance-format/lance/pull/5243
fix: docs and comment have broken links by @prrao87 in https://github.com/lance-format/lance/pull/5261
fix: handle logical rows deletion properly for zonemap and bloomfilter by @HaochengLIU in https://github.com/lance-format/lance/pull/5140
fix: blob version should be passed in Projection by @Xuanwo in https://github.com/lance-format/lance/pull/5295
docs: fix broken links and 404s by @prrao87 in https://github.com/lance-format/lance/pull/5284
fix: index overestimates the posting list size by @BubbleCal in https://github.com/lance-format/lance/pull/5327
fix: update CachedFileMetadata version API to V2.0 by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5330
fix: correctly handle OSS commit protocol to prevent data loss by @Pmathsun in https://github.com/lance-format/lance/pull/5332
fix: update btree index with its own zone size instead of DEFAULT_BTR… by @ztorchan in https://github.com/lance-format/lance/pull/5301
fix: join job may cause inconsistent delta indices by @BubbleCal in https://github.com/lance-format/lance/pull/5328
fix: panicException when calling compaction by @yanghua in https://github.com/lance-format/lance/pull/5282
fix: parallelize bitmap partition loading in IsIn expressions by @wkalt in https://github.com/lance-format/lance/pull/5355
fix: avx512 related symbol not found in mac x86 by @jackye1995 in https://github.com/lance-format/lance/pull/5379
fix: add graceful shutdown and start for rest namespace adapter by @jackye1995 in https://github.com/lance-format/lance/pull/5325
fix: respect index metric when user overrides by @BubbleCal in https://github.com/lance-format/lance/pull/5397

Documentation 📚

docs: introduce lance as a lakehouse format by @jackye1995 in https://github.com/lance-format/lance/pull/5209
docs: update readme with latest lakehouse format info by @jackye1995 in https://github.com/lance-format/lance/pull/5216
docs: capitalization change by @timsaucer in https://github.com/lance-format/lance/pull/5269
docs: build project specific guidelines into web doc by @jackye1995 in https://github.com/lance-format/lance/pull/5324

Performance Improvements 🚀

perf: speed up filtered scan by up to 18.9× by moving the heavy CPU task out by @Xuanwo in https://github.com/lance-format/lance/pull/5165
perf: parallelize split job assigning by @BubbleCal in https://github.com/lance-format/lance/pull/5265
perf: use CPU pool to run WAND algo by @BubbleCal in https://github.com/lance-format/lance/pull/5363
perf: avoid allocating filtered nodes on HNSW search path by @BubbleCal in https://github.com/lance-format/lance/pull/5377

Other Changes

refactor: move blob version as a table level config by @Xuanwo in https://github.com/lance-format/lance/pull/5220
refactor: add helper functions to delta.rs tests by @yanghua in https://github.com/lance-format/lance/pull/5298
refactor: move LanceNamespace interface to pylance and java lance-core by @jackye1995 in https://github.com/lance-format/lance/pull/5345
refactor: allow datafiles to contain columns without field id by @Xuanwo in https://github.com/lance-format/lance/pull/5348
refactor: rename RowIdSelection to RowAddrSelection by @yanghua in https://github.com/lance-format/lance/pull/5263
refactor: separate out python and java LanceNamespace interface by @jackye1995 in https://github.com/lance-format/lance/pull/5364
refactor: align with blob v2 logical types change by @Xuanwo in https://github.com/lance-format/lance/pull/5375

New Contributors

@prrao87 made their first contribution in https://github.com/lance-format/lance/pull/5196
@fenfeng9 made their first contribution in https://github.com/lance-format/lance/pull/4994
@fangyinc made their first contribution in https://github.com/lance-format/lance/pull/5185
@shiyajuan123 made their first contribution in https://github.com/lance-format/lance/pull/5252
@Pmathsun made their first contribution in https://github.com/lance-format/lance/pull/5332
@ztorchan made their first contribution in https://github.com/lance-format/lance/pull/5301
@fMeow made their first contribution in https://github.com/lance-format/lance/pull/5371

Full Changelog: https://github.com/lance-format/lance/compare/release-root/1.0.0-beta.N...v1.0.0-rc.2

Open source →

1.0.0rc12 Dec 2025pre-releaseDeprecated

refactor!: deprecate TFRecord support by @jackye1995 in https://github.com/lance-format/lance/pull/4593

What's Changed

Breaking Changes 🛠

perf!: dynamic pruning for vector search by @BubbleCal in https://github.com/lance-format/lance/pull/4773
feat!: remove unnecessary mut of dataset::sql by @ddupg in https://github.com/lance-format/lance/pull/5207
refactor!: move all previous code into previous mod by @Xuanwo in https://github.com/lance-format/lance/pull/5217
refactor!: deprecate TFRecord support by @jackye1995 in https://github.com/lance-format/lance/pull/4593
refactor!: use org.lance namespace for java package by @jackye1995 in https://github.com/lance-format/lance/pull/5339
refactor!: deprecate mac x86 support by @jackye1995 in https://github.com/lance-format/lance/pull/5391

Critical Fixes ‼️

fix: convert some panics into errors by @wjones127 in https://github.com/lance-format/lance/pull/5258

New Features 🎉

feat: provide inline_transaction model for IO optimizing by @majin1102 in https://github.com/lance-format/lance/pull/4774
feat(python): add support for HuggingFace IterableDataset by @changhiskhan in https://github.com/lance-format/lance/pull/2599
feat: add public accessors for count plan construction by @wkalt in https://github.com/lance-format/lance/pull/5103
feat: add adapter for REST namespace with manifest namespace backend by @jackye1995 in https://github.com/lance-format/lance/pull/4984
feat: add blob compaction support by @Xuanwo in https://github.com/lance-format/lance/pull/5189
feat: add inline optimization for dir namespace by @jackye1995 in https://github.com/lance-format/lance/pull/5244
feat: conflict resolution for DataReplacement by @wjones127 in https://github.com/lance-format/lance/pull/3631
feat: support namespace vended credentials for write by @jackye1995 in https://github.com/lance-format/lance/pull/5161
docs: correct the comment in util.py by @shiyajuan123 in https://github.com/lance-format/lance/pull/5252
feat: introduce community governance by @jackye1995 in https://github.com/lance-format/lance/pull/5262
feat: add describe_indices function by @westonpace in https://github.com/lance-format/lance/pull/5221
feat: add target_bases extension to python write_fragments API by @jaystarshot in https://github.com/lance-format/lance/pull/5234
feat: support credentials vending for file reader and session by @jackye1995 in https://github.com/lance-format/lance/pull/5256
feat: introduce blob arrow extension type by @Xuanwo in https://github.com/lance-format/lance/pull/5239
feat(java): add binding for rest and dir namespaces by @jackye1995 in https://github.com/lance-format/lance/pull/5292
feat: expose file upload and download in Lance file session by @jackye1995 in https://github.com/lance-format/lance/pull/5336
feat(java): support credential vending at write time by @jackye1995 in https://github.com/lance-format/lance/pull/5309
feat(ds-sql-api): support JSON bulit in functions in ds.sql API by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5350
feat: support GEO types by @ddupg in https://github.com/lance-format/lance/pull/4678
feat(python): expose DatasetDeltaBuilder and relevant apis by @yanghua in https://github.com/lance-format/lance/pull/5091
feat: return Unprocessable error while expected error happened by @Xuanwo in https://github.com/lance-format/lance/pull/5347
feat: add huggingface native support by @Xuanwo in https://github.com/lance-format/lance/pull/5353
feat: dynamically choose distance type by @BubbleCal in https://github.com/lance-format/lance/pull/5370
feat(java): support writing schema metadata through java LanceFileWriter API by @steFaiz in https://github.com/lance-format/lance/pull/5310

Bug Fixes 🐛

fix: ensure recheck for IsNotNull in bloom filter by @Xuanwo in https://github.com/lance-format/lance/pull/5192
fix: contributing URL gives 404 by @prrao87 in https://github.com/lance-format/lance/pull/5196
fix: merge struct array use wrong child values by @wojiaodoubao in https://github.com/lance-format/lance/pull/5106
fix: avoid unnecessary get_fragments calling during plan compaction by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5179
fix: split partition may be assigned to itself by @BubbleCal in https://github.com/lance-format/lance/pull/5190
fix: improve schema validation for nullability and subschemas by @fenfeng9 in https://github.com/lance-format/lance/pull/4994
fix: compile error in test_inline_transaction by @wojiaodoubao in https://github.com/lance-format/lance/pull/5206
docs: fix batch udf with checkpoint document error by @fangyinc in https://github.com/lance-format/lance/pull/5185
fix: memory-limited string BTREE index building by @wjones127 in https://github.com/lance-format/lance/pull/5175
fix: home page code snippets cuasing various problems by @jackye1995 in https://github.com/lance-format/lance/pull/5245
fix: panic if only one partition and split is triggered by @BubbleCal in https://github.com/lance-format/lance/pull/5241
fix: clearer error in dataset take by @wkalt in https://github.com/lance-format/lance/pull/5243
fix: docs and comment have broken links by @prrao87 in https://github.com/lance-format/lance/pull/5261
fix: handle logical rows deletion properly for zonemap and bloomfilter by @HaochengLIU in https://github.com/lance-format/lance/pull/5140
fix: blob version should be passed in Projection by @Xuanwo in https://github.com/lance-format/lance/pull/5295
docs: fix broken links and 404s by @prrao87 in https://github.com/lance-format/lance/pull/5284
fix: index overestimates the posting list size by @BubbleCal in https://github.com/lance-format/lance/pull/5327
fix: update CachedFileMetadata version API to V2.0 by @zhangyue19921010 in https://github.com/lance-format/lance/pull/5330
fix: correctly handle OSS commit protocol to prevent data loss by @Pmathsun in https://github.com/lance-format/lance/pull/5332
fix: update btree index with its own zone size instead of DEFAULT_BTR… by @ztorchan in https://github.com/lance-format/lance/pull/5301
fix: join job may cause inconsistent delta indices by @BubbleCal in https://github.com/lance-format/lance/pull/5328
fix: panicException when calling compaction by @yanghua in https://github.com/lance-format/lance/pull/5282
fix: parallelize bitmap partition loading in IsIn expressions by @wkalt in https://github.com/lance-format/lance/pull/5355
fix: avx512 related symbol not found in mac x86 by @jackye1995 in https://github.com/lance-format/lance/pull/5379
fix: add graceful shutdown and start for rest namespace adapter by @jackye1995 in https://github.com/lance-format/lance/pull/5325

Documentation 📚

docs: introduce lance as a lakehouse format by @jackye1995 in https://github.com/lance-format/lance/pull/5209
docs: update readme with latest lakehouse format info by @jackye1995 in https://github.com/lance-format/lance/pull/5216
docs: capitalization change by @timsaucer in https://github.com/lance-format/lance/pull/5269
docs: build project specific guidelines into web doc by @jackye1995 in https://github.com/lance-format/lance/pull/5324

Performance Improvements 🚀

perf: speed up filtered scan by up to 18.9× by moving the heavy CPU task out by @Xuanwo in https://github.com/lance-format/lance/pull/5165
perf: parallelize split job assigning by @BubbleCal in https://github.com/lance-format/lance/pull/5265
perf: use CPU pool to run WAND algo by @BubbleCal in https://github.com/lance-format/lance/pull/5363
perf: avoid allocating filtered nodes on HNSW search path by @BubbleCal in https://github.com/lance-format/lance/pull/5377

Other Changes

refactor: move blob version as a table level config by @Xuanwo in https://github.com/lance-format/lance/pull/5220
refactor: add helper functions to delta.rs tests by @yanghua in https://github.com/lance-format/lance/pull/5298
refactor: move LanceNamespace interface to pylance and java lance-core by @jackye1995 in https://github.com/lance-format/lance/pull/5345
refactor: allow datafiles to contain columns without field id by @Xuanwo in https://github.com/lance-format/lance/pull/5348
refactor: rename RowIdSelection to RowAddrSelection by @yanghua in https://github.com/lance-format/lance/pull/5263
refactor: separate out python and java LanceNamespace interface by @jackye1995 in https://github.com/lance-format/lance/pull/5364
refactor: align with blob v2 logical types change by @Xuanwo in https://github.com/lance-format/lance/pull/5375

New Contributors

@prrao87 made their first contribution in https://github.com/lance-format/lance/pull/5196
@fenfeng9 made their first contribution in https://github.com/lance-format/lance/pull/4994
@fangyinc made their first contribution in https://github.com/lance-format/lance/pull/5185
@shiyajuan123 made their first contribution in https://github.com/lance-format/lance/pull/5252
@Pmathsun made their first contribution in https://github.com/lance-format/lance/pull/5332
@ztorchan made their first contribution in https://github.com/lance-format/lance/pull/5301
@fMeow made their first contribution in https://github.com/lance-format/lance/pull/5371

Full Changelog: https://github.com/lance-format/lance/compare/release-root/1.0.0-beta.N...v1.0.0-rc.1

Open source →

0.38.328 Oct 2025

What's Changed

Breaking Changes 🛠

refactor!: cleanup public API, remove lance_arrow re-exports by @westonpace in https://github.com/lancedb/lance/pull/4991

New Features 🎉

feat: handle forking better by @cmccabe in https://github.com/lancedb/lance/pull/4903
feat: add multi-path support for lance data paths by @jaystarshot in https://github.com/lancedb/lance/pull/4765
feat: support general compression zstd/lz4 in blocks by @lyang24 in https://github.com/lancedb/lance/pull/4900
feat: fts supports custom stop words by @wojiaodoubao in https://github.com/lancedb/lance/pull/4866
feat: implement add_bases api to add bases to lance dataset by @jaystarshot in https://github.com/lancedb/lance/pull/4945
feat: allow a commit message to be specified in the python dataset commit method by @westonpace in https://github.com/lancedb/lance/pull/4952
feat: support tracking newly inserted and updated rows between versions by @yanghua in https://github.com/lancedb/lance/pull/4741
feat: support fetching _rowid and _rowaddr in take operation by @chunshao90 in https://github.com/lancedb/lance/pull/4794
feat: support mTLS in REST namespace by @jackye1995 in https://github.com/lancedb/lance/pull/4981
fix: populate all fields from ExecutionSummaryCounts to ScannerStats by @jaystarshot in https://github.com/lancedb/lance/pull/4980
feat: support WhenMatched::Fail for MergeInsert by @ddupg in https://github.com/lancedb/lance/pull/4956
feat(java): support shallow_clone by @majin1102 in https://github.com/lancedb/lance/pull/4962
feat: add VariablePackedStruct defination by @Xuanwo in https://github.com/lancedb/lance/pull/4950
feat: log version on dataset load event by @wjones127 in https://github.com/lancedb/lance/pull/4988
feat(java): support blob api by @majin1102 in https://github.com/lancedb/lance/pull/4769
feat(python): support shallow_clone by @majin1102 in https://github.com/lancedb/lance/pull/4949
feat: expose storage options in dataset by @jackye1995 in https://github.com/lancedb/lance/pull/5016
feat: should dictionary encode based on size by @yingjianwu98 in https://github.com/lancedb/lance/pull/4972
feat: add variable packed struct support by @Xuanwo in https://github.com/lancedb/lance/pull/5003
feat(python): support index for nested field by @jackye1995 in https://github.com/lancedb/lance/pull/5027
feat(java): support branch operations by @majin1102 in https://github.com/lancedb/lance/pull/5012
feat(python): support branch operations by @majin1102 in https://github.com/lancedb/lance/pull/5010
feat: add json fts in python by @wojiaodoubao in https://github.com/lancedb/lance/pull/5020
feat: add file audit events for deleting staging manifest paths in external commit handler by @wjones127 in https://github.com/lancedb/lance/pull/4989
feat: implement create index and data replacement conflict rules by @westonpace in https://github.com/lancedb/lance/pull/5077
feat: support fragment level update columns operation in Python by @xloya in https://github.com/lancedb/lance/pull/5001

Bug Fixes 🐛

fix: let Java module use LanceFileVersion::Stable (#4558) by @ColdL in https://github.com/lancedb/lance/pull/4559
fix: fts match query on column without inverted index by @wojiaodoubao in https://github.com/lancedb/lance/pull/4859
fix: fix broken FTS example by replacing ROW_ID with DOC_ID by @niebayes in https://github.com/lancedb/lance/pull/4917
fix: correctly record output_rows in filtered read with hard range_after by @westonpace in https://github.com/lancedb/lance/pull/4919
fix: rewrap LanceFilterExec with_new_children by @wkalt in https://github.com/lancedb/lance/pull/4920
fix: optimize_indices may unexpectly delete delta indices by @BubbleCal in https://github.com/lancedb/lance/pull/4931
fix: rebuild HNSW graph while remapping it by @BubbleCal in https://github.com/lancedb/lance/pull/4941
fix: be compatible to old pack metadata in 2.0 by @Xuanwo in https://github.com/lancedb/lance/pull/4964
fix: filter with < current_date() should expand with correct time by @Xuanwo in https://github.com/lancedb/lance/pull/4963
fix: use correct logic to detect old/new scheme in binary block decoder by @westonpace in https://github.com/lancedb/lance/pull/4966
fix: support preview relase in writer version by @jackye1995 in https://github.com/lancedb/lance/pull/4974
fix: index cache assumed_entry_size is inconsistent by @ddupg in https://github.com/lancedb/lance/pull/4975
fix: clarify column names in missing column errors by @wkalt in https://github.com/lancedb/lance/pull/4942
fix: the KMeans may result in all zeros centroids by @BubbleCal in https://github.com/lancedb/lance/pull/4977
fix: full text search may miss some results by @BubbleCal in https://github.com/lancedb/lance/pull/4986
fix: do not modify Lance schema when projecting system columns by @jackye1995 in https://github.com/lancedb/lance/pull/4997
fix: handle empty batches in dictionary decode helper by @yingjianwu98 in https://github.com/lancedb/lance/pull/4995
fix: coerce nested regexp_match to boolean in filters by @BubbleCal in https://github.com/lancedb/lance/pull/5019
fix: update to respect file version from write params when writing fragments in java by @morales-t-netflix in https://github.com/lancedb/lance/pull/5014
fix: general block decompression mismatch for Lance 2.2 dictionaries by @Xuanwo in https://github.com/lancedb/lance/pull/5025
fix: handle List types in Substrait field counting by @LuQQiu in https://github.com/lancedb/lance/pull/5015
fix: ensure limit cancels scan by @westonpace in https://github.com/lancedb/lance/pull/5032
fix: don't panic in 2.1 if one page has nulls and the other doesn't by @westonpace in https://github.com/lancedb/lance/pull/5074
fix: correct read column ordering in Fragment::update_columns by @wayneli-vt in https://github.com/lancedb/lance/pull/4983
fix(java): the CompactionOptions is not serializable by @steFaiz in https://github.com/lancedb/lance/pull/4819
fix: fix the deduplicated prefix in the opendal s3 store root setting by @xloya in https://github.com/lancedb/lance/pull/5082

Documentation 📚

docs: add RabitQ in vector index spec by @BubbleCal in https://github.com/lancedb/lance/pull/4913
docs: document the metadata cache by @westonpace in https://github.com/lancedb/lance/pull/4953
docs: add docs for newly added options by @Xuanwo in https://github.com/lancedb/lance/pull/4976
docs: minor fix for docs/src/format/table/index.md#Deletion by @zhangyue19921010 in https://github.com/lancedb/lance/pull/5033
docs: minor doc fix for stable row id description by @zhangyue19921010 in https://github.com/lancedb/lance/pull/5047
docs: add initial migration guide and document some steps for 0.39 by @westonpace in https://github.com/lancedb/lance/pull/5052
docs: add migration guide for diff_meta and namespace by @jackye1995 in https://github.com/lancedb/lance/pull/5054
docs: add json to full text search document by @wojiaodoubao in https://github.com/lancedb/lance/pull/4865

Other Changes

refactor: move lance-namespace into lance repo by @jackye1995 in https://github.com/lancedb/lance/pull/4978
refactor: use lance-io object store for dir namespace and improve builder by @jackye1995 in https://github.com/lancedb/lance/pull/5045
refactor: don't add new with_... columns to scanner by @westonpace in https://github.com/lancedb/lance/pull/5007

New Contributors

@yingjianwu98 made their first contribution in https://github.com/lancedb/lance/pull/4901
@niebayes made their first contribution in https://github.com/lancedb/lance/pull/4917
@zhangyue19921010 made their first contribution in https://github.com/lancedb/lance/pull/4915
@dependabot[bot] made their first contribution in https://github.com/lancedb/lance/pull/4954
@chunshao90 made their first contribution in https://github.com/lancedb/lance/pull/4794

Full Changelog: https://github.com/lancedb/lance/compare/v0.38.2...v0.38.3

Open source →

0.32.023 Jul 2025

What's Changed

Breaking Changes 🛠

feat!: use FilteredReadExec in the planner by @westonpace in https://github.com/lancedb/lance/pull/3813
feat!: consolidated index cache by @wjones127 in https://github.com/lancedb/lance/pull/4047

New Features 🎉

feat: allow users to control the full zip rep index cache by @Xuanwo in https://github.com/lancedb/lance/pull/4157
feat: add rle encoding support by @Xuanwo in https://github.com/lancedb/lance/pull/4177
feat: support inferring system index type by @jackye1995 in https://github.com/lancedb/lance/pull/4210
feat(rust): support trace dataset events by @yanghua in https://github.com/lancedb/lance/pull/4145
feat: support prewarm for btree by @BubbleCal in https://github.com/lancedb/lance/pull/4235
feat(java): support get lance schema with field ids by @majin1102 in https://github.com/lancedb/lance/pull/4207
feat: support sql api for dataset by @yanghua in https://github.com/lancedb/lance/pull/4086
feat: add native oss support for lance by @Xuanwo in https://github.com/lancedb/lance/pull/4247
feat: enable oss support in python and java by @Xuanwo in https://github.com/lancedb/lance/pull/4254
refactor: rewrite delete implementation to use dataset-level scanner (#4158) by @wjones127 in https://github.com/lancedb/lance/pull/4165
feat: add retry-based conflict resolution for update operations by @wjones127 in https://github.com/lancedb/lance/pull/4167
feat: add 64-bit offset support to FSST compression for large binary/string data in V2.1 by @HaochengLIU in https://github.com/lancedb/lance/pull/4092
feat: extend merge_insert fast path for UpdateIf and update-only operations by @wjones127 in https://github.com/lancedb/lance/pull/4253
feat: expose sql api for python dataset by @Jay-ju in https://github.com/lancedb/lance/pull/4267
feat(java): support replace schema and field metadata by @majin1102 in https://github.com/lancedb/lance/pull/4119
feat: add claude md for future agentic work by @Xuanwo in https://github.com/lancedb/lance/pull/4245
feat: implement byte stream split for doubles by @Xuanwo in https://github.com/lancedb/lance/pull/4236
feat: support ordering for pylance sdk by @SaintBacchus in https://github.com/lancedb/lance/pull/3556
feat: support skipping auto cleanup during write by @jackye1995 in https://github.com/lancedb/lance/pull/4293
feat: expose IVF centroids to the python API by @Sbargaoui in https://github.com/lancedb/lance/pull/4288

Bug Fixes 🐛

fix: forward compatibility of pylance load_indices by @jackye1995 in https://github.com/lancedb/lance/pull/4208
fix: contributing link fixed on README.md by @lalitx17 in https://github.com/lancedb/lance/pull/4211
fix: move BYTES_READ_COUNTER update after await to ensure accuracy by @allenanswerzq in https://github.com/lancedb/lance/pull/4203
fix: broken image link in README by @LeoReeYang in https://github.com/lancedb/lance/pull/4228
fix: normalize_fsl should handle nulls by @Xuanwo in https://github.com/lancedb/lance/pull/4226
fix: phrase query may miss some results by @BubbleCal in https://github.com/lancedb/lance/pull/4223
fix: handle index statistics for system indexes by @jackye1995 in https://github.com/lancedb/lance/pull/4229
fix: btree may load the same page multiple times by @BubbleCal in https://github.com/lancedb/lance/pull/4246
fix: allow rowid in columns for scanner by @Jay-ju in https://github.com/lancedb/lance/pull/4116
fix: fix errors caused by simultaneous merge by @westonpace in https://github.com/lancedb/lance/pull/4279
fix: escape type keyword in tracing::info! macros by @bjurkovski in https://github.com/lancedb/lance/pull/4284
fix: proper case-insensitive comparison for filter by @wjones127 in https://github.com/lancedb/lance/pull/4278
fix: ensure _rowid comes after _distance and _score by @westonpace in https://github.com/lancedb/lance/pull/4294

Documentation 📚

docs: optional values are "present" or "absent" by @jbapple in https://github.com/lancedb/lance/pull/4184
docs: separate Quickstart into three tutorials by @davidmyriel in https://github.com/lancedb/lance/pull/4200
docs: add complete blob writing example and fix take_blobs usage by @TaoKevinKK in https://github.com/lancedb/lance/pull/4214
docs: add new homepage and sidebar by @davidmyriel in https://github.com/lancedb/lance/pull/4231
docs: separate table and file format spec by @jackye1995 in https://github.com/lancedb/lance/pull/4264

Performance Improvements 🚀

perf: improve btree performance for range query by @BubbleCal in https://github.com/lancedb/lance/pull/4248
perf: wrap Dataset with Arc in SqlQueryBuilder by @wojiaodoubao in https://github.com/lancedb/lance/pull/4275

Other Changes

refactor: unify two branches of add_offsets to make it more clear by @allenanswerzq in https://github.com/lancedb/lance/pull/4225
refactor: avoid fri abbrevation by @jackye1995 in https://github.com/lancedb/lance/pull/4285

New Contributors

@lalitx17 made their first contribution in https://github.com/lancedb/lance/pull/4211
@davidmyriel made their first contribution in https://github.com/lancedb/lance/pull/4200
@TaoKevinKK made their first contribution in https://github.com/lancedb/lance/pull/4214
@Sbargaoui made their first contribution in https://github.com/lancedb/lance/pull/4288

Full Changelog: https://github.com/lancedb/lance/compare/v0.31.1...v0.32.0

Open source →

0.31.03 Jul 2025

What's Changed

Breaking Changes 🛠

feat!: refactor Dataset#config api and expose it via pylance by @yanghua in https://github.com/lancedb/lance/pull/4041

New Features 🎉

feat: add column projection to LanceFileReader by @wojiaodoubao in https://github.com/lancedb/lance/pull/4031
feat: add consecutive batch shard sampler for pytorch by @Jay-ju in https://github.com/lancedb/lance/pull/3886
feat: add Trino and PostgreSQL integrations by @jackye1995 in https://github.com/lancedb/lance/pull/4054
feat: expose num_rows api for python LanceFileReader by @yanghua in https://github.com/lancedb/lance/pull/4053
feat: add IVF_HNSW_FLAT index by @BubbleCal in https://github.com/lancedb/lance/pull/3671
feat: introduce IndexFileVersion for Python API by @BubbleCal in https://github.com/lancedb/lance/pull/4040
feat: add created_at for indices by @wjones127 in https://github.com/lancedb/lance/pull/4051
feat: support large_string/large_binary in lance format v2.1 by @HaochengLIU in https://github.com/lancedb/lance/pull/3967
feat: upgrade arrow object_store to 0.12.2 by @jackye1995 in https://github.com/lancedb/lance/pull/4080
feat: add random take bench based on file reader api and fragment api by @wojiaodoubao in https://github.com/lancedb/lance/pull/4076
feat: support IVF_SQ index by @BubbleCal in https://github.com/lancedb/lance/pull/4096
feat: add ignore_missing flag for safe dataset deletion by @Jay-ju in https://github.com/lancedb/lance/pull/3667
feat: add a toggle for automatic rank/world_size discovery by @Jay-ju in https://github.com/lancedb/lance/pull/3633
feat: add N-Gram tokenizer for FTS index by @BubbleCal in https://github.com/lancedb/lance/pull/4102
feat(java): support config operation for Dataset by @majin1102 in https://github.com/lancedb/lance/pull/4039
feat(datagen): add random words generator by @Xuanwo in https://github.com/lancedb/lance/pull/4099
feat: allow _rowid and _rowaddr in read_columns for dataset.add_columns / fragment.merge_columns by @westonpace in https://github.com/lancedb/lance/pull/4120
feat(encoding): cache repetition index for FullZip encoding by @Xuanwo in https://github.com/lancedb/lance/pull/4104

Bug Fixes 🐛

fix: can't search on a *_SQ index with f64 vectors by @BubbleCal in https://github.com/lancedb/lance/pull/3672
fix: torchdataset for lance does not support S3 by @Jay-ju in https://github.com/lancedb/lance/pull/4045
fix: incorrect parameter order when initializing ShardedFixedBatchSam… by @ddupg in https://github.com/lancedb/lance/pull/4064
fix: take benchmark can not read dataset by @yanghua in https://github.com/lancedb/lance/pull/4065
fix: escape type keyword in tracing::info! macros by @bjurkovski in https://github.com/lancedb/lance/pull/4068
fix: better errors on bad filters by @westonpace in https://github.com/lancedb/lance/pull/4073
fix(doc): fix doc typo in Ray integration documentation for Lance Data Sink by @kilavvy in https://github.com/lancedb/lance/pull/3984
fix: don't check vector indices when loading scalar index by @BubbleCal in https://github.com/lancedb/lance/pull/4077
fix: handle the case of all partitions are empty by @BubbleCal in https://github.com/lancedb/lance/pull/4085
fix: use std::fs::copy for local filesystem cross-filesystem copies by @wjones127 in https://github.com/lancedb/lance/pull/4088
fix: typo in feature_flags.rs by @b4l in https://github.com/lancedb/lance/pull/4093
fix: wrong cp command for java native lib name by @yanghua in https://github.com/lancedb/lance/pull/4105
fix: make max_fragment_id optional to prevent fragment ID reuse by @wjones127 in https://github.com/lancedb/lance/pull/4083
fix: panic when optimize empty PQ partition by @BubbleCal in https://github.com/lancedb/lance/pull/4108
fix: panic when commit fragment with format v2.1 by @ddupg in https://github.com/lancedb/lance/pull/4122

Documentation 📚

docs: fix write_dataset append mode obsolete description by @wojiaodoubao in https://github.com/lancedb/lance/pull/3866

Performance Improvements 🚀

perf: fast upsert with no indices by @wjones127 in https://github.com/lancedb/lance/pull/3858
perf: faster kmeans by @BubbleCal in https://github.com/lancedb/lance/pull/4070

Other Changes

refactor: don't use schema dictionary unless using legacy storage by @westonpace in https://github.com/lancedb/lance/pull/4023
refactor: move 2.0 code into its own module by @westonpace in https://github.com/lancedb/lance/pull/4046
refactor: move delete implementation into it's own file by @wjones127 in https://github.com/lancedb/lance/pull/4074
Revert "chore: update tantivy version to 0.24.1 (#4098)" by @Xuanwo in https://github.com/lancedb/lance/pull/4101
refactor(datagen): add RandomWord and RandomSentence to replace RandomWords by @Xuanwo in https://github.com/lancedb/lance/pull/4130

New Contributors

@wojiaodoubao made their first contribution in https://github.com/lancedb/lance/pull/4031
@ddupg made their first contribution in https://github.com/lancedb/lance/pull/4064
@bjurkovski made their first contribution in https://github.com/lancedb/lance/pull/4068
@kilavvy made their first contribution in https://github.com/lancedb/lance/pull/3984
@b4l made their first contribution in https://github.com/lancedb/lance/pull/4093
@Dig-Doug made their first contribution in https://github.com/lancedb/lance/pull/4098
@xhwhis made their first contribution in https://github.com/lancedb/lance/pull/4106

Full Changelog: https://github.com/lancedb/lance/compare/v0.30.0...v0.31.0

Open source →