arrow-pyarrow
Pyarrow bindings
59.2.0
6.2M downloads/mo
#4044 most downloaded on crates.io
apache/arrow-rs
What this package is like to depend on
Last release 17 days ago
06 Aug 2026
Ships fairly regularly
a new release about every 4 weeks
Most releases are documented
notes for 16 of 18 stable releases
Nothing withdrawn
no release was ever pulled
1 years old
18 releases · first in 2025
15 releases in the last 12 months
see the full history below
Release timeline
18 releases · Jun 2025 to Aug 2026Releases
latest 18-
59.2.006 Aug 2026Release notes
Open source →Changelog
59.2.0 - (2026-08-02)
Enhancements
- arrow-avro: Deprecate object_store integration by @brancz in #10484
- chore(arrow): add unchecked array builder methods by @Rich-T-kid in #10440
- feat(arrow-ipc): add sans-IO stream encoder by @Phoenix500526 in #10277
- chore(arrow-cast): support suffix white space in arrow-cast parse by @Rich-T-kid in #10396
- feat(coalesce): add size function by @rluvaton in #10331
- fix(parquet): support mask filtering across skipped pages by @hhhizzz in #10288
- chore(arrow-cast): support white space prefixed parse for ints and floats by @Rich-T-kid in #10374
- chore(arrow-buffer): Make more of i256 available in const code by @AdamGS in #10363
- feat(arrow-array): Expose builder buffer capacity accessors by @Weijun-H in #10342
- feat(arrow-arith): Add interval multiplication by i64 by @peterxcli in #10336
- fix: Make
parquet-indexwork with column paths by @korowa in #10330 - feat(arrow-csv): add support for parsing
Float16by @Glatzel in #10343 - Remove the unmaintained paste dependency from arrow by @Phoenix500526 in #10303
- chore: formalize the default map field names to match default arrow spec by @rluvaton in #10297
- feat(variant): Add
variant_to_arrowMaptype support by @sdf-jkl in #10307
Bug fixes
- fix(arrow-schema): Persist dictionary ordered flag on FFI schema import by @borchero in #10514
- fix(arrow-json): validate ListView child nullability by @dk3yyyy in #10486
- fix(arrow-json): validate map value nullability by @subotac in #10475
- avro: bound VLQDecoder::long against overlong varints by @STiFLeR7 in #10407
- arrow-row: Fix decode_fixed_size_list to apply the corrected_type step for dictionary children by @zhuqi-lucas in #10414
- [Variant] make
valuemandatory field forVariantArray/ShreddingStateby @sdf-jkl in #10318 - fix off by one error for slice accounting by @Rich-T-kid in #10406
- fix:
GenericByteViewArray::gc()drops inline views on the multi-buffer slow path by @adriangb in #10287 - fix(arrow-json): render coerced f32 as its value in the string decoder by @hareshkh in #10386
- fix(arrow-cast): make
b64_encodereject invalid UTF-8 from misbehavingEngineimpls by @bit2swaz in #10324 - fix: take FFI_ArrowArrayStream errno values from libc by @fornwall in #10299
- fix(arrow-data): allow full dictionary key range when concatenating by @raphaelroshan in #10323
- Don't panic on invalid c ffi schema name by @robert3005 in #10328
- fix(REE): check upfront if sorting empty array or 0 limit by @Jefffrey in #10293
- fix(arrow-avro): bound untrusted OCF block size and item counts by @miniex in #10237
- fix(arrow-array): disallow creating
MapArraywith nullable key field by @rluvaton in #10272 - fix: don't panic on
ArrayData::try_newon bad input even whenforce_validatefeature is on by @rluvaton in #10282
Performance improvements
- perf(parquet): slice up contiguous buffer for decimals and fsb by @MassivePizza in #10364
- feat(parquet):
RowSelectioncan be backed by aBooleanBufferby @haohuaijin in #10141 - perf(parquet): use Cursor in ZSTDCodec to avoid Vec alloc and copy by @MassivePizza in #10345
- optimize(parquet): Nested list batching child.write calls by @mapleFU in #10085
- perf(parquet): splice buffered pages with
write_allinstead ofio::copy(adapts #10052) by @adriangb in #10353 - hoist calls for null_sentinel by @Rich-T-kid in #10356
- perf: speed up substring_by_char with an ASCII fast path and single-pass bounds by @andygrove in #10334
- Cache encoded field name in FieldEncoder by @MassivePizza in #10296
- perf: allow users to skip utf8 validation in arrow-row by @Rich-T-kid in #10319
- perf: Improve decimal addition and subtraction when scale is equal by @AdamGS in #10333
- optimize(interleave): implement interleave for FixedSizeList/Map type by @mapleFU in #10046
- Perf: Pre-size buffer allocations to avoid intermediate allocations by @Rich-T-kid in #10262
- Perf: create dictionary reader config and default unsafeflag to false by @Rich-T-kid in #10260
- Perf: Introduce zero copy path when tonic returns an aligned buffer by @Rich-T-kid in #10273
- Validate short view strings in separate buffer in arrow-row by @Jefffrey in #10250
Documentation updates
- chore(parquet): add link to ticket in object_store deprecation message by @alamb in #10502
- chore(avro): add link to ticket in object_store deprecation message by @alamb in #10503
- docs: clarify decimal negative scale behavior by @ByteBaker in #10304
- Docs: fix mutableArrayData comments by @Rich-T-kid in #10326
- docs: trim release schedule for released versions by @alamb in #10280
- Align parquet-geospatial crate docs with README by @paleolimbot in #10302
Miscellaneous
- Revert "chore: formalize the default map field names to match default arrow spec (#10297)" by @alamb in #10506
- parquet: deprecate explicit
object_storeintegration by @brancz in #10354 - fix(parquet): restore opaque return type for
RowSelection::iterby @haohuaijin in #10450 - refactor(parquet): split
arrow_reader/selectioninto smaller modules by @haohuaijin in #10434 - chore: deduplicate filter nulls code in coalesce/filter kernel by @Jefffrey in #10348
- chore: remove parquet dependency from parquet_derive by @ByteBaker in #10327
-
59.1.007 Jul 2026Release notes
Open source →Changelog
59.1.0 (2026-07-03)
Implemented enhancements:
- Fast path for nested
DictionaryArraycasting #10247 [arrow] - parquet/arrow: reading multiple nested columns fails with "Not all children array length are the same!" when a list continues across DataPageV2 page boundary #10243 [parquet]
- Add product aggregate kernel to arrow-rs #10150 [arrow]
- Stricter
DataTypeparsing #10146 [arrow] - Support validating CSV headers against Schema #10143 [arrow]
- arrow-ipc: Supports compression level configuration for arrow-ipc writer #10132 [arrow]
- [Variant]
VariantArrayfield API naming #10093 - Add
StructArray::field_APIs symmetric toStructArray::column_ones #10092 [arrow] - arrow-buffer: implement Saturating, CheckedShl, Not num-traits for i256 #10087 [arrow]
- feat: native concat for
MapArray#10047 [arrow] - [Variant] Add
variant_to_arrowDictionary/REEtype support #10013
Fixed bugs:
- arrow-row on fixed size binary/list with size 0 and no nulls return wrong length #10270
- casting list to 0-size fixedsizelist can cause incorrect output length #10227 [arrow]
- Buffer count mismatched with metadata when encoding records with dictionary of dictionaries #10213 [arrow]
Buffer::into_mutableis not consistent regarding sliced data and can lead to panics #10117 [arrow]- parquet_derive: cannot read or write columns whose name is a Rust keyword (raw identifiers like r#type become column "r#type") #10112
- parquet: fix OffsetBuffer panic on corrupt input #10107 [parquet]
- Parquet geospatial conversion uses metadata key "algorithm" instead of "edges" in geoarrow metadata #9929 [parquet]
Documentation updates:
- doc: More comments to
concat_batches#10178 (2010YOUY01) - Minor: improve PageStore docs with a temp-file spilling example #10074 [parquet] (alamb)
Performance improvements:
- perf: interleave_list for List<Primitive> could be optimized? #10022 [arrow]
- perf(arrow-ord): Avoid full index materialization for small-limit lexsorts #9990 [arrow]
- Replace conversion of binary->string in arrow-row from arraydata to direct construction #10261 [arrow] (Jefffrey)
- [arrow-flight encode path]re-use flatbufferbuilder #10220 [arrow] (Rich-T-kid)
- [10125] arrow-flight decode path optimizations (add
skip_validationto arrow-flight) #10206 [arrow] [arrow-flight] (Rich-T-kid) - Improve performance of
concat_elementsByteViewArray concatenation #10161 [arrow] (pepijnve) - [arrow-flight] Optimize flight, remove some allocations, add dictionary focused benchmarks #10126 [arrow] [arrow-flight] (Rich-T-kid)
- optimize(concat): concat map implementation #10048 [arrow] (mapleFU)
- Reduce copies in Arrow IPC writer #10044 [arrow] (Rich-T-kid)
- perf(interleave): Optimize list interleave_list when child is primitive #10025 [arrow] (mapleFU)
Closed issues:
- Soundness: Unsound alignment contract in public
FromBytestrait andBitReader::get_batch#10164 [parquet] - ParquetPushDecoder: expose the next row-group index that try_next_reader will yield #10148 [parquet]
- arrow-ipc: Extend writer benchmarks to include dictionaries #10119 [arrow]
- bench(parquet): benchmark for nested list write #10083 [parquet]
- Support i256 implement From<i128> #10080 [arrow]
Merged pull requests:
- fix(arrow-row): allow to convert non empty fixed size binary/list array with size length 0 and no nulls #10271 [arrow] (rluvaton)
- chore: Fix audit CI run by ignore quick-xml audit advisories #10267 (alamb)
- fix main: parquet test compilation failure #10266 [parquet] (Jefffrey)
- minor: drive-by refactors for dicts in substring & filter #10264 [arrow] (Jefffrey)
- Add validated row decode benchmark #10259 [arrow] (alamb)
- arrow-cast: Add optimized path for unnesting a dict #10248 [arrow] (brancz)
- feat: support uuid from fixed type of length 16 #10241 [arrow] (ariel-miculas)
- chore(deps): bump actions/cache from 6.0.0 to 6.1.0 #10240 (dependabot[bot])
- fix: Rename parquet feature flag 'flate2-rust_backened' to 'flate2-rust_backend' #10239 [parquet] (dannycjones)
- chore: Make clippy::question_mark happy #10231 (Tpt)
- fix(ipc): reject dictionary-encoded dictionary values #10230 [arrow] (goutamadwant)
- Replace
ArrayDatawith directArrayconstruction inarrow-row#10229 [arrow] (Jefffrey) - fix: casting list to fixedsizelist didn't respect input length #10228 [arrow] (Jefffrey)
- chore: Fix clippy::byte_char_slices (use byte strings instead of explicit arrays) #10225 [parquet] [arrow] (Tpt)
- nit: arrow-pyarrow: Use string interning #10224 (Tpt)
- Support concatenation of mixed FixedSizeBinary via
concat_elements_dyn#10222 [arrow] (pepijnve) - rename Compression struct #10221 [arrow] [arrow-flight] (Rich-T-kid)
- chore(deps): bump the all-other-cargo-deps group across 1 directory with 16 updates #10218 (dependabot[bot])
- chore(deps): bump actions/setup-python from 6.2.0 to 6.3.0 #10210 (dependabot[bot])
- [10125] Introduce mult-batch decode benchmarks #10207 [arrow] [arrow-flight] (Rich-T-kid)
- chore(deps): bump actions/cache from 5.0.5 to 6.0.0 #10203 (dependabot[bot])
- introduce decode benchmarks #10202 [arrow] [arrow-flight] (Rich-T-kid)
- Fix
merge_kernelsbenchmark panic due to not wrapping withScalar#10199 [arrow] (Jefffrey) - Benchmarks and performance improvement for parquet boolean reader #10196 [parquet] (jhorstmann)
- add stale PR workflow #10194 (Jefffrey)
- chore: group minor/patch dependabot updates #10193 (Jefffrey)
- chore(deps): bump http from 1.4.0 to 1.4.2 #10191 (dependabot[bot])
- chore(deps): bump syn from 2.0.117 to 2.0.118 #10190 (dependabot[bot])
- chore(deps): bump chrono from 0.4.44 to 0.4.45 #10188 (dependabot[bot])
- chore(deps): bump uuid from 1.23.1 to 1.23.3 #10186 (dependabot[bot])
- chore: run
cargo updateto bump quinn #10181 (Jefffrey) - test: cover signed integers and bool in BitReader::get_batch test #10180 [parquet] (alamb)
- [arrow-select] perf: Replace
ArrayDatawith directArrayconstruction in take kernels #10176 [arrow] (liamzwbao) - Return PyValueError for nullable PyArrow struct imports #10174 (fallintoplace)
- Fix Variant time microsecond JSON formatting #10173 (fallintoplace)
- Split traits for plain and bitpacked decoding and fix soundness issue in BitReader::get_batch #10172 [parquet] (jhorstmann)
- fix: switch generic usages of
i128toIntervalMonthDayNanofor MonthDayNano type #10171 [arrow] (Jefffrey) - chore: specify
--lockedwhen cargo installingcargo-audit#10170 (Jefffrey) - chore: Fix clippy::useless_borrows_in_formatting #10163 [parquet] [arrow] (Tpt)
- fix(arrow-cast): respect cast safety for overflowing temporal casts #10162 [arrow] (SAY-5)
- chore(deps): bump actions/checkout from 6 to 7 #10159 (dependabot[bot])
- feat(parquet): add ParquetPushDecoder::peek_next_row_group() #10158 [parquet] (zhuqi-lucas)
- feat(pyarrow)
FromPyArrowonVec<T>: allow any iterable for input #10155 (Tpt) - nit: pyarrow: simplify class validation error creation #10154 (Tpt)
- [Variant] add doc reference to
VariantArrayBuilder#10152 (sdf-jkl) - feat: Adds product aggregate compute kernel #10151 [arrow] (devanbenz)
- Stricter datatype parsing for decimals, fixedsizelists and time32/64 #10147 [arrow] (Jefffrey)
- feat(arrow_csv): add header validation option #10144 [arrow] (XiNiHa)
- [Parquet] route dictionary page through the PageStore #10142 [parquet] (liamzwbao)
- chore: update pyo3 dependency to 0.29 #10134 (timsaucer)
- feat(ipc): Supports compression level configuration #10133 [arrow] (wForget)
- fix: write error for dbg output of out of range timestamps #10130 [arrow] (Jefffrey)
- [Variant]
VariantArrayfield API naming #10124 (sdf-jkl) - feat(arrow_array): add helper function to create MapArray from
Vec<Option<Vec<(Key, Option<Value>)>>>for tests #10123 [arrow] [arrow-flight] (rluvaton) - perf(arrow-ipc): Add writer benchmarks for dictionaries #10122 [arrow] (JakeDern)
- feat: support
MapArrayin lengths kernel #10121 [arrow] (rluvaton) - feat: add
OffsetBuffer::subtractto allow to shift offsets by value #10120 [arrow] (rluvaton) - fix:
Buffer::into_mutablereturn error instead of panic for converting owned sliced when not start at 0 and fix returned Mutable length #10118 [arrow] (rluvaton) - chore: update
Bytesvisibility to correctly reflect the actual visibility #10115 [arrow] (rluvaton) - fix(parquet_derive): support raw identifiers as column names #10113 (cbmixx)
- removed clippy ignore statment #10111 [arrow] (Rich-T-kid)
- Add
StructArray::field_APIs symmetric toStructArray::column_ones #10110 [arrow] (sdf-jkl) - fix(parquet): return error instead of panicking in pad_nulls on corrupt input #10108 [parquet] (thepenguinco)
- Minor: Add interleave tests for List<Decimal128> and List<Timestamp(tz)> #10099 [arrow] (alamb)
- Add arrow-flight test coverage for IPC compression #10097 [arrow] [arrow-flight] (alamb)
- chore(deps): bump pyspark from 3.3.2 to 3.4.4 in /parquet/pytest #10091 [parquet] (dependabot[bot])
- refactor(parquet): bundle array reader recursion args into
ReaderArgs#10089 [parquet] (HippoBaro) - arrow-buffer: implement Saturating, Checked num-traits for i256 #10088 [arrow] (theirix)
- bench(parquet): add nested list writer benchmarks #10084 [parquet] (mapleFU)
- Implement From<i128> for i256 #10081 [arrow] (AdamGS)
- test(parquet): drop confusing
mainreference in page-roundtrip test comment #10072 [parquet] (adriangb) - ci: Split miri tests into 4 parallel shards #10067 (AdamGS)
- Add tests and fix corner cases for Parquet/GeoArrow extension type conversion #10065 [parquet] (paleolimbot)
- Support writing REE arrays directly to Parquet #10064 [parquet] (Rich-T-kid)
- test(arrow-select): additional tests for inline-view filter fast path (tests for #9755) #10054 [arrow] (alamb)
- test(arrow-select): add take_bytes coverage for sliced values and nullable offset overflow #10053 [arrow] (alamb)
- Consolidate
filter_null_maskintoFilterPredicate::filter_nulls#10049 [arrow] (alamb) - [Variant] Add
VariantBuildervalues check #10016 (sdf-jkl) - [Variant] Preserve
UUIDextension type metadata for Parquet writer #10015 (sdf-jkl) - feat(parquet-variant): add Dictionary and REE variant_to_arrow support #10014 (mneetika)
- perf(arrow-ord): Avoid full index materialization for small-limit lexsorts #9991 [arrow] (pchintar)
* This Changelog was automatically generated by github_changelog_generator
Release notes
Open source →Implemented enhancements:
- Fast path for nested
DictionaryArraycasting #10247 [arrow] - parquet/arrow: reading multiple nested columns fails with "Not all children array length are the same!" when a list continues across DataPageV2 page boundary #10243 [parquet]
- Add product aggregate kernel to arrow-rs #10150 [arrow]
- Stricter
DataTypeparsing #10146 [arrow] - Support validating CSV headers against Schema #10143 [arrow]
- arrow-ipc: Supports compression level configuration for arrow-ipc writer #10132 [arrow]
- [Variant]
VariantArrayfield API naming #10093 - Add
StructArray::field_APIs symmetric toStructArray::column_ones #10092 [arrow] - arrow-buffer: implement Saturating, CheckedShl, Not num-traits for i256 #10087 [arrow]
- feat: native concat for
MapArray#10047 [arrow] - [Variant] Add
variant_to_arrowDictionary/REEtype support #10013
Fixed bugs:
- arrow-row on fixed size binary/list with size 0 and no nulls return wrong length #10270
- casting list to 0-size fixedsizelist can cause incorrect output length #10227 [arrow]
- Buffer count mismatched with metadata when encoding records with dictionary of dictionaries #10213 [arrow]
Buffer::into_mutableis not consistent regarding sliced data and can lead to panics #10117 [arrow]- parquet_derive: cannot read or write columns whose name is a Rust keyword (raw identifiers like r#type become column "r#type") #10112
- parquet: fix OffsetBuffer panic on corrupt input #10107 [parquet]
- Parquet geospatial conversion uses metadata key "algorithm" instead of "edges" in geoarrow metadata #9929 [parquet]
Documentation updates:
- doc: More comments to
concat_batches#10178 (2010YOUY01) - Minor: improve PageStore docs with a temp-file spilling example #10074 [parquet] (alamb)
Performance improvements:
- perf: interleave_list for List<Primitive> could be optimized? #10022 [arrow]
- perf(arrow-ord): Avoid full index materialization for small-limit lexsorts #9990 [arrow]
- Replace conversion of binary->string in arrow-row from arraydata to direct construction #10261 [arrow] (Jefffrey)
- [arrow-flight encode path]re-use flatbufferbuilder #10220 [arrow] (Rich-T-kid)
- [10125] arrow-flight decode path optimizations (add
skip_validationto arrow-flight) #10206 [arrow] [arrow-flight] (Rich-T-kid) - Improve performance of
concat_elementsByteViewArray concatenation #10161 [arrow] (pepijnve) - [arrow-flight] Optimize flight, remove some allocations, add dictionary focused benchmarks #10126 [arrow] [arrow-flight] (Rich-T-kid)
- optimize(concat): concat map implementation #10048 [arrow] (mapleFU)
- Reduce copies in Arrow IPC writer #10044 [arrow] (Rich-T-kid)
- perf(interleave): Optimize list interleave_list when child is primitive #10025 [arrow] (mapleFU)
Closed issues:
- Soundness: Unsound alignment contract in public
FromBytestrait andBitReader::get_batch#10164 [parquet] - ParquetPushDecoder: expose the next row-group index that try_next_reader will yield #10148 [parquet]
- arrow-ipc: Extend writer benchmarks to include dictionaries #10119 [arrow]
- bench(parquet): benchmark for nested list write #10083 [parquet]
- Support i256 implement From<i128> #10080 [arrow]
Merged pull requests:
- fix(arrow-row): allow to convert non empty fixed size binary/list array with size length 0 and no nulls #10271 [arrow] (rluvaton)
- chore: Fix audit CI run by ignore quick-xml audit advisories #10267 (alamb)
- fix main: parquet test compilation failure #10266 [parquet] (Jefffrey)
- minor: drive-by refactors for dicts in substring & filter #10264 [arrow] (Jefffrey)
- Add validated row decode benchmark #10259 [arrow] (alamb)
- arrow-cast: Add optimized path for unnesting a dict #10248 [arrow] (brancz)
- feat: support uuid from fixed type of length 16 #10241 [arrow] (ariel-miculas)
- chore(deps): bump actions/cache from 6.0.0 to 6.1.0 #10240 (dependabot[bot])
- fix: Rename parquet feature flag 'flate2-rust_backened' to 'flate2-rust_backend' #10239 [parquet] (dannycjones)
- chore: Make clippy::question_mark happy #10231 (Tpt)
- fix(ipc): reject dictionary-encoded dictionary values #10230 [arrow] (goutamadwant)
- Replace
ArrayDatawith directArrayconstruction inarrow-row#10229 [arrow] (Jefffrey) - fix: casting list to fixedsizelist didn't respect input length #10228 [arrow] (Jefffrey)
- chore: Fix clippy::byte_char_slices (use byte strings instead of explicit arrays) #10225 [parquet] [arrow] (Tpt)
- nit: arrow-pyarrow: Use string interning #10224 (Tpt)
- Support concatenation of mixed FixedSizeBinary via
concat_elements_dyn#10222 [arrow] (pepijnve) - rename Compression struct #10221 [arrow] [arrow-flight] (Rich-T-kid)
- chore(deps): bump the all-other-cargo-deps group across 1 directory with 16 updates #10218 (dependabot[bot])
- chore(deps): bump actions/setup-python from 6.2.0 to 6.3.0 #10210 (dependabot[bot])
- [10125] Introduce mult-batch decode benchmarks #10207 [arrow] [arrow-flight] (Rich-T-kid)
- chore(deps): bump actions/cache from 5.0.5 to 6.0.0 #10203 (dependabot[bot])
- introduce decode benchmarks #10202 [arrow] [arrow-flight] (Rich-T-kid)
- Fix
merge_kernelsbenchmark panic due to not wrapping withScalar#10199 [arrow] (Jefffrey) - Benchmarks and performance improvement for parquet boolean reader #10196 [parquet] (jhorstmann)
- add stale PR workflow #10194 (Jefffrey)
- chore: group minor/patch dependabot updates #10193 (Jefffrey)
- chore(deps): bump http from 1.4.0 to 1.4.2 #10191 (dependabot[bot])
- chore(deps): bump syn from 2.0.117 to 2.0.118 #10190 (dependabot[bot])
- chore(deps): bump chrono from 0.4.44 to 0.4.45 #10188 (dependabot[bot])
- chore(deps): bump uuid from 1.23.1 to 1.23.3 #10186 (dependabot[bot])
- chore: run
cargo updateto bump quinn #10181 (Jefffrey) - test: cover signed integers and bool in BitReader::get_batch test #10180 [parquet] (alamb)
- [arrow-select] perf: Replace
ArrayDatawith directArrayconstruction in take kernels #10176 [arrow] (liamzwbao) - Return PyValueError for nullable PyArrow struct imports #10174 (fallintoplace)
- Fix Variant time microsecond JSON formatting #10173 (fallintoplace)
- Split traits for plain and bitpacked decoding and fix soundness issue in BitReader::get_batch #10172 [parquet] (jhorstmann)
- fix: switch generic usages of
i128toIntervalMonthDayNanofor MonthDayNano type #10171 [arrow] (Jefffrey) - chore: specify
--lockedwhen cargo installingcargo-audit#10170 (Jefffrey) - chore: Fix clippy::useless_borrows_in_formatting #10163 [parquet] [arrow] (Tpt)
- fix(arrow-cast): respect cast safety for overflowing temporal casts #10162 [arrow] (SAY-5)
- chore(deps): bump actions/checkout from 6 to 7 #10159 (dependabot[bot])
- feat(parquet): add ParquetPushDecoder::peek_next_row_group() #10158 [parquet] (zhuqi-lucas)
- feat(pyarrow)
FromPyArrowonVec<T>: allow any iterable for input #10155 (Tpt) - nit: pyarrow: simplify class validation error creation #10154 (Tpt)
- [Variant] add doc reference to
VariantArrayBuilder#10152 (sdf-jkl) - feat: Adds product aggregate compute kernel #10151 [arrow] (devanbenz)
- Stricter datatype parsing for decimals, fixedsizelists and time32/64 #10147 [arrow] (Jefffrey)
- feat(arrow_csv): add header validation option #10144 [arrow] (XiNiHa)
- [Parquet] route dictionary page through the PageStore #10142 [parquet] (liamzwbao)
- chore: update pyo3 dependency to 0.29 #10134 (timsaucer)
- feat(ipc): Supports compression level configuration #10133 [arrow] (wForget)
- fix: write error for dbg output of out of range timestamps #10130 [arrow] (Jefffrey)
- [Variant]
VariantArrayfield API naming #10124 (sdf-jkl) - feat(arrow_array): add helper function to create MapArray from
Vec<Option<Vec<(Key, Option<Value>)>>>for tests #10123 [arrow] [arrow-flight] (rluvaton) - perf(arrow-ipc): Add writer benchmarks for dictionaries #10122 [arrow] (JakeDern)
- feat: support
MapArrayin lengths kernel #10121 [arrow] (rluvaton) - feat: add
OffsetBuffer::subtractto allow to shift offsets by value #10120 [arrow] (rluvaton) - fix:
Buffer::into_mutablereturn error instead of panic for converting owned sliced when not start at 0 and fix returned Mutable length #10118 [arrow] (rluvaton) - chore: update
Bytesvisibility to correctly reflect the actual visibility #10115 [arrow] (rluvaton) - fix(parquet_derive): support raw identifiers as column names #10113 (cbmixx)
- removed clippy ignore statment #10111 [arrow] (Rich-T-kid)
- Add
StructArray::field_APIs symmetric toStructArray::column_ones #10110 [arrow] (sdf-jkl) - fix(parquet): return error instead of panicking in pad_nulls on corrupt input #10108 [parquet] (thepenguinco)
- Minor: Add interleave tests for List<Decimal128> and List<Timestamp(tz)> #10099 [arrow] (alamb)
- Add arrow-flight test coverage for IPC compression #10097 [arrow] [arrow-flight] (alamb)
- chore(deps): bump pyspark from 3.3.2 to 3.4.4 in /parquet/pytest #10091 [parquet] (dependabot[bot])
- refactor(parquet): bundle array reader recursion args into
ReaderArgs#10089 [parquet] (HippoBaro) - arrow-buffer: implement Saturating, Checked num-traits for i256 #10088 [arrow] (theirix)
- bench(parquet): add nested list writer benchmarks #10084 [parquet] (mapleFU)
- Implement From<i128> for i256 #10081 [arrow] (AdamGS)
- test(parquet): drop confusing
mainreference in page-roundtrip test comment #10072 [parquet] (adriangb) - ci: Split miri tests into 4 parallel shards #10067 (AdamGS)
- Add tests and fix corner cases for Parquet/GeoArrow extension type conversion #10065 [parquet] (paleolimbot)
- Support writing REE arrays directly to Parquet #10064 [parquet] (Rich-T-kid)
- test(arrow-select): additional tests for inline-view filter fast path (tests for #9755) #10054 [arrow] (alamb)
- test(arrow-select): add take_bytes coverage for sliced values and nullable offset overflow #10053 [arrow] (alamb)
- Consolidate
filter_null_maskintoFilterPredicate::filter_nulls#10049 [arrow] (alamb) - [Variant] Add
VariantBuildervalues check #10016 (sdf-jkl) - [Variant] Preserve
UUIDextension type metadata for Parquet writer #10015 (sdf-jkl) - feat(parquet-variant): add Dictionary and REE variant_to_arrow support #10014 (mneetika)
- perf(arrow-ord): Avoid full index materialization for small-limit lexsorts #9991 [arrow] (pchintar)
- Fast path for nested
-
59.0.009 Jun 2026Release notes
Open source →Changelog
59.0.0 (2026-06-04)
Breaking changes:
- chore: Remove some deprecated Arrow functions from the public API #10040 [arrow] (etseidl)
- chore: Remove some deprecated functions from parquet crate #10035 [parquet] (etseidl)
- Replace
From<Vec<_>>impls withTryFroms forFixedSizeBinaryArray#10019 [parquet] [arrow] (quantumish) - Use Thrift macro to generate Parquet
LogicalTypeserialization code #9997 [parquet] (etseidl) - refactor: make
BloomFilterPropertiesfpp/ndv private with accessors #9969 [parquet] (CuteChuanChuan) - Remove deprecated parquet::format module and thrift dependency #9962 [parquet] (jhorstmann)
- generic channel support for FlightClient #9933 [arrow] [arrow-flight] (rumenov)
- Add
CompressionCodecThrift enum for Parquet metadata #9864 [parquet] (etseidl) - [Variant] remove
BorrowedShreddingState#9791 (sdf-jkl) - Remove deprecated legacy
likekernels inarrow-string#9674 [arrow] (AdamGS)
Implemented enhancements:
- Allow casting plain struct to dictionary encoded struct #10038 [arrow]
- Optimize arrow-flight #10029
- Align buffers when importing via
from_ffi/ArrowArrayStreamReader#10028 [arrow] - Switch Parquet
LogicalTypeenum to macro generated version #9995 [parquet] - Future proof Parquet Thrift parser #9973 [parquet]
- Add
DatePart1-indexed variants #9964 [arrow] - perf: Rework Parquet Thrift handling of boolean fields #9946 [parquet]
- Add benchmarks for REE to parquet #9935 [parquet] [arrow]
- (re) Allow Large
FixedSizeBinaryArrays #9906 [arrow] - Add a is_normalized flag to DictionaryArray #9841
- [Variant] Remove
BorrowedShreddingState#9790 - [parquet] Expose whether FileDecryptionProperties uses a KeyRetriever #9721 [parquet]
- Align cast logic for from/to_decimal for variant to cast kernel #9688 [arrow]
Fixed bugs:
- parquet-variant build might fail on s390x #10026
FixedSizeBinaryArrayimplementsFrom<Vec<&[u8]>>etc despite conversion being fallible #10018 [parquet] [arrow]- string -> decimal cast should not treat empty string as 0 #10009 [arrow]
- Cast FixedSizeList to List will lost datatype metadata in list #10004 [arrow]
- Record reader panics with "index out of bounds" when row group num_rows exceeds actual column data #9992 [parquet]
- parquet predicate-cache: panic / silent row drop on single-leaf nullable struct #9982 [parquet]
- parquet-variant doesn't build on 32-bit targets #9977
- Date32 doesn't parse date with large year #9960 [arrow]
- msrv check failing on main due to
[email protected]#9938 [parquet] [arrow] [arrow-flight]
Documentation updates:
- Release arrow-rs / parquet Minor/Patch version
58.3.0or58.2.1(May 2026) #9859 - Add docs for
BitWriter#9949 [parquet] (alamb) - Add docs for
BitReader#9948 [parquet] (alamb)
Performance improvements:
- perf: parquet LevelInfoBuilder::write_list can be optimized? #10023 [parquet]
- perf(parquet): LevelInfoBuilder batch write when no repetition childs #10037 [parquet] (mapleFU)
- [arrow-select] Replace
ArrayDatawith directArrayconstruction in filter kernels #9986 [arrow] (liamzwbao) - Bulk-fill definition levels for majority-null leaf columns #9967 [parquet] (RyanJamesStewart)
- perf: Remove
bool_valfrom Parquet ThriftFieldIdentifier#9945 [parquet] (etseidl) - feat(parquet): compact level representation with generic writer dispatch #9831 [parquet] (HippoBaro)
Closed issues:
- Bound ArrowWriter peak memory #10071 [parquet]
- Parquet writer can produce massively oversized data pages for large variable-width values #10061 [parquet]
- Remove the
fused_inline_view_columnsfield fromBatchCoalescerif possible #10055 - DataType parser permits negative FixedSizeBinary size #10033 [arrow]
- Parquet: return error for overlong INT96 column metadata statistics #10002 [parquet]
Uuidextension type fails to deserialize whenARROW:extension:metadatais an empty string #10000 [arrow]- parquet: timeline for removing thrift crate dependency (CVE-2026-43868) #9999
- Failure in CI:
Archery test With other arrows-binary_view Rust producing, .NET consuming#9989 [arrow] - Validate FIXED_LEN_BYTE_ARRAY type_length for DECIMAL and INTERVAL in Parquet → Arrow schema conversion #9984 [parquet]
- IPC reader projection does not handle duplicate projection indices correctly #9950 [arrow]
AnyRunArraytrait #9909 [arrow]- Release arrow-rs / parquet Patch version
57.3.1(May 2026) #9858 [arrow] - Release arrow-rs / parquet Patch version
56.2.1(May 2026) #9857 [arrow] - parquet/arrow: should sync/async readers converge on a shared physical read planner #9764
arrow-stringhas a lot of macro-generated deprecated kernels inlike.rs#9675 [arrow]- [parquet] Add BloomFilterProperties builder API to make bloom filter configuration explicit #9667 [parquet]
Merged pull requests:
- Bump max throughput in
flightbenchmark before blocking #10070 [arrow] [arrow-flight] (Rich-T-kid) - Add coalesce inline-view filter benchmarks #10050 [arrow] (ClSlaid)
- fix: better error handling for negative size of FixedSizeBinary #10042 [arrow] (theirix)
- bench(parquet): add Sbbf check/insert benchmarks #10041 [parquet] (dmatth1)
- arrow-cast: Add ability to cast plain struct to dictionary #10039 [arrow] (brancz)
- [#10029][benchmarks] arrow-flight roundtrip as well as encode/decode #10031 [arrow] [arrow-flight] (Rich-T-kid)
- Call
align_buffers()infrom_ffi, remove redundant call fromarrow-pyarrow#10030 [arrow] (mbutrovich) - Adjust Variant size expectation for s390x architecture #10027 (frantisekz)
- bench(parquet): add short and large string
arrow_writerbenchmarks #10021 [parquet] (adriangb) - Pluggable page spilling API for the Parquet ArrowWriter (PageStore) #10020 [parquet] (adriangb)
- fix: Reject empty strings when casting strings to decimal #10010 [arrow] (neilconway)
- feat: Implement decimal <-> float16 casts #10008 [arrow] (neilconway)
- fix(cast): Trying to fix cast losting schema problem #10005 [arrow] (mapleFU)
- fix(parquet): validate INT96 column metadata statistics #10003 [parquet] (fallintoplace)
- fix(arrow-schema): allow empty metadata value for UUID extension type #10001 [arrow] (asubiotto)
- Add helper functions to create
LogicalTypestruct variants #9996 [parquet] (etseidl) - fix: prevent panic in record reader when row group metadata overcounts num_rows #9993 [parquet] (BoazC-MSFT)
- feat: extract
has_falseandhas_truefrom BooleanArray toBooleanBufferand reuse for no nulls #9987 [arrow] (rluvaton) - Validate FIXED_LEN_BYTE_ARRAY length for DECIMAL and INTERVAL types #9985 [parquet] (CynicDog)
- fix(parquet): exclude single-leaf struct roots from predicate cache #9983 [parquet] (imhy)
- Adds is_null function to RowAccessor #9979 [parquet] (choubacha)
- Fix parquet-variant build on wasm targets #9978 (AdamGS)
- Safely ignore Parquet fields with unimplemented Thrift types #9974 [parquet] (etseidl)
- fix(parquet): bound data page byte size for large variable-width values #9972 [parquet] (adriangb)
- feat(parquet): Add
ParquetPushDecoder::into_builderto allow swapping projections / row filters at row group boundaries #9968 [parquet] (adriangb) - chore(deps): bump peaceiris/actions-gh-pages from 4.0.0 to 4.1.0 #9966 (dependabot[bot])
- Add
DatePartenum 1-indexed variants #9965 [arrow] (sdf-jkl) - fix(arrow-cast): support full Date32 range when parsing extended-year dates #9961 [arrow] (swanandx)
- Implement AnyRee #9959 [arrow] (Rich-T-kid)
- test: add overflow tests for MutableBuffer #9958 [arrow] (SoimanVasile)
- feat(parquet): generalize value encoder inputs #9955 [parquet] (HippoBaro)
- feat(parquet): add all-null fast paths for level building #9954 [parquet] (HippoBaro)
- fix(ipc): handle duplicate projection indices in IPC reader #9952 [arrow] (pchintar)
- Fix MSRV check by checking in Cargo.lock #9941 (alamb)
- benchmarks for writing REE arrays to parquet #9936 [parquet] [arrow] (Rich-T-kid)
- Validate encoded Thrift lists match the schema #9924 [parquet] (etseidl)
- [arrow-array] use usize arithmetic in FixedSizeBinaryArray, aggressive overflow checks #9910 [arrow] (alamb)
- feat(parquet): add uses_key_retriever method to FileDecryptionProperties #9895 [parquet] (adamreeve)
- Support ListView/BinaryView/RunEndEncoded types in integration test JSON parser #9888 [arrow] (paleolimbot)
- feat(parquet): add BloomFilterPropertiesBuilder #9877 [parquet] (CuteChuanChuan)
- perf[arrow-select]: add specialized REE interleave #9856 [arrow] (asubiotto)
- bench(parquet): add
ListArraybenchmarks for runtime and peak memory #9846 [parquet] (HippoBaro) - feat(parquet): separate push decoder frontier state from row-group decoding #9804 [parquet] (HippoBaro)
- arrow: add oversized coalesce take benchmarks #9799 [arrow] (ClSlaid)
- Remove redundant benchmarks in
cast_kernels#9789 [arrow] (alamb) - [Variant] Align cast logic for from/to_decimal for variant #9689 [arrow] (klion26)
- [Parquet]: GH-563: Make
path_in_schemaoptional #9678 [parquet] (etseidl) - Add support for FixedSizeList to variant_to_arrow #9663 (rishvin)
- Reduce Miri runtime even more #9650 [arrow] (AdamGS)
* This Changelog was automatically generated by github_changelog_generator
Release notes
Open source →Breaking changes:
- chore: Remove some deprecated Arrow functions from the public API #10040 [arrow] (etseidl)
- chore: Remove some deprecated functions from parquet crate #10035 [parquet] (etseidl)
- Replace
From<Vec<_>>impls withTryFroms forFixedSizeBinaryArray#10019 [parquet] [arrow] (quantumish) - Use Thrift macro to generate Parquet
LogicalTypeserialization code #9997 [parquet] (etseidl) - refactor: make
BloomFilterPropertiesfpp/ndv private with accessors #9969 [parquet] (CuteChuanChuan) - Remove deprecated parquet::format module and thrift dependency #9962 [parquet] (jhorstmann)
- generic channel support for FlightClient #9933 [arrow] [arrow-flight] (rumenov)
- Add
CompressionCodecThrift enum for Parquet metadata #9864 [parquet] (etseidl) - [Variant] remove
BorrowedShreddingState#9791 (sdf-jkl) - Remove deprecated legacy
likekernels inarrow-string#9674 [arrow] (AdamGS)
Implemented enhancements:
- Allow casting plain struct to dictionary encoded struct #10038 [arrow]
- Optimize arrow-flight #10029
- Align buffers when importing via
from_ffi/ArrowArrayStreamReader#10028 [arrow] - Switch Parquet
LogicalTypeenum to macro generated version #9995 [parquet] - Future proof Parquet Thrift parser #9973 [parquet]
- Add
DatePart1-indexed variants #9964 [arrow] - perf: Rework Parquet Thrift handling of boolean fields #9946 [parquet]
- Add benchmarks for REE to parquet #9935 [parquet] [arrow]
- (re) Allow Large
FixedSizeBinaryArrays #9906 [arrow] - Add a is_normalized flag to DictionaryArray #9841
- [Variant] Remove
BorrowedShreddingState#9790 - [parquet] Expose whether FileDecryptionProperties uses a KeyRetriever #9721 [parquet]
- Align cast logic for from/to_decimal for variant to cast kernel #9688 [arrow]
Fixed bugs:
- parquet-variant build might fail on s390x #10026
FixedSizeBinaryArrayimplementsFrom<Vec<&[u8]>>etc despite conversion being fallible #10018 [parquet] [arrow]- string -> decimal cast should not treat empty string as 0 #10009 [arrow]
- Cast FixedSizeList to List will lost datatype metadata in list #10004 [arrow]
- Record reader panics with "index out of bounds" when row group num_rows exceeds actual column data #9992 [parquet]
- parquet predicate-cache: panic / silent row drop on single-leaf nullable struct #9982 [parquet]
- parquet-variant doesn't build on 32-bit targets #9977
- Date32 doesn't parse date with large year #9960 [arrow]
- msrv check failing on main due to
[email protected]#9938 [parquet] [arrow] [arrow-flight]
Documentation updates:
- Release arrow-rs / parquet Minor/Patch version
58.3.0or58.2.1(May 2026) #9859 - Add docs for
BitWriter#9949 [parquet] (alamb) - Add docs for
BitReader#9948 [parquet] (alamb)
Performance improvements:
- perf: parquet LevelInfoBuilder::write_list can be optimized? #10023 [parquet]
- perf(parquet): LevelInfoBuilder batch write when no repetition childs #10037 [parquet] (mapleFU)
- [arrow-select] Replace
ArrayDatawith directArrayconstruction in filter kernels #9986 [arrow] (liamzwbao) - Bulk-fill definition levels for majority-null leaf columns #9967 [parquet] (RyanJamesStewart)
- perf: Remove
bool_valfrom Parquet ThriftFieldIdentifier#9945 [parquet] (etseidl) - feat(parquet): compact level representation with generic writer dispatch #9831 [parquet] (HippoBaro)
Closed issues:
- Bound ArrowWriter peak memory #10071 [parquet]
- Parquet writer can produce massively oversized data pages for large variable-width values #10061 [parquet]
- Remove the
fused_inline_view_columnsfield fromBatchCoalescerif possible #10055 - DataType parser permits negative FixedSizeBinary size #10033 [arrow]
- Parquet: return error for overlong INT96 column metadata statistics #10002 [parquet]
Uuidextension type fails to deserialize whenARROW:extension:metadatais an empty string #10000 [arrow]- parquet: timeline for removing thrift crate dependency (CVE-2026-43868) #9999
- Failure in CI:
Archery test With other arrows-binary_view Rust producing, .NET consuming#9989 [arrow] - Validate FIXED_LEN_BYTE_ARRAY type_length for DECIMAL and INTERVAL in Parquet → Arrow schema conversion #9984 [parquet]
- IPC reader projection does not handle duplicate projection indices correctly #9950 [arrow]
AnyRunArraytrait #9909 [arrow]- Release arrow-rs / parquet Patch version
57.3.1(May 2026) #9858 [arrow] - Release arrow-rs / parquet Patch version
56.2.1(May 2026) #9857 [arrow] - parquet/arrow: should sync/async readers converge on a shared physical read planner #9764
arrow-stringhas a lot of macro-generated deprecated kernels inlike.rs#9675 [arrow]- [parquet] Add BloomFilterProperties builder API to make bloom filter configuration explicit #9667 [parquet]
Merged pull requests:
- Bump max throughput in
flightbenchmark before blocking #10070 [arrow] [arrow-flight] (Rich-T-kid) - Add coalesce inline-view filter benchmarks #10050 [arrow] (ClSlaid)
- fix: better error handling for negative size of FixedSizeBinary #10042 [arrow] (theirix)
- bench(parquet): add Sbbf check/insert benchmarks #10041 [parquet] (dmatth1)
- arrow-cast: Add ability to cast plain struct to dictionary #10039 [arrow] (brancz)
- [#10029][benchmarks] arrow-flight roundtrip as well as encode/decode #10031 [arrow] [arrow-flight] (Rich-T-kid)
- Call
align_buffers()infrom_ffi, remove redundant call fromarrow-pyarrow#10030 [arrow] (mbutrovich) - Adjust Variant size expectation for s390x architecture #10027 (frantisekz)
- bench(parquet): add short and large string
arrow_writerbenchmarks #10021 [parquet] (adriangb) - Pluggable page spilling API for the Parquet ArrowWriter (PageStore) #10020 [parquet] (adriangb)
- fix: Reject empty strings when casting strings to decimal #10010 [arrow] (neilconway)
- feat: Implement decimal <-> float16 casts #10008 [arrow] (neilconway)
- fix(cast): Trying to fix cast losting schema problem #10005 [arrow] (mapleFU)
- fix(parquet): validate INT96 column metadata statistics #10003 [parquet] (fallintoplace)
- fix(arrow-schema): allow empty metadata value for UUID extension type #10001 [arrow] (asubiotto)
- Add helper functions to create
LogicalTypestruct variants #9996 [parquet] (etseidl) - fix: prevent panic in record reader when row group metadata overcounts num_rows #9993 [parquet] (BoazC-MSFT)
- feat: extract
has_falseandhas_truefrom BooleanArray toBooleanBufferand reuse for no nulls #9987 [arrow] (rluvaton) - Validate FIXED_LEN_BYTE_ARRAY length for DECIMAL and INTERVAL types #9985 [parquet] (CynicDog)
- fix(parquet): exclude single-leaf struct roots from predicate cache #9983 [parquet] (imhy)
- Adds is_null function to RowAccessor #9979 [parquet] (choubacha)
- Fix parquet-variant build on wasm targets #9978 (AdamGS)
- Safely ignore Parquet fields with unimplemented Thrift types #9974 [parquet] (etseidl)
- fix(parquet): bound data page byte size for large variable-width values #9972 [parquet] (adriangb)
- feat(parquet): Add
ParquetPushDecoder::into_builderto allow swapping projections / row filters at row group boundaries #9968 [parquet] (adriangb) - chore(deps): bump peaceiris/actions-gh-pages from 4.0.0 to 4.1.0 #9966 (dependabot[bot])
- Add
DatePartenum 1-indexed variants #9965 [arrow] (sdf-jkl) - fix(arrow-cast): support full Date32 range when parsing extended-year dates #9961 [arrow] (swanandx)
- Implement AnyRee #9959 [arrow] (Rich-T-kid)
- test: add overflow tests for MutableBuffer #9958 [arrow] (SoimanVasile)
- feat(parquet): generalize value encoder inputs #9955 [parquet] (HippoBaro)
- feat(parquet): add all-null fast paths for level building #9954 [parquet] (HippoBaro)
- fix(ipc): handle duplicate projection indices in IPC reader #9952 [arrow] (pchintar)
- Fix MSRV check by checking in Cargo.lock #9941 (alamb)
- benchmarks for writing REE arrays to parquet #9936 [parquet] [arrow] (Rich-T-kid)
- Validate encoded Thrift lists match the schema #9924 [parquet] (etseidl)
- [arrow-array] use usize arithmetic in FixedSizeBinaryArray, aggressive overflow checks #9910 [arrow] (alamb)
- feat(parquet): add uses_key_retriever method to FileDecryptionProperties #9895 [parquet] (adamreeve)
- Support ListView/BinaryView/RunEndEncoded types in integration test JSON parser #9888 [arrow] (paleolimbot)
- feat(parquet): add BloomFilterPropertiesBuilder #9877 [parquet] (CuteChuanChuan)
- perf[arrow-select]: add specialized REE interleave #9856 [arrow] (asubiotto)
- bench(parquet): add
ListArraybenchmarks for runtime and peak memory #9846 [parquet] (HippoBaro) - feat(parquet): separate push decoder frontier state from row-group decoding #9804 [parquet] (HippoBaro)
- arrow: add oversized coalesce take benchmarks #9799 [arrow] (ClSlaid)
- Remove redundant benchmarks in
cast_kernels#9789 [arrow] (alamb) - [Variant] Align cast logic for from/to_decimal for variant #9689 [arrow] (klion26)
- [Parquet]: GH-563: Make
path_in_schemaoptional #9678 [parquet] (etseidl) - Add support for FixedSizeList to variant_to_arrow #9663 (rishvin)
- Reduce Miri runtime even more #9650 [arrow] (AdamGS)
-
58.4.022 Jul 2026Release notes
Open source →Changelog
58.4.0 (2026-07-17)
Merged pull requests:
- [58_maintenance] [parquet] Allow more encryption algorithms (#9203) #10351 [parquet] (mbutrovich)
- [58_maintenance] Backport cargo audit fixes #10369 (alamb)
- [58_maintenance] chore: Ignore py03 vulnerabilities until upgrade #10370 (alamb)
- [58_maintenance] Add test for `parquet-testing/bad_data/ARROW-GH-47662.parquet` (#10077) #10371 [parquet] (alamb)
* This Changelog was automatically generated by github_changelog_generator
-
58.3.011 May 2026Release notes
Open source →Changelog
58.3.0 (2026-05-07)
Implemented enhancements:
- Add
DatePart::from_strAPI #9930 [arrow] - should use DictionaryArray::with_values instead of try_new on the dictionary fast path #9889 [arrow]
- [arrow-string] add concat_elements for BinaryViewArray and FixedSizeBinary #9875 [arrow]
- Expose eq ignore ascii case from arrow-string #9870 [arrow]
- Configurable data page v2 compression threshold #9827 [parquet]
Fixed bugs:
- [arrow-cast] incorrect Time32 -> Time64 conversion #9851 [arrow]
- Panic when reading malformed compact-Thrift bool fields in Parquet page metadata #9839 [parquet]
- Parquet
DeltaBitPackDecoder::skipcould panic on "non-standard" miniblocks #9793 [parquet]
Documentation updates:
- docs: Add guidance for AI assisted submissions to CONTRIBUTING.md #9892 (etseidl)
- Update release schedule on README #9881 (alamb)
- Add more documentation for FixedSizeBinary arrays #9866 [arrow] (alamb)
- Minor: document why FixedSizeBinary offset is always 0 #9861 [arrow] (alamb)
- docs: Update contributing guidelines with benchmark results #9782 (alamb)
Closed issues:
- GenericByteDictionaryBuilder::with_capacity does not pre-size dedup HashTable #9907 [arrow]
- [arrow-buffer] Integer overflow in repeat_slice_n_times leads to undefined behavior #9904 [arrow]
- [arrow-buffer] Integer overflow in BitChunks::new leads to undefined behavior #9903 [arrow]
- [arrow-row] Integer overflow in Rows::row index handling leads to undefined behavior #9901 [arrow]
- [arrow-data] Integer overflow in ArrayData validation leads to undefined behavior #9900 [arrow]
- [arrow-data] Integer overflow in ArrayData::slice leads to undefined behavior #9899 [arrow]
- [arrow-array] Integer overflow in FixedSizeBinaryArray::value leads to undefined behavior #9898 [arrow]
- [arrow-buffer] Integer overflow in BufferBuilder::reserve leads to undefined behavior #9897 [arrow]
- arrow-csv: integer overflow panic in Reader::records::flush #9885 [arrow]
- Make an API to help with the pattern of 'replaces the values of the REE array' #9854 [arrow]
- Parquet reader rejects canonical UNKNOWN logical type on BOOLEAN physical columns #9844 [parquet]
- ColumnIndex length mismatch can cause panic during decoding in Parquet #9832 [parquet]
- Bug converting json to fixed list of zero size #9780 [arrow]
Merged pull requests:
- impl
FromStrforDatePart#9931 [arrow] (sdf-jkl) - Pre-size dedup HashTable in GenericByteDictionaryBuilder::with_capacity #9908 [arrow] (rabenhorst)
- [arrow-array] Use consistent
value_lengthname in FixedSizeBinaryArray #9905 [arrow] (alamb) - replace Dictionary::try_new() calls with with_values. #9894 [arrow] (Rich-T-kid)
- API to help with the pattern of 'replaces the values of the REE array #9891 [arrow] (Rich-T-kid)
- fix(arrow-csv): bound RecordDecoder::flush offset accumulation #9886 [arrow] (masumi-ryugo)
- fix(parquet): bound schema num_children before Vec::with_capacity #9884 [parquet] (masumi-ryugo)
- feat(arrow-string): concat_elements for view, fixed binary #9876 [arrow] (theirix)
- Prevent
FixedSizeBinaryArrayi32offset overflows (try 2) #9872 [arrow] (alamb) - [arrow-string]: add
like::eq_ascii_ignore_casekernel #9871 [arrow] (albertlockett) - fix(parquet): Prevent negative list sizes in Thrift compact protocol parser #9868 [parquet] (masumi-ryugo)
- [PARQUET] Allow
UNKNOWNlogical type annotation on any physical type #9855 [parquet] (etseidl) - [arrow-ipc]: dictionary builders for delta - doc fix and integration tests for nested types #9853 [arrow] (albertlockett)
- fix(arrow-cast): fix incorrect conversion #9852 [arrow] (bboissin)
- chore[benches]: add REE interleave benchmarks #9849 [arrow] (asubiotto)
- test(parquet): replace
InMemoryArrayReaderwithPrimitiveArrayReaderin tests #9847 [parquet] (HippoBaro) - REE row conversion speed up #9845 [arrow] (Rich-T-kid)
- fix(parquet): Avoid panic on malformed thrift bool fields in parquet metadata #9840 [parquet] (BoazC-MSFT)
- fix(parquet): avoid panic on ColumnIndex length mismatch #9833 [parquet] (pchintar)
- configurable data page v2 compression threshold #9826 [parquet] (leoyvens)
- Prevent
ArrayData::slicelength overflow #9813 [arrow] (alamb) - Fix JSON reader panic for non-nullable zero-size FixedSizeList #9810 [arrow] (liamzwbao)
- [Parquet] Do not panic when trying to skip records in delta encoded files using non-standard block sizes #9794 [parquet] (etseidl)
* This Changelog was automatically generated by github_changelog_generator
Release notes
Open source →Implemented enhancements:
- Add
DatePart::from_strAPI #9930 [arrow] - should use DictionaryArray::with_values instead of try_new on the dictionary fast path #9889 [arrow]
- [arrow-string] add concat_elements for BinaryViewArray and FixedSizeBinary #9875 [arrow]
- Expose eq ignore ascii case from arrow-string #9870 [arrow]
- Configurable data page v2 compression threshold #9827 [parquet]
Fixed bugs:
- [arrow-cast] incorrect Time32 -> Time64 conversion #9851 [arrow]
- Panic when reading malformed compact-Thrift bool fields in Parquet page metadata #9839 [parquet]
- Parquet
DeltaBitPackDecoder::skipcould panic on "non-standard" miniblocks #9793 [parquet]
Documentation updates:
- docs: Add guidance for AI assisted submissions to CONTRIBUTING.md #9892 (etseidl)
- Update release schedule on README #9881 (alamb)
- Add more documentation for FixedSizeBinary arrays #9866 [arrow] (alamb)
- Minor: document why FixedSizeBinary offset is always 0 #9861 [arrow] (alamb)
- docs: Update contributing guidelines with benchmark results #9782 (alamb)
Closed issues:
- GenericByteDictionaryBuilder::with_capacity does not pre-size dedup HashTable #9907 [arrow]
- [arrow-buffer] Integer overflow in repeat_slice_n_times leads to undefined behavior #9904 [arrow]
- [arrow-buffer] Integer overflow in BitChunks::new leads to undefined behavior #9903 [arrow]
- [arrow-row] Integer overflow in Rows::row index handling leads to undefined behavior #9901 [arrow]
- [arrow-data] Integer overflow in ArrayData validation leads to undefined behavior #9900 [arrow]
- [arrow-data] Integer overflow in ArrayData::slice leads to undefined behavior #9899 [arrow]
- [arrow-array] Integer overflow in FixedSizeBinaryArray::value leads to undefined behavior #9898 [arrow]
- [arrow-buffer] Integer overflow in BufferBuilder::reserve leads to undefined behavior #9897 [arrow]
- arrow-csv: integer overflow panic in Reader::records::flush #9885 [arrow]
- Make an API to help with the pattern of 'replaces the values of the REE array' #9854 [arrow]
- Parquet reader rejects canonical UNKNOWN logical type on BOOLEAN physical columns #9844 [parquet]
- ColumnIndex length mismatch can cause panic during decoding in Parquet #9832 [parquet]
- Bug converting json to fixed list of zero size #9780 [arrow]
Merged pull requests:
- impl
FromStrforDatePart#9931 [arrow] (sdf-jkl) - Pre-size dedup HashTable in GenericByteDictionaryBuilder::with_capacity #9908 [arrow] (rabenhorst)
- [arrow-array] Use consistent
value_lengthname in FixedSizeBinaryArray #9905 [arrow] (alamb) - replace Dictionary::try_new() calls with with_values. #9894 [arrow] (Rich-T-kid)
- API to help with the pattern of 'replaces the values of the REE array #9891 [arrow] (Rich-T-kid)
- fix(arrow-csv): bound RecordDecoder::flush offset accumulation #9886 [arrow] (masumi-ryugo)
- fix(parquet): bound schema num_children before Vec::with_capacity #9884 [parquet] (masumi-ryugo)
- feat(arrow-string): concat_elements for view, fixed binary #9876 [arrow] (theirix)
- Prevent
FixedSizeBinaryArrayi32offset overflows (try 2) #9872 [arrow] (alamb) - [arrow-string]: add
like::eq_ascii_ignore_casekernel #9871 [arrow] (albertlockett) - fix(parquet): Prevent negative list sizes in Thrift compact protocol parser #9868 [parquet] (masumi-ryugo)
- [PARQUET] Allow
UNKNOWNlogical type annotation on any physical type #9855 [parquet] (etseidl) - [arrow-ipc]: dictionary builders for delta - doc fix and integration tests for nested types #9853 [arrow] (albertlockett)
- fix(arrow-cast): fix incorrect conversion #9852 [arrow] (bboissin)
- chore[benches]: add REE interleave benchmarks #9849 [arrow] (asubiotto)
- test(parquet): replace
InMemoryArrayReaderwithPrimitiveArrayReaderin tests #9847 [parquet] (HippoBaro) - REE row conversion speed up #9845 [arrow] (Rich-T-kid)
- fix(parquet): Avoid panic on malformed thrift bool fields in parquet metadata #9840 [parquet] (BoazC-MSFT)
- fix(parquet): avoid panic on ColumnIndex length mismatch #9833 [parquet] (pchintar)
- configurable data page v2 compression threshold #9826 [parquet] (leoyvens)
- Prevent
ArrayData::slicelength overflow #9813 [arrow] (alamb) - Fix JSON reader panic for non-nullable zero-size FixedSizeList #9810 [arrow] (liamzwbao)
- [Parquet] Do not panic when trying to skip records in delta encoded files using non-standard block sizes #9794 [parquet] (etseidl)
- Add
-
58.2.002 May 2026Release notes
Open source →Implemented enhancements:
- Expose ColumnCloseResult on ArrowColumnChunk #9774 [parquet]
- Expose FFI data structures fields #9771 [arrow]
- short-circuit last predicate in
RowFilterwhenwith_limit(N)is set #9765 [parquet] - vectorise dict-index bounds check #9747 [parquet]
- Refactor
RleEncoder::flush_bit_packed_run#9734 [parquet] - Add benchmark for cast from/to decimals #9728 [arrow]
- Add a security policy for arrow-rs #9727 [parquet] [arrow] [arrow-flight]
- Support
FixedSizeListin arrow-json reader #9714 [arrow] - [Variant] Add
VariantArrayBuilder::append_nullsAPI #9684 - [Json] RunEndEncoded decoder optimization #9645 [arrow]
- [Variant]
variant_get(..., List<_>)non-Struct types support #9615 - [Variant] Add unshredded
Structfast-path forvariant_get(..., Struct)#9596 - Allow setting custom line terminator for CSV writer #9571 [arrow]
- [Variant] Align cast logic for
variant_getto cast kernel for numeric/bool types #9564 [arrow] - ci: use ubuntu-slim where applicable #9536
- Publicly export
arrow_string::Predicateand its methods? #9480 - Don't create CompressionContext when no compression is selected [IPC] #9463 [arrow]
- Parquet: Raw level buffering causes unbounded memory growth for sparse columns #9446 [parquet]
- Parallel Parquet Reading #9381 [parquet]
Fixed bugs:
- [Variant]
unshred_variantpanics on malformed bytes despite returningResult#9740 - RecordBatch::normalize() does not propagate top level null bitmap into the results #9732 [arrow]
- Incorrect accounting in
DictEncoder::estimated_memory_size#9719 [parquet] - arrow-ipc writer does not comply with spec for empty variable-size arrays #9716 [arrow]
- Panic when reading corrupt parquet file with truncated data instead of ParquetError #9705 [parquet]
- NOTICE.txt is inaccurate #9703 [arrow]
- Unnecessary dependency on regex crate #9672
- [arrow-avro] Avro reader produces incorrect results when reader schema and writer schema differ #9655 [arrow]
- parquet docs are broken on docs.rs #9649
- [Parquet] ArrowWriter with CDC panics on nested ListArrays #9637 [parquet] [arrow] [arrow-flight]
- Use release KEYS file for verification instead of dev KEYS #9603
- IPC reader: handling of dictionaries with only null values #9595 [arrow]
- Parquet RleDecoder::get_batch_with_dict panics on oob dictionary indices #9434 [parquet]
Documentation updates:
- docs(variant): link VariantArray doc to official Parquet Variant extension type #9779 (mcharrel)
- Document Security Policy #9730 [parquet] [arrow] [arrow-flight] (alamb)
- Docs: add example of how to read parquet row groups in parallel #9396 [parquet] (alamb)
Performance improvements:
- parquet: avoid decode and heap allocation on terminal skip in DeltaBitPackDecoder #9784 [parquet]
- parquet: O(1) skip for bw=0 miniblocks in DeltaBitPackDecoder #9783 [parquet]
- Remove per-message flush overhead in Arrow IPC writer #9762 [arrow]
- Support
GenericListViewArray::new_uncheckedand refactor ListView json decoder #9646 [arrow] - Support nested REE in arrow-ord
partitionfunction #9640 [arrow] - [Parquet] Remove the BIT_PACKED encoder #9635 [parquet]
- Pre-reserve output capacity in ByteView/ByteArray dictionary decoding #9587 [parquet]
- Fuse RLE decoding and view gathering for StringView dictionary decoding #9582 [parquet]
- Use branchless index clamping and add get_batch_direct to RleDecoder #9581 [parquet]
- Reduce per-byte overhead in VLQ integer decoding #9580 [parquet]
- feat(parquet): batch RLE runs in level encoder via scan-ahead #9830 [parquet] (HippoBaro)
- fix: lazy-init zstd compression contexts to avoid unnecessary FFI calls #9808 [arrow] (mbutrovich)
- parquet: O(1) skip for bw=0 miniblocks in DeltaBitPackDecoder #9786 [parquet] (sahuagin)
- chore: add benchmark for row filters with LIMIT short-circuit #9767 [parquet] (haohuaijin)
- Push
LIMIT/OFFSETinto the lastRowFilterpredicate and skip unused row groups #9766 [parquet] (haohuaijin) - feat(ipc): Remove per-message flush in IPC writer hot path #9763 [arrow] (pchintar)
- perf(parquet): Defer fixed length byte array buffer alloc and skip zero-batch init #9756 [parquet] (lyang24)
- feat(parquet): batch consecutive null/empty rows in
write_list#9752 [parquet] (HippoBaro) - Remove
lenfield from buffer builder #9750 [arrow] (cetra3) - perf(parquet): Vectorize dict-index bounds check in RleDecoder::get_batch_with_dict (up to -7.9%) #9746 [parquet] (Dandandan)
- feat(parquet): precompute
offset_index_disabledat build-time #9724 [parquet] (HippoBaro) - [Parquet] Improve dictionary decoder by unrolling loops #9662 [parquet] (Dandandan)
- [Json] Use
partitionandtakein RunEndEncoded decoder #9658 [arrow] (liamzwbao) - Improve take performance on List arrays #9643 [arrow] (AdamGS)
- [Json] Replace
ArrayDatawith typed Array construction in json-reader #9497 [arrow] (liamzwbao) - feat(parquet): stream-encode definition/repetition levels incrementally #9447 [parquet] (HippoBaro)
Closed issues:
- Incorrect buffer skipping for V4 Union types in IPC
skip_field#9828 [arrow] - Replace wildcard match in
skip_fieldwith explicitDataTypehandling #9821 [arrow] - Column projection misalignment for ListView / LargeListView in IPC reader #9805 [arrow]
- Avoid panic on malformed compressed buffer prefix in IPC #9801 [arrow]
- DeltaByteArrayDecoder panics on invalid prefix lengths #9796 [parquet]
- Use NullBufferBuilder when reading json #9781 [arrow]
- Perfectly shredded arrays with top-level null values loss nullability when
typed_valueis extracted #9701 - [Parquet Metadata] API to determine page-index presence separately from page-index load #9693
- Union cast is incorrect for duplicate field names #9664 [arrow]
- List and ListView are missing
takebenchmarks #9627 [arrow] - Support RunEndEncoded arrays in comparison kernels (eq, lt, etc.) #9620 [arrow]
- variant_get should follow JSONpath semantics #9606
- GenericByteViewArray: support finding total length of all strings #9435 [arrow]
Merged pull requests:
- support length() on Run-end encoding arrays #9838 [arrow] (Rich-T-kid)
- fix(ipc): correct skip_field handling for V4 Union #9829 [arrow] (pchintar)
- fix(ipc): replace wildcard in skip_field with explicit DataType handling #9822 [arrow] (pchintar)
- Prevent buffer builder length overflow in
MutableBuffer::extend_zeros#9820 [arrow] (alamb) - Prevent repeat slice length overflow #9819 [arrow] (alamb)
- Prevent BitChunks length overflow #9818 [arrow] (alamb)
- Prevent Rows row index overflow #9817 [arrow] (alamb)
- Prevent ArrayData validation length overflow #9816 [arrow] (alamb)
- [Json] Remove arrow-data dependency from arrow-json #9812 [arrow] (liamzwbao)
- Replace
BooleanBufferBuilderwithNullBufferBuilderin arrow-json if applicable #9811 [arrow] (liamzwbao) - refactor(ipc): derive Default for CompressionContext #9809 [arrow] (mbutrovich)
- fix(ipc): reader misalignment when skipping ListView / LargeListView columns #9806 [arrow] (pchintar)
- fix(ipc): Avoid panic on malformed compressed buffer prefix #9802 [arrow] (pchintar)
- parquet: fix panic in DeltaByteArrayDecoder on invalid prefix lengths #9797 [parquet] (pchintar)
- feat(parquet): fuse level encoding with counting and histogram updates #9795 [parquet] (HippoBaro)
- Expose ColumnCloseResult on ArrowColumnChunk #9773 [parquet] (leoyvens)
- feat: make FFI structs fields
pub#9772 [arrow] (ashdnazg) - chore: Refine the error message for List to non List cast #9757 [arrow] (comphead)
- refactor(parquet): replace magic
8literals with named constants #9751 [parquet] (HippoBaro) - feat(ipc): add with_skip_validation to StreamDecoder #9749 [arrow] (pantShrey)
- remove panics in unshred variant #9741 (friendlymatthew)
- Add benchmark for ListView interleave #9738 [arrow] (vegarsti)
- arrow-arith: fix 'occured' -> 'occurred' in arity.rs comments #9736 [arrow] (SAY-5)
- Refactor
RleEncoder::flush_bit_packed_runto make flow clearer #9735 [parquet] (etseidl) - Fix RecordBatch::normalize() null bitmap bug and add StructArray::flatten() #9733 [arrow] (sqd)
- Add benchmark for cast from/to decimals #9729 [arrow] (klion26)
- refactor(arrow-avro): use
Decoder::flush_blockin async reader #9726 [arrow] (mzabaluev) - fix: ParquetError when reading corrupt parquet file with truncated data instead of Panic #9725 [parquet] (xuzifu666)
- feat(parquet): add wide-schema writer overhead benchmark #9723 [parquet] (HippoBaro)
- fix: correct accounting in
DictEncoder::estimated_memory_size,Interner::estimated_memory_size#9720 [parquet] (mzabaluev) - arrow-ipc: Write 0 offset buffer for length-0 variable-size arrays #9717 [arrow] (atwam)
- [Json] Support
FixedSizeListin json decoder #9715 [arrow] (liamzwbao) - chore(deps): bump actions/upload-pages-artifact from 4 to 5 #9713 (dependabot[bot])
- Fix clippy warning in fixed_size_binary_array.rs #9712 [arrow] (AdamGS)
- feat: add
has_non_empty_nullshelper function inOffsetBuffer#9711 [arrow] (rluvaton) - chore(deps): bump pytest from 7.2.0 to 9.0.3 in /parquet/pytest #9706 [parquet] (dependabot[bot])
- Fedora license audit #9704 [arrow] (michel-slm)
- [Variant] Take top-level nulls into consideration when extracting perfectly shredded children #9702 (AdamGS)
- feat(parquet): add
push_decoderbenchmark forPushBuffersoverhead #9696 [parquet] (HippoBaro) - Add mutable bitwise operations to
BooleanArrayandNullBuffer::union_many#9692 [arrow] (mbutrovich) - chore(deps): update hashbrown requirement from 0.16.0 to 0.17.0 #9691 [parquet] [arrow] (dependabot[bot])
- chore(deps): bump actions/github-script from 8 to 9 #9690 (dependabot[bot])
- minor: Re-enable CDC bench #9686 [parquet] (etseidl)
- [Variant] Add
VariantArrayBuilder::append_nullsAPI #9685 (sdf-jkl) - feat(parquet): add struct-column writer benchmarks #9679 [parquet] (HippoBaro)
- [Arrow] Add API to check if
Fieldhas a validExtensionType#9677 [parquet] [arrow] (sdf-jkl) - [Variant]
variant_getshould follow JSONPath semantics for Field path element #9676 (sdf-jkl) - ParquetMetaDataPushDecoder API to clear all buffered ranges #9673 [parquet] (nathanb9)
- Fix union cast incorrectness for duplicate field names #9666 [arrow] (friendlymatthew)
- chore: re-export
MAX_INLINE_VIEW_LENfromarrow_data#9665 [arrow] (rluvaton) - No longer allow BIT_PACKED level encoding in Parquet writer #9656 [parquet] (etseidl)
- feat(parquet): add sparse-column writer benchmarks #9654 [parquet] (HippoBaro)
- Support
GenericListViewArray::new_uncheckedand refactorListViewjson decoder #9648 [arrow] (liamzwbao) - [Json] Add json reader benchmarks for ListView #9647 [arrow] (liamzwbao)
- fix(parquet): fix CDC panic on nested ListArrays with null entries #9644 [parquet] (kszucs)
- Add a test for reading nested REE data in json #9634 [arrow] (alamb)
- [Variant] Fix
variant_getto returnList<T>instead ofList<Struct>#9631 (liamzwbao) - ci: use ubuntu-slim runner for lightweight CI jobs #9630 (CuteChuanChuan)
- Add bloom filter folding to automatically size SBBF filters #9628 [parquet] (adriangb)
- Add List and ListView take benchmarks #9626 [arrow] (AdamGS)
- ParquetPushDecoder API to clear all buffered ranges #9624 [parquet] (nathanb9)
- fix: handle missing dictionary batch for null-only columns in IPC reader #9623 [arrow] (joaquinhuigomez)
- Fix
MutableBuffer::clear#9622 [parquet] [arrow] (Rafferty97) - feat[arrow-ord]: suppport REE comparisons #9621 [arrow] (asubiotto)
- chore(deps): update sha2 requirement from 0.10 to 0.11 #9618 [arrow] (dependabot[bot])
- Expose option to set line terminator for CSV writer #9617 [arrow] (svranesevic)
- [Json] Add json reader benchmarks for Map and REE #9616 [arrow] (liamzwbao)
- deps: fix
object_storebreakage for 0.13.2 #9612 (mzabaluev-flarion) - [Variant] Support Binary/LargeBinary children #9610 (AdamGS)
- fix: use writer types in Skipper for resolved named record types #9605 [arrow] (ariel-miculas)
- feat(parquet): derive
PartialEqandEqforCdcOptions#9602 [parquet] (kszucs) - Add
finish_preserve_valuestoArrayBuildertrait #9601 [arrow] (adamreichold) - [Variant] extend shredded null handling for arrays #9599 (sdf-jkl)
- [Variant] Add unshredded
Structfast-path forvariant_get(..., Struct)#9597 (sdf-jkl) - Pre-reserve output capacity in ByteView/ByteArray dictionary decoding #9590 [parquet] (Dandandan)
- [Variant] Align cast logic for variant_get to cast kernel for numeric/bool types #9563 [arrow] (klion26)
- Add support to cast from
UnionArray#9544 [arrow] (friendlymatthew) - Support
ListViewcodec in arrow-json #9503 [arrow] (liamzwbao)
-
58.1.024 Mar 2026Release notes
Open source →Implemented enhancements:
- Reuse compression dict lz4_block #9566
- [Variant] Add
variant_to_arrowStructtype support #9529 - [Variant] Add
unshred_variantsupport forBinaryandLargeBinarytypes #9526 - [Variant] Add
shred_variantsupport forLargeUtf8andLargeBinarytypes #9525 - [Variant]
variant_gettests clean up #9517 - parquet_variant: Support LargeUtf8 typed value in
unshred_variant#9513 - parquet-variant: Support string view typed value in
unshred_variant#9512 - Deprecate ArrowTimestampType::make_value in favor of from_naive_datetime #9490 [arrow]
- Followup for support ['fieldName'] in VariantPath #9478
- Speedup DELTA_BINARY_PACKED decoding when bitwidth is 0 #9476 [parquet]
- Support CSV files encoded with charsets other than UTF-8 #9465 [arrow]
- Expose Avro writer schema when building the reader #9460 [arrow]
- Python: avoid importing pyarrow classes ever time #9438
- Add
append_nullstoMapBuilder#9431 [arrow] - Add
append_non_nullstoStructBuilder#9429 [arrow] - Add
append_value_nto GenericByteBuilder #9425 [arrow] - Optimize
from_bitwise_binary_op#9378 [arrow] - Configurable Arrow representation of UTC timestamps for Avro reader #9279 [arrow]
Fixed bugs:
- MutableArrayData::extend does not copy child values for ListView arrays #9561 [arrow]
- ListView interleave bug #9559 [arrow]
- Flight encoding panics with "no dict id for field" with nested dict arrays #9555 [arrow] [arrow-flight]
- "DeltaBitPackDecoder only supports Int32Type and Int64Type" but unsigned types are supported too #9551 [parquet]
- Potential overflow when calling
util::bit_mask::set_bits(soundness issue) #9543 [arrow] - handle Null type in try_merge for Struct, List, LargeList, and Union #9523 [arrow]
- Invalid offset in sparse column chunk data for multiple predicates #9516 [parquet]
- debug_assert_eq! in BatchCoalescer panics in debug mode when batch_size < 4 #9506 [arrow]
- Parquet Statistics::null_count_opt wrongly returns Some(0) when stats are missing #9451 [parquet]
- Error "Not all children array length are the same!" when decoding rows spanning across page boundaries in parquet file when using
RowSelection#9370 [parquet] - Avro schema resolution not properly supported for complex types #9336 [arrow]
Documentation updates:
Performance improvements:
- Introduce
NullBuffer::try_from_unslicedto simplify array construction #9385 [parquet] [arrow] - perf: Coalesce page fetches when RowSelection selects all rows #9578 [parquet] (Dandandan)
- Use chunks_exact for has_true/has_false to enable compiler unrolling #9570 [arrow] (adriangb)
- pyarrow: Cache the imported classes to avoid importing them each time #9439 (Tpt)
Closed issues:
- Duplicate macro definition:
partially_shredded_variant_array_gen#9492 - Enable
LargeList/ListView/LargeListViewforVariantArray::try_new#9455 - Support variables/expressions in record_batch! macro #9245 [arrow]
Merged pull requests:
- [Variant] Add unshred_variant support for Binary and LargeBinary types #9576 (kunalsinghdadhwal)
- [Variant] Add
variant_to_arrowStructtype support #9572 (sdf-jkl) - Make Sbbf Constructers Public #9569 [parquet] (cetra3)
- fix: Used
checked_addfor bounds checks to avoid UB #9568 [arrow] (etseidl) - Add mutable operations to BooleanBuffer (Bit*Assign) #9567 [arrow] (Dandandan)
- chore(deps): update lz4_flex requirement from 0.12 to 0.13 #9565 [parquet] [arrow] (dependabot[bot])
- arrow-select: fix MutableArrayData interleave for ListView #9560 [arrow] (asubiotto)
- Move
ValueIterinto own module, and add publicrecord_countfunction #9557 [arrow] (Rafferty97) - arrow-flight: generate dict_ids for dicts nested inside complex types #9556 [arrow] [arrow-flight] (asubiotto)
- add
shred_variantsupport forLargeUtf8andLargeBinary#9554 (sdf-jkl) - [minor] Download clickbench file when missing #9553 [parquet] (Dandandan)
- DeltaBitPackEncoderConversion: Fix panic message on invalid type #9552 [parquet] (progval)
- Replace interleave overflow panic with error #9549 [arrow] (xudong963)
- feat(arrow-avro):
HeaderInfoto expose OCF header #9548 [arrow] (mzabaluev) - chore: Protect
mainbranch with required reviews #9547 (comphead) - Add benchmark for
infer_json_schema#9546 [arrow] (Rafferty97) - chore(deps): bump black from 24.3.0 to 26.3.1 in /parquet/pytest #9545 [parquet] (dependabot[bot])
- Unroll interleave -25-30% #9542 [arrow] (Dandandan)
- Optimize
take_fixed_size_binaryFor Predefined Value Lengths #9535 [arrow] (tobixdev) - feat: expose arrow schema on async avro reader #9534 [arrow] (mzabaluev)
- Make with_file_decryption_properties pub instead of pub(crate) #9532 [parquet] (Dandandan)
- fix: handle Null type in try_merge for Struct, List, LargeList, and Union #9524 [arrow] (zhuqi-lucas)
- chore: extend record_batch macro to support variables and expressions #9522 [arrow] (buraksenn)
- [Variant] clean up
variant_gettests #9518 (sdf-jkl) - support large string for unshred variant #9515 (friendlymatthew)
- support string view unshred variant #9514 (friendlymatthew)
- Add has_true() and has_false() to BooleanArray #9511 [arrow] (adriangb)
- Fix Invalid offset in sparse column chunk data error for multiple predicates #9509 [parquet] (cetra3)
- fix: remove incorrect debug assertion in BatchCoalescer #9508 [arrow] (Tim-53)
- [Json] Add benchmarks for list json reader #9507 [arrow] (liamzwbao)
- fix: first next_back() on new RowsIter panics #9505 [arrow] (rluvaton)
- Add some benchmarks for decoding delta encoded Parquet #9500 [parquet] (etseidl)
- chore: remove duplicate macro
partially_shredded_variant_array_gen#9498 (codephage2020) - Deprecate ArrowTimestampType::make_value in favor of from_naive_datetime #9491 [arrow] (codephage2020)
- fix: Do not assume missing nullcount stat means zero nullcount #9481 [parquet] (scovich)
- [Variant] Enahcne bracket access for VariantPath #9479 (klion26)
- Optimize delta binary decoder in the case where bitwidth=0 #9477 [parquet] (etseidl)
- Add PrimitiveRunBuilder::with_data_type() to customize the values' DataType #9473 [arrow] (brunal)
- Convert
prettyprinttests inarrow-casttoinstainline snapshots #9472 [parquet] [arrow] (grtlr) - Update strum_macros requirement from 0.27 to 0.28 #9471 [arrow] (dependabot[bot])
- docs(parquet): Fix broken links in README #9467 [parquet] (SYaoJun)
- Add list-like types support to VariantArray::try_new #9457 (sdf-jkl)
- Simplify downcast_...!() macro definitions #9454 [arrow] (brunal)
- feat(parquet): add content defined chunking for arrow writer #9450 [parquet] (kszucs)
- refactor: simplify iterator using cloned().map(Some) #9449 [parquet] (SYaoJun)
- feat: Optimize from_bitwise_binary_op with 64-bit alignment #9441 [arrow] (kunalsinghdadhwal)
- docs: fix markdown link syntax in README #9440 (SYaoJun)
- Move
ListLikeArrayto arrow-array to be shared with json writer and parquet unshredding #9437 [arrow] (liamzwbao) - Add
claimmethod to recordbatch for memory accounting #9433 [arrow] (cetra3) - Add
append_nullstoMapBuilder#9432 [arrow] (Fokko) - Add
append_non_nullstoStructBuilder#9430 [arrow] (Fokko) - Add
append_value_nto GenericByteBuilder #9426 [arrow] (Fokko) - refactor: simplify dynamic state for Avro record projection #9419 [arrow] (mzabaluev)
- Add
NullBuffer::from_unsliced_bufferhelper and refactor call sites #9411 [parquet] [arrow] (Eyad3skr) - Implement min, max, sum for run-end-encoded arrays. #9409 [arrow] (brunal)
- feat: add
RunArray::new_uncheckedandRunArray::into_parts#9376 [arrow] (rluvaton) - Fix skip_records over-counting when partial record precedes num_rows page skip #9374 [parquet] (jonded94)
- fix: resolution of complex type variants in Avro unions #9328 [arrow] (mzabaluev)
- feat(arrow-avro): Configurable Arrow timezone ID for Avro timestamps #9280 [arrow] (mzabaluev)
-
58.0.023 Feb 2026Release notes
Open source →Breaking changes:
- Remove support for List types in bit_length kernel #9350 [arrow] (codephage2020)
- Optimize
from_bitwise_unary_op#9297 [arrow] (Dandandan) - Mark
BufferBuilder::new_from_bufferas unsafe #9292 [arrow] (Jefffrey) - [Variant] Support
['fieldName']in VariantPath parser #9276 (klion26) - Remove parquet arrow_cast dependency #9077 [parquet] (tustvold)
- feat: change default behavior for Parquet
PageEncodingStatsto bitmask #9051 [parquet] (WaterWhisperer) - [arrow] Minimize allocation in GenericViewArray::slice() #9016 [arrow] (maxburke)
Implemented enhancements:
- Avoid allocating a
VecinStructBuilder#9427 - Zstd context reuse #9401
- Optimize
from_bitwise_unary_op#9364 - Support
RunEndEncodedin ord comparator #9360 - Support
RunEndEncodedarrays inarrow-json#9359 - Support
BinaryViewinbit_lengthkernel #9351 - Remove support for
Listtypes inbit_lengthkernel #9349 - Support roundtrip
ListViewin parquet arrow writer #9344 - Support
ListViewinlengthkernel #9343 - Support
ListViewin sort kernel #9341 - Add some way to create a Timestamp from a
DateTime#9337 - Introduce
DataType::is_listandDataType::IsBinary#9326 - Performance of creating all null dictionary array can be improved #9321
- [arrow-avro] Add missing Arrow DataType support with
avro_custom_typesround-trip + non-custom fallbacks #9290
Fixed bugs:
- ArrowArrayStreamReader errors on zero-column record batches #9394
- Regression on main (58): Parquet argument error: Parquet error: Required field type_ is missing #9315 [parquet]
Documentation updates:
- Improve safety documentation of the
Arraytrait #9314 [arrow] (alamb) - Improve docs and add build() method to
{Null,Boolean,}BufferBuilder#9155 [arrow] (alamb) - Improve
ArrowReaderBuilder::with_row_filterdocumentation #9153 [parquet] (alamb) - docs: Improve main README.md and highlight community #9119 (alamb)
- Docs: Add additional documentation and example for
make_array#9112 [arrow] (alamb) - doc: fix link on FixedSizeListArray doc #9033 [arrow] (Jefffrey)
Performance improvements:
- Replace
ArrayDatawith direct Array construction #9338 [arrow] (liamzwbao) - Remove some
unsafeand allocations when creating PrimitiveArrays from Vec andfrom_trusted_len_iter#9299 [arrow] (alamb) - parquet: rle skip decode loop when batch contains all max levels (aka no nulls) #9258 [parquet] (lyang24)
- Improve parquet BinaryView / StringView decoder performance (up to -35%) #9236 [parquet] (Dandandan)
- Avoid a clone when creating
BooleanArrayfrom ArrayData #9159 [arrow] (alamb) - Avoid overallocating arrays in coalesce primitives / views #9132 [arrow] (Dandandan)
- perf: Avoid ArrayData allocation in PrimitiveArray::reinterpret_cast #9129 [arrow] (alamb)
- [Parquet] perf: Create StructArrays directly rather than via
ArrayData(1% improvement) #9120 [parquet] [arrow] (alamb) - Avoid clones in
make_arrayforStructArrayandGenericByteViewArray#9114 [arrow] (alamb) - perf: optimize hex decoding in json (1.8x faster in binary-heavy) #9091 [arrow] (Weijun-H)
- Speed up binary kernels (30% faster
andandor), addBooleanBuffer::from_bitwise_binary_op#9090 [arrow] (alamb) - perf: improve field indexing in JSON StructArrayDecoder (1.7x speed up) #9086 [arrow] (Weijun-H)
- bench: added to row_format benchmark conversion of 53 non-nested columns #9081 [arrow] (rluvaton)
- perf: improve calculating length performance for view byte array in row conversion #9080 [arrow] (rluvaton)
- perf: improve calculating length performance for nested arrays in row conversion #9079 [arrow] (rluvaton)
- perf: improve calculating length performance for
GenericByteArrayin row conversion #9078 [arrow] (rluvaton)
Closed issues:
- BatchCoalescer::push_batch panics on schema mismatch instead of returning error #9389
- Release arrow-rs / parquet Minor version
57.3.0(January 2026) #9240 - [Variant] support
..and['fieldName']syntax in the VariantPath parser #9050 - Support Float16 for create_random_array #9028
Merged pull requests:
- Avoid allocating a
VecinStructBuilder#9428 [arrow] (Fokko) - fix: fixed trait functions clash get_date_time_part_extract_fn (#8221) #9424 [arrow] (esavier)
- [Minor] Use per-predicate projection masks in arrow_reader_clickbench benchmark #9413 [parquet] (Dandandan)
- Fix
ArrowArrayStreamReaderfor 0-columns record batch streams #9405 [arrow] (jonded94) - Use zstd::bulk API in IPC and Parquet with context reuse for compression and decompression #9400 [parquet] [arrow] (Dandandan)
- Reproduce the issue of #9370 in a minimal, end-to-end way #9399 [parquet] (jonded94)
- perf: optimize skipper for varint values used when projecting Avro record types #9397 [arrow] (mzabaluev)
- fix: return error instead of panic on schema mismatch in BatchCoalescer::push_batch #9390 [arrow] (bvolpato-dd)
- Minor: Add additional test coverage for WriterProperties::{max_row_group_row_count,max_row_group_size} #9387 [parquet] (alamb)
- Moving invalid_utf8 tests into a separate mod #9384 [parquet] (sdf-jkl)
- Update sysinfo requirement from 0.37.1 to 0.38.1 #9383 [parquet] (dependabot[bot])
- feat: support RunEndEncoded arrays in arrow-json reader and writer #9379 [arrow] (Abhisheklearn12)
- Remove lint issues in parquet-related code. #9375 [parquet] (brunal)
- Add RunEndEncoded array comparator #9368 [arrow] (codephage2020)
- feat: support BinaryView in bit_length kernel #9363 [arrow] (Abhisheklearn12)
- Add regression tests for Parquet large binary offset overflow #9361 [parquet] (vigneshsiva11)
- feat: add max_row_group_bytes option to WriterProperties #9357 [parquet] (yonipeleg33)
- doc: remove disclaimer about
ListViewnot being fully supported #9356 [arrow] (Jefffrey) - Move row_filter async tests from parquet async reader #9355 [parquet] (sdf-jkl)
- [Parquet] Allow setting page size per column #9353 [parquet] (XiangpengHao)
- feat: Support roundtrip ListView in parquet arrow writer #9352 [parquet] (codephage2020)
- feat: add ListView and LargeListView support to arrow-ord #9347 [arrow] (codephage2020)
- Support ListView in length kernel #9346 [arrow] (vegarsti)
- feat: Add from_datetime method to Timestamp types #9345 [arrow] (codephage2020)
- [main] Update version to 57.3.0, add changelog #9334 (alamb)
- build(deps): update pyo3 requirement from 0.27.1 to 0.28.0 #9331 (dependabot[bot])
- Add
DataType::is_listandDataType::is_binary#9327 [arrow] (AdamGS) - Fix string array equality when the values buffer is the same and only the offsets to access it differ #9325 [arrow] (jhorstmann)
- perf: skip validation of dictionary keys if all null #9322 [arrow] (albertlockett)
- parquet: use rwlock instead of mutex in predicate cache #9319 [parquet] (lyang24)
- nit: remove usused code #9318 [parquet] (lyang24)
- Remove unnecessary Arc<ArrayRef> #9316 [parquet] [arrow] (scovich)
- Optimize data page statistics conversion (up to 4x) #9303 [parquet] [arrow] (Dandandan)
- [regression] Error with adaptive predicate pushdown: "Invalid offset in sparse column chunk data: 754, no matching page found." #9301 [parquet] (sdf-jkl)
- Improve
PrimitiveArray::from_iterperf #9294 [arrow] (Dandandan) - Add additional Arrow type support #9291 [arrow] (jecsand838)
- fix: ensure
BufferBuilder::truncatedoesn't overset length #9288 [arrow] (Jefffrey) - Add benchmark for row group index reader perf #9285 [parquet] (friendlymatthew)
- fix union array row converter to handle non-sequential type ids #9283 [arrow] (friendlymatthew)
- parquet: reduce clone in delta byte array decoder #9282 [parquet] (lyang24)
- fix: fix [[NULL]] array doesn't roundtrip in arrow-row bug #9275 [arrow] (lichuang)
- Enhance list casting, adding more cases for list views #9274 [arrow] (Jefffrey)
- [Variant] Add path index access tests for list #9273 (liamzwbao)
- Factor out json reader's static make_decoder args to a struct #9271 [arrow] (scovich)
- make_decoder accepts borrowed DataType instead of owned #9270 [arrow] (scovich)
- Implement a more generic from_nested_iter method for list arrays #9268 [arrow] (jhorstmann)
- Move extension type construction logic out of Field #9266 [parquet] [arrow] (scovich)
- fix: support casting string to f16 #9262 [arrow] (Jefffrey)
- Add additional coverage for StringViewArray comparisons #9257 [arrow] (alamb)
- Upgrade to object store 0.13.1 #9256 [parquet] (Dandandan)
- [Parquet] test adaptive predicate pushdown with skipped page #9251 [parquet] (sdf-jkl)
- Speed up string view comparison (up to 3x) #9250 [arrow] (Dandandan)
- Add bench for LocalFileSystem #9248 [parquet] (Dandandan)
- [Parquet] Add test for reading/writing long UTF8 StringViews #9246 [parquet] (alamb)
- [Parquet] test adaptive predicate pushdown with skipped page #9243 [parquet] (erratic-pattern)
- Add tests and fixes for schema resolution bug #9237 [arrow] (jecsand838)
- Revert "Seal Array trait (#9092)", mark
Arrayasunsafe#9234 [arrow] (gabotechs) - Speedup filter (up to ~1.5x)
FilterBuilder::Optimize/BitIndexIterator/iter_set_bits_rev#9229 [parquet] [arrow] (Dandandan) - optimize
RowGroupIndexReaderfor single row group reads #9226 [parquet] (friendlymatthew) - test: improve arrow-row fuzz tests #9222 [arrow] (rluvaton)
- arrow-cast: support packing to Dictionary(_, Utf8View/BinaryView) #9220 [arrow] (ethan-tyler)
- Add additional test coverage for
BatchCoalescerpush_batch_with_filter #9218 [arrow] (alamb) - [Parquet] Optimize appending max level comparison in DefinitionLevelDecoder #9217 [parquet] (jhorstmann)
- Remove dead code to fix clippy failure on main #9215 [arrow] (alamb)
- [Parquet] perf: reuse seeked File clone in ChunkReader::get_read() #9214 [parquet] (fvaleye)
- fix: [9018]Fixed RunArray slice offsets(row, cast, eq) #9213 [arrow] (manishkr)
- Add benchmarks for reading struct arrays from parquet #9210 [parquet] (jhorstmann)
- Support casting negative scale decimals to numeric #9207 [arrow] (Chiicake)
- Deprecate
ArrowReaderOptions::with_page_indexand update API #9199 [parquet] (friendlymatthew) - arrow-ipc: add reset method to DictionaryTracker #9196 [arrow] (albertlockett)
- Avoid a clone when creating
ListArrayfrom ArrayData #9194 [arrow] (alamb) - Avoid a clone when creating
ListViewArrayfrom ArrayData #9193 [arrow] (alamb) - Avoid a clone when creating
MapArrayfrom ArrayData #9192 [arrow] (alamb) - Avoid a clone when creating
NullArrayfrom ArrayData #9191 [arrow] (alamb) - Avoid a clone when creating
PrimitiveArrayfrom ArrayData #9190 [arrow] (alamb) - Avoid a clone when creating
RunEndArrayfrom ArrayData #9189 [arrow] (alamb) - Avoid a clone when creating
UnionArrayfrom ArrayData #9188 [arrow] (alamb) - Avoid a clone when creating
FixedSizeListArrayfrom ArrayData #9187 [arrow] (alamb) - Avoid a clone when creating
FixedSizeBinaryArrayfrom ArrayData #9186 [arrow] (alamb) - Avoid a clone when creating
DictionaryArrayfrom ArrayData #9185 [arrow] (alamb) - fix: take_run return empty array instead of panic. #9182 [arrow] (thorfour)
- lint: remove unused function (fix clippy #9178 [arrow] (rluvaton)
- add
#[inline]toBitIteratornextfunction #9177 [arrow] (rluvaton) - Add ListView support to
arrow-rowandarrow-ord#9176 [arrow] (brancz) - arrow-cast: Add display formatter for ListView #9175 [arrow] (brancz)
- Add BinaryFormatSupport and Row Encoder to
arrow-avroWriter #9171 [arrow] (jecsand838) - docs(parquet): move async parquet example into ArrowReaderBuilder docs #9167 [parquet] (vigneshsiva11)
- feat(array): add
RecordBatchStreamtrait #9166 [arrow] (lidavidm) - refactor: streamline date64 tests #9165 [arrow] (cht42)
- docs: update examples in ArrowReaderOptions to use in-memory buffers #9163 [parquet] (AndreaBozzo)
- Add Avro Reader projection API #9162 [arrow] (jecsand838)
- Avoid a clone when creating StringArray/BinaryArray from ArrayData #9160 [arrow] (alamb)
- fix missing utf8 check for conversion from BinaryViewArray to StringViewArray #9158 [arrow] (alamb)
- Minor: try and avoid an allocation creating
GenericByteViewArrayfromArrayData#9156 [arrow] (alamb) - Add find_nth_set_bit_position #9151 [arrow] (Dandandan)
- feat: add null comparison handling in make_comparator #9150 [arrow] (Weijun-H)
- Uncomment part of test_utf8_single_column_reader_test #9148 [parquet] (sdf-jkl)
- arrow-ipc: Add tests for nested dicts for Map and Union arrays #9146 [arrow] (brancz)
- Update ASF copyright year in NOTICE #9145 (mohit7705)
- Avoid panic on Date32 overflow #9144 [arrow] (cht42)
- feat: add
reservetoRows#9142 [arrow] (rluvaton) - docs(variant): fix VariantObject::get documentation to reflect Option return type #9139 (mohit7705)
- Add
BooleanBufferBuilder::extend_trusted_len#9137 [arrow] (Dandandan) - fix: support cast from
Nullto list view/run encoded/union types #9134 [arrow] (Jefffrey) - Fix clippy #9130 (alamb)
- Fix IPC roundtripping dicts nested in ListViews #9126 [arrow] (brancz)
- Update readme for geospatial crate #9124 (paleolimbot)
- [Parquet] perf: Create
PrimitiveArrays directly rather than viaArrayData#9122 [parquet] (alamb) - [Parquet] perf: Create Utf8/BinaryViewArray directly rather than via
ArrayData#9121 [parquet] (alamb) - [parquet] Add row group index virtual column #9117 [parquet] (friendlymatthew)
- docs(parquet): add example for preserving dictionary encoding #9116 [parquet] (AndreaBozzo)
- doc: add example of RowFilter usage #9115 [parquet] (sonhmai)
- docs: Update release schedule in README.md #9111 (alamb)
- feat: add benchmarks for json parser #9107 [arrow] (Weijun-H)
- chore: switch test from
bincodeto maintainedpostcardcrate (RUSTSEC-2025-0141 ) #9104 [arrow] (alamb) - Add nullif_kernel benchmark #9089 [arrow] (alamb)
- [Variant] Support Shredded Lists/Array in
variant_get#9049 (liamzwbao) - fix:[9018]Fixed RunArray slice offsets #9036 [arrow] (manishkr)
- Support Float16 for create_random_array #9029 [arrow] (niebayes)
- fix: display
0 secsfor empty DayTime/MonthDayNano intervals #9023 [arrow] (Jefffrey) - Add options to skip decoding
StatisticsandSizeStatisticsin Parquet metadata #9008 [parquet] (etseidl)
-
57.3.111 May 2026Nothing published for this version
-
57.3.006 Feb 2026Release notes
Open source →Breaking changes:
- Revert "Seal Array trait", mark
Arrayasunsafe#9313 (alamb, gabotechs) - Mark
BufferBuilder::new_from_bufferas unsafe #9312 (alamb, Jefffrey)
Fixed bugs:
- Fix string array equality when the values buffer is the same and only the offsets to access it differ #9330 (alamb, jhorstmann)
- Ensure
BufferBuilder::truncatedoesn't overset length #9311 (alamb, Jefffrey) - [parquet] Provide only encrypted column stats in plaintext footer #9310 (alamb, rok, adamreeve)
- [regression] Error with adaptive predicate pushdown: "Invalid offset …" #9309 (alamb, erratic-pattern, sdf-jkl)
- Revert "Seal Array trait", mark
-
57.2.011 Jan 2026Release notes
Open source →Breaking changes:
- Seal Array trait #9092 [arrow] (tustvold)
- [Variant] Unify the CastOptions usage in parquet-variant-compute #8984 (klion26)
Implemented enhancements:
- [parquet] further relax
LevelInfoBuilder::types_compatibleforArrowWriter#9098 - Update arrow-row documentation with Union encoding #9084
- Add code examples for min and max compute functions #9055
- Add
append_nto bytes view builder API #9034 [arrow] - Move
RunArray::get_physical_indicestoRunEndBuffer#9025 [arrow] - Allow quote style in csv writer #9003 [arrow]
- IPC support for ListView #9002 [arrow]
- Implement
BinaryArrayTypefor&FixedSizeBinaryArrays #8992 [arrow] - arrow-buffer: implement num-traits for i256 #8976 [arrow]
- Support for
Arc<str>inParquetRecordWriterderive macro #8972 - [arrow-avro] suggest switching from xz to liblzma #8970 [arrow]
- arrow-buffer: add i256::trailing_zeros #8968 [arrow]
- arrow-buffer: make i256::leading_zeros public #8965 [arrow]
- Add spark like
ignoreLeadingWhiteSpaceandignoreTrailingWhiteSpaceoptions to the csv writer #8961 [arrow] - Add round trip benchmark for Parquet writer/reader #8955 [parquet]
- Support performant
interleavefor List/LargeList #8952 [arrow] - [Variant] Support array access when parsing
VariantPath#8946 - Some panic!s could be represented as unimplemented!s #8932 [arrow]
- [Variant] easier way to construct a shredded schema #8922
- Support
DataType::ListViewandDataType::LargeListViewinArrayData::new_null#8908 [arrow] - Add
GenericListViewArray::from_iter_primitive#8906 [arrow] - [Variant] Unify the cast option usage in ParquentVariant #8873
- Blog post about efficient filter representation in Parquet filter pushdown #8843 [parquet]
- Add comparison support for Union arrays in the
cmpkernel #8837 [arrow] - [Variant] Support array shredding into
List/LargeList/ListView/LargeListView#8830 - Support
Uniondata types for row format #8828 [arrow] - FFI support for ListView #8819 [arrow]
- [Variant] Support more Arrow Datatypes from Variant primitive types #8805
FixedSizeBinaryBuildersupportsappend_array#8750 [arrow]- Implement special case
zipwith scalar for Utf8View #8724 [arrow] - [geometry] Wire up arrow reader/writer for
GEOMETRYandGEOGRAPHY#8717 [parquet]
Fixed bugs:
- Soundness Bug in
try_binarywhenArrayis implemented incorrectly in external crate #9106 - casting
Dict(_, LargeUtf8)toUtf8View(StringViewArray) panics #9101 - wrong results for null count of
nullifkernel #9085 [parquet] [arrow] - Empty first line in some code examples #9063
- GenericByteViewArray::slice is not zero-copy but ought to be #9014
- Regression in struct casting in 57.2.0 (not yet released) #9005 [arrow]
- Fix panic when decoding multiple Union columns in RowConverter #8999 [arrow]
take_fixed_size_binaryDoes Not Consider NULL Indices #8947 [arrow]- [arrow-avro] RecordEncoder Bugs #8934 [arrow]
FixedSizeBinaryArray::try_new(...)Panics with Item Length of Zero #8926 [arrow]cargo test -p arrow-castfails on main #8910 [arrow]GenericListViewArray::new_nullignoreslenand returns an empty array #8904 [arrow]FixedSizeBinaryArray::new_nullDoes Not Properly Set the Length of the Values Buffer #8900 [arrow]- Struct casting requires same order of fields #8870 [arrow]
- Cannot cast string dictionary to binary view #8841 [arrow]
Documentation updates:
- Add Union encoding documentation #9102 [arrow] (EduardAkhmetshin)
- docs: fix misleading reserve documentation #9076 (WaterWhisperer)
- Fix headers and empty lines in code examples #9064 (EduardAkhmetshin)
- Add examples for min and max functions #9062 (EduardAkhmetshin)
- Improve arrow-buffer documentation #9020 [arrow] (alamb)
- Move examples in arrow-csv to docstrings, polish up docs #9001 [arrow] (alamb)
- Add example of parsing field names as VariantPath #8945 (alamb)
- Improve documentation for `prep_null_mask_flter #8722 [arrow] (alamb)
Performance improvements:
- [parquet] Avoid a clone while resolving the read strategy #9056 [parquet] (alamb)
- perf: improve performance of encoding
GenericByteArrayby 8% #9054 [arrow] (rluvaton) - Speed up unary
notkernel by 50%, addBooleanBuffer::from_bitwise_unary#8996 [arrow] (alamb) - arrow-select: improve dictionary interleave fallback performance #8978 [arrow] (asubiotto)
- Add special implementation for zip for Utf8View/BinaryView scalars #8963 [arrow] (mkleen)
- arrow-select: implement specialized interleave_list #8953 [arrow] (asubiotto)
Closed issues:
Merged pull requests:
- Add
DataType::is_decimal#9100 (AdamGS) - feat(parquet): relax type compatility check in parquet ArrowWriter #9099 [parquet] (gruuya)
- [Variant] Move
ArrayVariantToArrowRowBuildertovariant_to_arrow#9094 (liamzwbao) - chore: increase row count and batch size for more deterministic tests #9088 [arrow] (Weijun-H)
- Fix
nullifkernel #9087 [arrow] (alamb) - Add
FlightInfo::with_endpointsmethod #9075 [arrow] [arrow-flight] (lewiszlw) - chore: run validation when debug assertion enabled and not only for test #9073 [arrow] (rluvaton)
- Minor: make it clear cache array reader is not cloning arrays #9057 [parquet] (alamb)
- Minor: avoid clone in RunArray row decoding via buffer stealing #9052 [arrow] (lyang24)
- Minor: avoid some clones when reading parquet #9048 [parquet] (alamb)
- fix: don't generate nulls for
Decimal128andDecimal256when field is non-nullable and have non-zeronull_density#9046 [arrow] (rluvaton) - fix:
Rowssizeshould usecapacityand notlen#9044 [arrow] (rluvaton) - fix: integration / Archery test With other arrows container ran out of space #9043 (lyang24)
- feat: add new
try_append_value_n()function toGenericByteViewBuilder#9040 [arrow] (lyang24) - Rename fields in BooleanBuffer for clarity #9039 [arrow] (alamb)
- Allocate buffers before work in
boolean_kernelsbenchmark #9035 [arrow] (alamb) - Move RunArray::get_physical_indices to RunEndBuffer #9027 [arrow] (lyang24)
- Improve
RunArraydocumentation #9019 [arrow] (Jefffrey) - Add BooleanArray tests for null and slice behavior #9013 [arrow] (UtkarshSahay123)
- feat: support array indices in VariantPath dot notation #9012 (foskey51)
- arrow-cast: Bring back in-order field casting for
StructArray#9007 [arrow] (brancz) - arrow-ipc: Add ListView support #9006 [arrow] (brancz)
- Add quote style to csv writer #9004 [arrow] (xanderbailey)
- Fix row slice bug in Union column decoding with many columns #9000 [arrow] (friendlymatthew)
- build(deps): bump actions/download-artifact from 6 to 7 #8995 (dependabot[bot])
- minor: Add comment blocks to PR template #8994 (Jefffrey)
- Implement
BinaryArrayTypefor&FixedSizeBinaryArrays #8993 [arrow] (Jefffrey) - feat: impl BatchCoalescer::push_batch_with_indices #8991 [arrow] (ClSlaid)
- [Arrow]Configure max deduplication length for
StringView#8990 [arrow] (lichuang) - feat: implement append_array for FixedSizeBinaryBuilder #8989 [arrow] (ClSlaid)
- Add benchmarks for Utf8View scalars for zip #8988 [arrow] (mkleen)
- build(deps): bump actions/cache from 4 to 5 #8986 (dependabot[bot])
- Take fsb null indices #8981 [arrow] (alamb)
- Add List to
interleave_kernelsbenchmark #8980 [arrow] (alamb) - Fix ipc errors for
LargeListcontaining slicedStringViews#8979 [arrow] (fabianmurariu) - arrow-buffer: implement num-traits numeric operations #8977 [arrow] (theirix)
- Update
xzcrate dependency to useliblzmain arrow-avro #8975 [arrow] (jecsand838) - arrow-data: avoid allocating in get_last_run_end #8974 [arrow] (asubiotto)
- Support for
Arc<str>inParquetRecordWriterderive macro #8973 (heilhead) - feat: support casting
Time32toInt64#8971 [arrow] (tshauck) - arrow-buffer: add i256::trailing_zeros #8969 [arrow] (theirix)
- Perf: Vectorize check_bounds(2x speedup) #8966 [arrow] (gstvg)
- arrow-buffer: make i256::leading_zeros public and tested #8964 [arrow] (theirix)
- Add ignore leading and trailing white space to csv parser #8960 [arrow] (xanderbailey)
- Access
UnionFieldselements by index #8959 [arrow] (friendlymatthew) - Add Parquet roundtrip benchmarks #8956 [parquet] (etseidl)
- [Variant] Add variant to arrow for Date64/Timestamp(Second/Millisecond)/Time32/Time64 #8950 (klion26)
- Let
ArrowArrayStreamReaderhandle schema with attached metadata + do schema checking #8944 [arrow] (jonded94) - Adds ExtensionType for Parquet geospatial WKB arrays #8943 [parquet] (BlakeOrth)
- Add builder to help create Schemas for shredding (
ShreddedSchemaBuilder) #8940 (XiangpengHao) - build(deps): update criterion requirement from 0.7.0 to 0.8.0 #8939 (dependabot[bot])
- fix: Resolve Avro RecordEncoder bugs related to nullable Struct fields and Union type ids #8935 [arrow] (jecsand838)
- Some panic!s could more semantically be unimplemented! #8933 [arrow] (abacef)
- fix: ipc decode panic with invalid data #8931 [arrow] (leiysky)
- Allow creating zero-sized FixedSizeBinary arrays #8927 [arrow] (tobixdev)
- Update
test_variant_get_error_when_cast_failure...tests to uses a validVariantArray#8921 (alamb) - Make flight sql client generic #8915 [arrow] [arrow-flight] (lewiszlw)
- [minor] Name Magic Number "8" in
FixedSizeBinaryArray::new_null#8914 [arrow] (tobixdev) - fix: cast Binary/String dictionary to view #8912 [arrow] (Jefffrey)
- [8910]Fixed doc test with feature prettyprint #8911 [arrow] (manishkr)
- feat:
ArrayData::new_nullforListView/LargeListView#8909 [arrow] (dqkqd) - fead: add
GenericListViewArray::from_iter_primitive#8907 [arrow] (dqkqd) - fix:
GenericListViewArray::new_nullreturns empty array #8905 [arrow] (dqkqd) - Allocate a zeroed buffer for FixedSizeBinaryArray::null #8901 [arrow] (tobixdev)
- build(deps): bump actions/checkout from 5 to 6 #8899 (dependabot[bot])
- Add getters to
UnionFields#8895 [arrow] (friendlymatthew) - Add validated constructors for UnionFields #8891 [arrow] [arrow-flight] (friendlymatthew)
- Add bit width check #8888 [parquet] (rambleraptor)
- [Variant] Improve
variant_getperformance on a perfect shredding #8887 (XiangpengHao) - Add UnionArray::fields #8884 [arrow] (friendlymatthew)
- Struct casting field order #8871 [arrow] (brancz)
- Add support for
Uniontypes inRowConverter#8839 [arrow] (friendlymatthew) - Add comparison support for Union arrays #8838 [arrow] (friendlymatthew)
- [Variant] Support array shredding into
List/LargeList/ListView/LargeListView#8831 (liamzwbao) - Add support for using ListView arrays and types through FFI #8822 [arrow] (AdamGS)
- Add ability to skip or transform page encoding statistics in Parquet metadata #8797 [parquet] (etseidl)
- Implement a
Vec<RecordBatch>wrapper forpyarrow.Tableconvenience #8790 (jonded94) - Make Parquet SBBF serialize/deserialize helpers public for external reuse #8762 [parquet] (RoseZhang123)
- Add cast support for (Large)ListView <-> (Large)List #8735 [arrow] (vegarsti)
-
57.1.024 Nov 2025Release notes
Open source →Implemented enhancements:
- Eliminate bound checks in filter kernels #8865 [arrow]
- Respect page index policy option for ParquetObjectReader when it's not skip #8856 [parquet]
- Speed up collect_bool and remove
unsafe#8848 [arrow] - Error reading parquet FileMetaData with empty lists encoded as element-type=0 #8826 [parquet]
- ValueStatistics methods can't be used from generic context in external crate #8823 [parquet]
- Custom Pretty-Printing Implementation for Column when Formatting Record Batches #8821 [arrow]
- Parquet-concat: supports bloom filter and page index #8804 [parquet]
- [Parquet] virtual row number support #7299
- [Variant] Enforce shredded-type validation in
shred_variant#8795 [arrow] - Simplify decision logic to call
FilterBuilder::optimizeor not #8781 [arrow] - [Variant] Add variant to arrow for DataType::{Binary, LargeBinary, BinaryView} #8767 [arrow]
- Provide algorithm that allows zipping arrays whose values are not prealigned #8752 [arrow]
- [Parquet] ParquetMetadataReader decodes too much metadata under point-get scenerio #8751 [parquet]
arrow-jsonsupports encoding binary arrays, but not decoding #8736 [arrow]- Allow
FilterPredicateinstances to be reused for RecordBatches #8692 [arrow] - ArrowJsonBatch::from_batch is incomplete #8684 [arrow]
- parquet-layout: More info about layout including footer size, page index, bloom filter? #8682 [parquet]
- Rewrite
ParquetRecordBatchStream(async API) in terms of the PushDecoder #8677 [parquet] - [JSON] Add encoding for binary view #8674 [arrow]
- Refactor arrow-cast decimal casting to unify the rescale logic used in Parquet variant casts #8670 [arrow]
- [Variant] Support Uuid/
FixedSizeBinary(16)shredding #8665 - [Parquet]There should be an encoding counter to know how many encodings the repo supports in total #8662 [parquet]
- Improve
parse_data_typeforList,ListView,LargeList,LargeListView,FixedSizeList,Union,Map,RunEndCoded. #8648 [arrow] - [Variant] Support variant to arrow primitive support null/time/decimal_* #8637
- Return error from
RleDecoder::resetrather than panic #8632 [parquet] - Add bitwise ops on
BooleanBufferBuilderandMutableBufferthat mutate directly the buffer #8618 [arrow] - [Variant] Add variant_to_arrow Utf-8, LargeUtf8, Utf8View types support #8567 [arrow]
Fixed bugs:
- Regression: Parsing
List(Int64)results in nullable list in 57.0.0 and a non-nullable list in 57.1.0 #8883 - Regression: FixedSlizeList data type parsing fails on 57.1.0 #8880
- (dyn ArrayFormatterFactory + 'static) can't be safely shared between threads #8875
- RowNumber reader has wrong row group ordering #8864 [parquet]
ThriftMetadataWriter::write_column_indexescannot handle aColumnIndexMetaData::NONE#8815 [parquet]- "Archery test With other arrows" Integration test failing on main: #8813 [arrow]
- [Parquet] Writing in 57.0.0 seems 10% slower than 56.0.0 #8783 [parquet]
- Parquet reader cannot handle files with unknown logical types #8776 [parquet]
- zip now treats nulls as false in provided mask regardless of the underlying bit value #8721 [arrow]
- [avro] Incorrect version in crate.io landing page #8691 [arrow]
- Array: ViewType gc() has bug when array sum length exceed i32::MAX #8681 [arrow]
- Parquet 56: encounter
error: item_reader def levels are Nonewhen reading nested field with row filter #8657 [parquet] - Degnerate and non-nullable
FixedSizeListArrays are not handled #8623 [arrow] - [Parquet]Performance Degradation with RowFilter on Unsorted Columns due to Fragmented ReadPlan #8565 [parquet]
Documentation updates:
- docs: Add example for creating a
MutableBufferfromBuffer#8853 [arrow] (alamb) - docs: Add examples for creating MutableBuffer from Vec #8852 [arrow] (alamb)
- Improve ParquetDecoder docs #8802 [parquet] (alamb)
- Update docs for zero copy conversion of ScalarBuffer #8772 [arrow] (alamb)
- Add example to convert
PrimitiveArrayto aVec#8771 [arrow] (alamb) - docs: Add links for arrow-avro #8770 [arrow] (alamb)
- [Parquet] Minor: Update comments in page decompressor #8764 [parquet] (alamb)
- Document limitations of the
arrow_integration_testcrate #8738 [arrow] (phil-opp) - docs: Add link to the Arrow implementation status page #8732 [arrow] (alamb)
- docs: Update Parquet readme implementation status #8731 [parquet] (alamb)
Performance improvements:
RowConverter::from_binaryshould opportunistically take ownership of the buffer #8685 [arrow]- Speed up filter some more (up to 2x) #8868 [arrow] (Dandandan)
- Speed up
collect_booland removeunsafe, optimizetake_bits,take_nativefor null values #8849 [arrow] (Dandandan) - Change
BooleanBuffer::append_packed_rangeto useapply_bitwise_binary_op#8812 [arrow] (alamb) - [Parquet] Avoid copying
LogicalTypeinColumnOrder::get_sort_order, deprecateget_logical_type#8789 [parquet] (alamb) - perf: Speed up Parquet file writing (10%, back to speed of 56) #8786 [parquet] (etseidl)
- perf: override
ArrayIterdefault impl fornth,nth_back,lastandcount#8785 [arrow] (rluvaton) - [Parquet] Reduce one copy in
SerializedPageReader#8745 [parquet] (XiangpengHao) - Small optimization in Parquet varint decoder #8742 [parquet] (etseidl)
- perf: override
count,nth,nth_back,lastandmaxfor BitIterator #8696 [arrow] (rluvaton) - Add
FilterPredicate::filter_record_batch#8693 [arrow] (pepijnve) - perf: zero-copy path in
RowConverter::from_binary#8686 [arrow] (mzabaluev) - perf: add optimized zip implementation for scalars #8653 [arrow] (rluvaton)
- feat: add
apply_unary_opandapply_binary_opbitwise operations #8619 [arrow] (rluvaton) - [Parquet]Optimize the performance in record reader #8607 [parquet] (hhhizzz)
Closed issues:
- Variant to NullType conversion ignores strict casting #8810
- Unify display representation for
Field#8784 - Misleading configuration name: skip_arrow_metadata #8780
- Inconsistent display for types with Metadata #8761 [arrow]
- Internal
arrow-integration-testcrate is linked fromarrowdocs #8739 [arrow] - Add benchmark for RunEndEncoded casting #8709 [arrow]
- [Varaint] Support
VariantArray::valueto return aResult<Variant>#8672
Merged pull requests:
- Fix regression caused by changes in Display for DataType - display (
List(non-null Int64)instead ofList(nullable Int64)#8890 [parquet] [arrow] (etseidl) - Support parsing for old style FixedSizeList #8882 [arrow] (alamb)
- Make ArrayFormatterFactory Send + Sync and add a test #8878 [arrow] (tobixdev)
- Make
ArrowReaderOptions::with_virtual_columnserror rather than panic on invalid input #8867 [parquet] (alamb) - Fix errors when reading nested Lists with pushdown predicates. #8866 [parquet] (alamb)
- Fix
RowNumberReaderwhen not all row groups are selected #8863 [parquet] (vustef) - Respect page index policy option for ParquetObjectReader when it's not skip #8857 [parquet] (zhuqi-lucas)
- build(deps): update apache-avro requirement from 0.20.0 to 0.21.0 #8832 [arrow] (dependabot[bot])
- Allow Users to Provide Custom
ArrayFormatters when Pretty-Printing Record Batches #8829 [arrow] (tobixdev) - Allow reading of improperly constructed empty lists in Parquet metadata #8827 [parquet] (etseidl)
- [Variant] Fix cast logic for Variant to Arrow for DataType::Null #8825 (klion26)
- remove T: ParquetValueType bound on ValueStatistics #8824 [parquet] (pmarks)
- build(deps): update lz4_flex requirement from 0.11 to 0.12 #8820 [parquet] [arrow] (dependabot[bot])
- Fix bug in handling of empty Parquet page index structures #8817 [parquet] (etseidl)
- Parquet-concat: supports page index and bloom filter #8811 [parquet] (mapleFU)
- [Doc] Correct
ListArraydocumentation #8803 [arrow] (liamzwbao) - [Parquet] Add additional docs for
ArrowReaderOptionsandArrowReaderMetadata#8798 [parquet] (alamb) - [Variant] Enforce shredded-type validation in
shred_variant#8796 (liamzwbao) - Add
VariantPath::is_empty#8791 (friendlymatthew) - Add FilterBuilder::is_optimize_beneficial #8782 [arrow] (pepijnve)
- [Parquet] Allow reading of files with unknown logical types #8777 [parquet] (etseidl)
- bench: add
ArrayIterbenchmarks #8774 [arrow] (rluvaton) - Update Rust toolchain to 1.91 #8769 [parquet] [arrow] (mbrobbel)
- [Variant] Add variant to arrow for
DataType::{Binary/LargeBinary/BinaryView}#8768 [arrow] (klion26) - feat: parse
DataType::Union,DataType::Map,DataType::RunEndEncoded#8765 [arrow] (dqkqd) - Add options to control various aspects of Parquet metadata decoding #8763 [parquet] (etseidl)
- feat: Ensure consistent metadata display for data types #8760 [arrow] (mhilton)
- Clean up predicate_cache tests #8755 [parquet] (alamb)
- refactor
test_cache_projection_excludes_nested_columnsto use high level APIs #8754 [parquet] (alamb) - Add
mergeandmerge_nkernels #8753 [arrow] (pepijnve) - Fix lint in arrow-flight by updating assert_cmd after it upgraded #8741 [arrow] [arrow-flight] (vegarsti)
- Remove link to internal
arrow-integration-testcrate from mainarrowcrate #8740 [arrow] (phil-opp) - Implement hex decoding of JSON strings to binary arrays #8737 [arrow] (phil-opp)
- [Parquet] Adaptive Parquet Predicate Pushdown #8733 [parquet] (hhhizzz)
- [Parquet] Return error from
RleDecoder::reloadrather than panic #8729 [parquet] (liamzwbao) - fix:
ArrayIterdoes not report size hint correctly after advancing from the iterator back #8728 [arrow] (rluvaton) - perf: Use Vec::with_capacity in cast_to_run_end_encoded #8726 [arrow] (vegarsti)
- [Variant] Fix the index of an item in VariantArray in a unit test #8725 (martin-g)
- build(deps): bump actions/download-artifact from 5 to 6 #8720 (dependabot[bot])
- [Variant] Add try_value/value for VariantArray #8719 (klion26)
- General virtual columns support + row numbers as a first use-case #8715 [parquet] (vustef)
- feat: Parquet-layout add Index and Footer info #8712 [parquet] (mapleFU)
- fix:
zipnow treats nulls as false in provided mask regardless of the underlying bit value #8711 [arrow] (rluvaton) - Add benchmark for casting to RunEndEncoded (REE) #8710 [arrow] (vegarsti)
- [Minor]: Document visibility for enums produced by Thrift macros #8706 [parquet] (etseidl)
- Update
arrow-avroREADME.mdversion to 57 #8695 [arrow] (jecsand838) - Fix: ViewType gc on huge batch would produce bad output #8694 [arrow] (mapleFU)
- Refactor arrow-cast decimal casting to unify the rescale logic used in Parquet variant casts #8689 [arrow] (liamzwbao)
- check bit width to avoid panic in DeltaBitPackDecoder #8688 [parquet] (rambleraptor)
- [thrift-remodel] Use
thrift_enummacro forConvertedType#8680 [parquet] (etseidl) - [JSON] Map key supports utf8 view #8679 [arrow] (mapleFU)
- [JSON] Add encoding for binary view #8675 [arrow] (mapleFU)
- [Parquet] Account for FileDecryptor in ParquetMetaData heap size calculation #8671 [parquet] (adamreeve)
- chore: update
OffsetBuffer::from_lengths(std::iter::repeat_n(<val>, <repeat>));withOffsetBuffer::from_repeated_length(<val>, <repeat>);#8669 [arrow] (rluvaton) - [Variant] Support
shred_variantfor Uuids #8666 (friendlymatthew) - [Variant] Remove
create_test_variant_arrayhelper method #8664 (friendlymatthew) - [parquet] Adding counting method in thrift_enum macro to support ENCODING_SLOTS #8663 [parquet] (hhhizzz)
- chore: add test case of RowSelection::trim #8660 [parquet] (lichuang)
- feat: add
new_repeatedtoByteArray#8659 [arrow] (rluvaton) - perf: add
repeat_slice_n_timestoMutableBuffer#8658 [arrow] (rluvaton) - perf: add optimized function to create offset with same length #8656 [arrow] (rluvaton)
- [Variant]
rescale_decimalfollowup #8655 [arrow] (liamzwbao) - feat: parse DataType
List,ListView,LargeList,LargeListView,FixedSizeList#8649 [arrow] (dqkqd) - Support more operations on ListView #8645 [arrow] (a10y)
- [Variant] Implement primitive type access for null/time/decimal* #8638 (klion26)
- [Variant] refactor: Split builder.rs into several smaller files #8635 (Weijun-H)
- add
try_new_with_lengthconstructor toFixedSizeList#8624 [arrow] (connortsui20) - Change some panics to errors in parquet decoder #8602 [parquet] (rambleraptor)
- Support
variant_to_arrowfor utf8 #8600 [arrow] (sdf-jkl) - Cast support for RunEndEncoded arrays #8589 [arrow] (vegarsti)
-
57.0.023 Oct 2025Release notes
Open source →Breaking changes:
- Use
Arc<FileEncryptionProperties>everywhere to be be consistent withFileDecryptionProperties#8626 [parquet] (alamb) - feat: Improve DataType display for
RunEndEncoded#8596 [arrow] (Weijun-H) - Add
ArrowError::AvroError, remaining types and roundtrip tests toarrow-avro, #8595 [arrow] (jecsand838) - [thrift-remodel] Refactor Thrift encryption and store encodings as bitmask #8587 [parquet] (etseidl)
- feat: Enhance
Mapdisplay formatting in DataType #8570 [arrow] (Weijun-H) - feat: Enhance DataType display formatting for
ListViewandLargeListViewvariants #8569 [arrow] (Weijun-H) - Use custom thrift parser for parquet metadata (phase 1 of Thrift remodel) #8530 [parquet] (etseidl)
- refactor: improve display formatting for Union #8529 [arrow] (Weijun-H)
- Use
Arc<FileDecryptionProperties>to reduce size of ParquetMetadata and avoid copying whenencryptionis enabled #8470 [parquet] (alamb) - Fix for column name based projection mask creation #8447 [parquet] (etseidl)
- Improve Display formatting of DataType::Timestamp #8425 [parquet] [arrow] (emilk)
- Use more compact Debug formatting of Field #8424 [arrow] (emilk)
- Reuse zstd compression context when writing IPC #8405 [arrow] [arrow-flight] (albertlockett)
- [Decimal] Add scale argument to validation functions to ensure accurate error logging #8396 [arrow] (Weijun-H)
- Quote
DataType::Structfield names inDisplayformatting #8291 [parquet] [arrow] (emilk) - Improve
DisplayforDataTypeandField#8290 [parquet] [arrow] (emilk) - Bump pyo3 to 0.26.0 #8286 (mbrobbel)
Implemented enhancements:
- Added Avro support (new
arrow-avrocrate) #4886 - parquet-rewrite: supports compression level and write batch size #8639
- Error not panic when int96 stastistics aren't size 12 #8614 [parquet]
- [Variant] Make
VariantArrayiterable #8612 - [Variant] impl
PartialEqforVariantArray#8610 - [Variant] Remove potential panics when probing
VariantArray#8609 - [Variant] Remove ceremony of going from list of
VarianttoVariantArray#8606 - Eliminate redundant validation in
RecordBatch::project#8591 [arrow] - [PARQUET][BENCH] Arrow writer bench with compression and/or page v2 #8559 [parquet]
- [Variant] casting functions are confusingly named #8531 [parquet]
- Support writing GeospatialStatistics in Parquet writer #8523 [parquet]
- [thrift-remodel] Optimize
convert_row_groups#8517 [parquet] - [Variant] Add variant to arrow primitive support for boolean/timestamp/time #8515
- Test
thrift-remodelbranch with DataFusion #8513 [parquet] - Make
UnionArray::is_denseMethod Public #8503 [arrow] - Add
append_nmethod toFixedSizeBinaryDictionaryBuilder#8497 [arrow] - [Parquet] Reduce size of ParquetMetadata when encryption feature is enabled #8469 [parquet]
- [Parquet] Remove useless mut requirements in geting bloom filter function #8461 [parquet]
- Change
serdedependency toserde_corewhere applicable #8451 [arrow] - [Parquet] Split
ParquetMetadataReaderinto IO/decoder state machine and thrift parsing #8439 [parquet] - Remove compiler warning for redundant config enablement #8412 [arrow]
- Add geospatial statistics creation support for GEOMETRY/GEOGRAPHY Parquet logical types #8411 [arrow]
arrow_jsonlackswith_timestamp_formatfunctions likearrow_csvhad offered #8398 [arrow]- Unify API for writing column chunks / row groups in parallel #8389 [parquet]
- Reuse zstd context in arrow IPC writer #8386 [arrow] [arrow-flight]
- [Variant] Support reading/writing Parquet Variant LogicalType #8370 [parquet]
- [Variant] Implement a
shred_variantfunction #8361 - [Parquet] Expose ReadPlan and ReadPlanBuilder #8347 [parquet]
- [Variant] [Shredding] Support typed_access for
List#8337 [parquet] - [Variant] [Shredding] Support typed_access for
Struct#8336 [parquet] - [Variant] [Shredding] Support typed_access for
Time64(Microsecond)#8334 [parquet] - [Variant] [Shredding] Support typed_access for
Decimal128#8332 [parquet] - [Variant] [Shredding] Support typed_access for
Timestamp(Microsecond, _)andTimestamp(Nanosecond, _)#8331 [parquet] - [Variant] [Shredding] Support typed_access for
Date32#8330 [parquet] - [Variant] Support strict casting for all data types #8303
- [Variant] Support typed access for string types in variant_get #8285
- [Variant]: Implement
DataType::FixedSizeListsupport forcast_to_variantkernel #8281
Fixed bugs:
- Fix arrow-avro Writer Documentation related to AvroBinaryFormat #8631 [arrow]
- Decimal -> Decimal cast wrongly fails for large scale reduction #8579 [arrow]
- [Parquet] Avoid fetching multiple pages when
max_predicate_cache_sizeis 0 #8542 [parquet] - DataType parsing no longer works correctly for old formatted timestamps #8539 [parquet] [arrow]
- [Parquet] ArrowWriter flush does not work #8534 [parquet]
arrow::compute::interleavefails with struct arrays with no fields #8533 [arrow]- [Parquet] Over memory consumation for writer page v1 compressed #8526 [parquet]
- Incorrect Behavior of Collecting a filtered iterator to a BooleanArray #8505 [arrow]
- [Parquet] ProjectionMask::columns name handling is bug prone #8443 [parquet]
- [Variant] Shredded typed_value columns must have valid variant types #8435 [parquet]
- cargo test -p parquet fails with default
ulimit#8406 [parquet] - Column with List(Struct) causes failed to decode level data for struct array #8404 [parquet]
- Binaryview Utf8 Cast Issue #8403 [arrow]
- Decimal precision validation displays value without accounting for scale #8382 [arrow]
- [Variant]
VariantArray::data_typereturnsStructType, causingArray::as_structto panic #8319 [parquet] - [Variant] writing a VariantArray to parquet panics #8296 [parquet]
Documentation updates:
Performance improvements:
- [parquet] Improve encoding mask API (wrap bare i32 in a struct w/ docs) #8588 [parquet]
- bench: create
zipkernel benchmarks #8654 [arrow] (rluvaton) - Skip redundant validation checks in RecordBatch#project #8583 [arrow] (pepijnve)
- [thrift-remodel] Remove conversion functions for row group and column metadata #8574 [parquet] (etseidl)
- [PARQUET] Improve memory efficency for compressed writer parquet 1.0 #8527 [parquet] (lilianm)
- perf: improve
GenericByteBuilder::append_arrayto use SIMD for extending the offsets #8388 [arrow] (rluvaton)
Closed issues:
- Utf-8, LargeUtf8, Utf8View #8601
- [Variant] Improve the get type logic for DataType in variant to arrow row builder #8538
- Add a README.md for arrow-avro #8504 [arrow]
- Fix UnionArray references to "positive" values #8418 [arrow]
- [Variant]
metadatafield should be marked is non-nullable #8410 [parquet] - [Avro] Example read_with_utf8view.rs fails to run with error "Error: ParseError("Unexpected EOF while reading Avro header")" #8380 [arrow]
- [Geospatial]: Add CI checks for
parquet-geospatialcrate #8377 - [Geospatial] Create new
parquet-geometrycrate #8374
Merged pull requests:
- parquet-rewrite: add write_batch_size and compression_level config #8642 [parquet] (mapleFU)
- Introduce a ThriftProtocolError to avoid allocating and formattings strings for error messages #8636 [parquet] (jhorstmann)
- [thrift-remodel] Add macro to reduce boilerplate necessary to implement Thrift serialization #8634 [parquet] (etseidl)
- Fix Writer docs and rename
AvroBinaryFormattoAvroSoeFormat#8633 [arrow] (jecsand838) - [Variant] Bulk insert elements into List and Object Builders #8629 (friendlymatthew)
- [Variant] impl
PartialEqandFromIterator<Option<..>>forVariantArray#8627 (friendlymatthew) - [Variant] Remove ceremony from iterator of variants into VariantArray #8625 (friendlymatthew)
- Undeprecate
ArrowWriter::into_serialized_writerand add docs #8621 [parquet] (alamb) - fix: incorrect assertion in
BitChunks::new#8620 [arrow] (rluvaton) - [Variant] Clean up redundant
get_type_name#8617 (liamzwbao) - [Minor] Hide thrift macros #8616 [parquet] (etseidl)
- Deprecate
parquet::formatmodule #8615 [parquet] (etseidl) - [Variant] Make
VariantArrayiterable #8613 (friendlymatthew) - [Variant] Impl
ExtendforVariantArrayBuilder#8611 (friendlymatthew) - build(deps): bump actions/setup-node from 5 to 6 #8604 (dependabot[bot])
- Check int96 min/max instead of panicking #8603 [parquet] (rambleraptor)
- [thrift-remodel] Refactor Parquet Thrift code into new
thriftmodule #8599 [parquet] (etseidl) - [Parquet] Remove use of
parquet::formatin metadata bench code #8598 [parquet] (lichuang) - Remove experimental warning from
extensionmodule #8597 [arrow] (mbrobbel) - Adding
try_append_valueimplementation toByteViewBuilder#8594 [arrow] (samueleresca) - Add RecordBatch::project microbenchmark #8592 [arrow] (pepijnve)
- [parquet] Add a sync fn to ArrowWriter that flushes Writer #8586 [parquet] (PiotrSrebrny)
- chore: use magic number
FOOTER_SIZEinstead of hard code number #8585 [parquet] (lichuang) - Add support for run-end encoded (REE) arrays in arrow-avro #8584 [arrow] (jecsand838)
- Unify API for writing column chunks / row groups in parallel #8582 [parquet] (adamreeve)
- Fix linting issues missed by #8506 #8581 [parquet] (etseidl)
- Fix broken decimal->decimal casting with large scale reduction #8580 [arrow] (scovich)
- Migrate
arrowand workspace to Rust 2024 #8578 [parquet] [arrow] [arrow-flight] (mbrobbel) - Fix doctests of parquet push decoded without default features #8577 [parquet] (mbrobbel)
- Avoid panics and warnings when building avro without default features #8576 [arrow] (mbrobbel)
- Add support for 64-bit Schema Registry IDs (Id64) in arrow-avro #8575 [arrow] (jecsand838)
- fix: bug when struct nullability determined from
Dict<_, ByteArray>>column #8573 [parquet] (albertlockett) - fix: Support
interleave_structto handle empty fields #8563 [arrow] (Weijun-H) - [Variant] Define and use VariantDecimalType trait #8562 (scovich)
- [PARQUET] Update parquet writer bench with compression and pagev2 #8560 [parquet] (lilianm)
- Replace serde with
serde_corewhen possible #8558 [arrow] (AdamGS) - fix: use default field name when name is None in Field conversion #8557 [arrow] (Weijun-H)
- Add arrow-avro README.md file #8556 [arrow] (jecsand838)
- minor(parquet): Fix test_not_found on Windows #8555 [parquet] (nuno-faria)
- [Parquet] Avoid fetching multiple pages when the predicate cache is disabled #8554 [parquet] (nuno-faria)
- [Variant] Support variant to
Decimal32/64/128/256#8552 [arrow] (liamzwbao) - Arrow-avro Writer Dense Union support #8550 [arrow] (nathaniel-d-ef)
- Arrow-Avro: Resolve named field discrepancies #8546 [arrow] (nathaniel-d-ef)
- Migrate
arrow-avroto Rust 2024 #8545 [arrow] (mbrobbel) - feat: Export
is_densepublic #8544 [arrow] (Weijun-H) - Fix "Incorrect Behavior of Collecting a filtered iterator to a BooleanArray" #8543 [arrow] (tobixdev)
- Support old syntax for DataType parsing #8541 [arrow] (alamb)
- [Variant] Decimal unshredding support #8540 [parquet] (scovich)
- [Variant] Improve documentation and make kernels consistent #8536 [parquet] (alamb)
- feat: support casting from null to float16 #8535 [arrow] (chenkovsky)
- Add benchmarks for FromIter (PrimitiveArray and BooleanArray) #8525 [arrow] (tobixdev)
- Support writing GeospatialStatistics in Parquet writer #8524 [parquet] (paleolimbot)
- Fix some new rustdoc warnings #8522 [parquet] (etseidl)
- [Variant] Reverse VariantAsPrimitive trait to PrimitiveFromVariant #8519 (scovich)
- [Variant] Add variant to arrow primitive support for boolean/timestamp/time #8516 (klion26)
- [Variant] Add list support to unshred_variant #8514 [parquet] (scovich)
- Migrate
parquet-variant-jsonto Rust 2024 #8512 (mbrobbel) - Migrate
parquet-variant-computeto Rust 2024 #8511 (mbrobbel) - Migrate
parquet-variantto Rust 2024 #8510 (mbrobbel) - Migrate
parquet-geospatialto Rust 2024 #8509 (mbrobbel) - Migrate
parquet_derive_testto Rust 2024 #8508 (mbrobbel) - Migrate
parquet_deriveto Rust 2024 #8507 (mbrobbel) - Migrate
parquetto Rust 2024 #8506 [parquet] (mbrobbel) - [Variant] ReadOnlyMetadataBuilder borrows its underlying VariantMetadata #8502 (scovich)
- [Variant] Add a VariantBuilderExt impl for VariantValueArrayBuilder #8501 (scovich)
- build(deps): update sysinfo requirement from 0.36.0 to 0.37.1 #8500 [parquet] (dependabot[bot])
- [Variant] Introduce new BorrowedShreddingState concept #8499 (scovich)
- Add
append_nmethod toFixedSizeBinaryDictionaryBuilder#8498 [arrow] (albertlockett) - Fix docs.rs build: Use
doc_cfginstead of removeddoc_auto_cfg#8494 [parquet] [arrow] [arrow-flight] (mbrobbel) - Remove allow unused from arrow-avro lib.rs file #8493 [arrow] (jecsand838)
- Regression Testing, Bug Fixes, and Public API Tightening for arrow-avro #8492 [arrow] (jecsand838)
- Migrate
arrow-stringto Rust 2024 #8491 [arrow] (mbrobbel) - Migrate
arrow-selectto Rust 2024 #8490 [arrow] (mbrobbel) - Migrate
arrow-schemato Rust 2024 #8489 [arrow] (mbrobbel) - Migrate
arrow-rowto Rust 2024 #8488 [arrow] (mbrobbel) - Migrate
arrow-pyarrow-testingto Rust 2024 #8487 (mbrobbel) - Migrate
arrow-pyarrow-integration-testingto Rust 2024 #8486 (mbrobbel) - Migrate
arrow-pyarrowto Rust 2024 #8485 (mbrobbel) - Migrate
arrow-ordto Rust 2024 #8484 [arrow] (mbrobbel) - [Variant] Support strict casting for Decimals #8483 (liamzwbao)
- feat(json): Add temporal formatting options when write to JSON #8482 [arrow] (linyihai)
- [Variant] Define and use unshred_variant function #8481 [parquet] (scovich)
- [Minor] Remove private APIs from Parquet metadata benchmark #8478 [parquet] (etseidl)
- Add examples of using
Field::try_extension_type#8475 [arrow] (alamb) - Fix Rustfmt in arrow-cast #8473 [arrow] (mbrobbel)
- Disable incremental builds in CI #8471 (mbrobbel)
- Update Rust toolchain to 1.90 #8468 [arrow] (mbrobbel)
- [Parquet] Minor: Remove mut ref for getting row-group bloom filter #8462 [parquet] (mapleFU)
- refactor: split
numdependency #8459 [parquet] [arrow] (crepererum) - Migrate
arrow-jsonto Rust 2024 #8458 [arrow] (mbrobbel) - Migrate
arrow-ipcto Rust 2024 #8457 [arrow] (mbrobbel) - Migrate
arrow-flightto Rust 2024 #8456 [arrow] [arrow-flight] (mbrobbel) - Migrate
arrow-datato Rust 2024 #8455 [arrow] (mbrobbel) - Migrate
arrow-csvto Rust 2024 #8454 [arrow] (mbrobbel) - Migrate
arrow-castto Rust 2024 #8453 [arrow] (mbrobbel) - Migrate
arrow-bufferto Rust 2024 #8452 [arrow] (mbrobbel) - Migrate
arrow-arrayto Rust 2024 #8450 [arrow] (mbrobbel) - Migrate
arrow-arithto Rust 2024 #8449 [arrow] (mbrobbel) - Expose
fieldsinStructBuilder#8448 [arrow] (lewiszlw) - [Variant] Simpler shredding state #8444 [parquet] (scovich)
- Unpin comfytable #8440 [arrow] (alamb)
- Variant integration fixes #8438 [parquet] (scovich)
- Refactor: extract FooterTail from ParquetMetadataReader #8437 [parquet] (alamb)
- Refactor: Move parquet metadata parsing code into its own module #8436 [parquet] (alamb)
- Update
UnionArraywording to 'non-negative' #8434 [arrow] (jdockerty) - Adds Duration(TimeUnit) support to arrow-avro reader and writer #8433 [arrow] (nathaniel-d-ef)
- Update release schedule #8432 (mbrobbel)
- expose read plan and plan builder via mod #8431 [parquet] (yeya24)
- Bump MSRV to 1.85 #8429 [arrow] (mbrobbel)
- Fix clippy #8426 (alamb)
- Fix red main by updating test #8421 [parquet] (emilk)
- Implement AsRef for Schema and Field #8417 [arrow] (findepi)
- [Variant] mark metadata field as non-nullable #8416 (ding-young)
- Respect
CastOptions.safewhen castingBinaryView→Utf8View(returnnullfor invalid UTF‑8) #8415 [arrow] (kosiew) - Add Parquet geospatial statistics utility #8414 [arrow] (paleolimbot)
- Remove explicit default cfg option #8413 [arrow] (abacef)
- Support parquet canonical extension type roundtrip #8409 [parquet] (alamb)
- Support reading/writing
VariantArrayto parquet with Variant LogicalType #8408 [parquet] (alamb) - Follow-up on arrow-avro Documentation #8402 [arrow] (jecsand838)
- [Variant][Shredding] Support typed_access for timestamp_micro/timestamp_nano #8401 [parquet] (klion26)
- Expose ReadPlan and ReadPlanBuilder #8399 [parquet] (yeya24)
- Propagate errors instead of panics: Replace usages of
newwithtry_newfor Array types #8397 [arrow] (Jefffrey) - [Variant] Fix NULL handling for shredded object fields #8395 (scovich)
- Add Arrow Variant Extension Type, remove
Arrayimpl forVariantArrayandShreddedVariantFieldArray#8392 [parquet] (alamb) - Minor cleanup creating Schema #8391 [parquet] (alamb)
- [Geospatial]: Add CI checks for
parquet-geospatialcrate #8390 (kylebarron) - Follow-up Improvements to Avro union handling #8385 [arrow] (jecsand838)
- fix: reset the offset of 'file_for_view' #8381 [arrow] (TrevorADHD)
- [Variant] [Shredding] feat: Support typed_access for Date32 #8379 [parquet] (PinkCrow007)
- [Geospatial]: Scaffolding for new
parquet-geospatialcrate #8375 (kylebarron) - Avro writer prefix support #8371 [arrow] (nathaniel-d-ef)
- [Variant] Define new shred_variant function #8366 (scovich)
- Add arrow-avro Reader support for Dense Union and Union resolution (Part 2) #8349 [arrow] (jecsand838)
- Move ParquetMetadata decoder state machine into ParquetMetadataPushDecoder #8340 [parquet] (alamb)
- [Variant]: Implement
DataType::FixedSizeListsupport forcast_to_variantkernel #8282 (liamzwbao)
- Use
-
56.2.111 May 2026Nothing published for this version
-
56.2.023 Sep 2025Release notes
Open source →- [Variant] [Shredding] Support typed_access for Utf8 and BinaryView #8364 [parquet] (petern48)
- Fix casting floats to Decimal64 #8363 [arrow] (AdamGS)
- [Variant] Implement new VariantValueArrayBuilder #8360 (scovich)
- [Variant] Add constants for empty variant metadata #8359 (scovich)
- [Variant] Allow lossless casting from integer to floating point #8357 (scovich)
- [Variant] Minor code cleanups #8356 (scovich)
- [Variant] Remove unused metadata from variant ShreddingState #8355 (scovich)
- Adds Map & Enum support, round-trip & benchmark tests #8353 [arrow] (nathaniel-d-ef)
- [Variant] [Shredding] feat: Support typed_access for FixedSizeBinary #8352 (petern48)
- Add arrow-avro Reader support for Dense Union and Union resolution (Part 1) #8348 [arrow] (jecsand838)
- [Variant] feat: Support typed_access for Boolean #8346 (Weijun-H)
- [Variant] Make VariantToArrowRowBuilder an enum #8345 (scovich)
- [Variant] Rename VariantShreddingRowBuilder to VariantToArrowRowBuilder #8344 (scovich)
- [Variant] Add tests for variant_get requesting Some struct #8343 (scovich)
- [Variant] Add nullable arg to StructArrayBuilder::with_field #8342 (scovich)
- Minor: avoid an
Arc::clonein CacheOptions for Parquet PredicateCache #8338 [parquet] (alamb) - Fix
can_cast_typesfor temporal toUtf8View#8328 [arrow] (findepi) - Update
variant_integrationtest to use final approvedparquet-testingdata #8325 [parquet] (alamb) - [Variant] ParentState tracks builder-specific state in a uniform way #8324 [parquet] (scovich)
- [Variant] Remove boilerplate from make_shredding_row_builder #8322 (scovich)
- [Variant] Move VariantAsPrimitive to type_conversions.rs #8321 (scovich)
- [Variant] Remove unused output builder files #8320 (scovich)
- Add arrow-avro examples and Reader documentation #8316 [arrow] (jecsand838)
- Expose predicates from RowFilter #8315 [parquet] (yeya24)
- [Variant] Implement row builders for cast_to_variant #8299 (scovich)
- Adds additional type support to arrow-avro writer #8298 [arrow] (nathaniel-d-ef)
- Use apache/arrow-dotnet for integration test #8295 (kou)
- Add projection with default values support to
RecordDecoder#8293 [arrow] (jecsand838) - Add array/map/fixed schema resolution and default value support to arrow-avro codec #8292 [arrow] (jecsand838)
- Bump actions/labeler from 6.0.0 to 6.0.1 #8288 (dependabot[bot])
- Bump actions/github-script from 7 to 8 #8287 (dependabot[bot])
- [Variant] Add as_u* for Variant #8284 (klion26)
- [Variant] Support Shredded Objects in variant_get (take 2) #8280 (scovich)
- Bump actions/setup-node from 4 to 5 #8279 (dependabot[bot])
- Bump actions/setup-python from 5 to 6 #8278 (dependabot[bot])
- Bump actions/labeler from 5.0.0 to 6.0.0 #8276 (dependabot[bot])
- Impl
DisplayforTz#8275 [arrow] (kylebarron) - Added List and Struct Encoding to arrow-avro Writer #8274 [arrow] (jecsand838)
- Add into_builder method for WriterProperties #8272 [parquet] (corwinjoy)
- chore(parquet/record/field): dont truncate timestamps on display #8266 [parquet] (Erigara)
- [Parquet] Write row group with async writer #8262 [parquet] (lilianm)
- Parquet: Do not compress v2 data page when compress is bad quality #8257 [parquet] (mapleFU)
- Add Decimal32 and Decimal64 support to arrow-avro Reader #8255 [arrow] (jecsand838)
- [Minor] Backport changes to metadata benchmark #8251 [parquet] (etseidl)
- Update hashbrown requirement from 0.15.1 to 0.16.0 #8248 [parquet] [arrow] (dependabot[bot])
- Sort: Change lexsort comment from stable to unstable #8245 [arrow] (mapleFU)
- pin comfy-table to 7.1.2 #8244 [arrow] (zachschuermann)
- Adds Confluent wire format handling to arrow-avro crate #8242 [arrow] (nathaniel-d-ef)
- feat: gRPC compression support for flight CLI #8240 [arrow] [arrow-flight] (crepererum)
- feat:
SSLKEYLOGFILEsupport for flight CLI #8239 [arrow] [arrow-flight] (crepererum) - [Variant] Refactor
cast_to_variant#8235 (liamzwbao) - [Variant] add strict mode to cast_to_variant #8233 (codephage2020)
- [Variant] Add Variant::as_f16 #8232 (klion26)
- Unpin nightly rust version (MIRI job) #8229 (mbrobbel)
- Update apache-avro requirement from 0.14.0 to 0.20.0 #8226 [arrow] (dependabot[bot])
- Bump actions/upload-pages-artifact from 3 to 4 #8224 (dependabot[bot])
- Added arrow-avro enum mapping support for schema resolution #8223 [arrow] (jecsand838)
- Added arrow-avro schema resolution value skipping #8220 [arrow] (jecsand838)
- Fix error condition in doc comment of
Field::try_canonical_extension_type#8216 [arrow] (mbrobbel) - [Variant]: Implement
DataType::Durationsupport forcast_to_variantkernel #8215 (liamzwbao) - [Variant] feat: remove unnecessary unwraps in
Object::finish#8214 (Weijun-H) - [avro] Fix Avro decoder bitmap corruption when nullable field decoding fails #8213 [arrow] (yongkyunlee)
- Restore accidentally removed method Block::to_ne_bytes #8211 [parquet] (jhorstmann)
- [avro] Support all default types for avro schema's record field #8210 [arrow] (yongkyunlee)
- [Variant] Support read-only metadata builders #8208 (scovich)
- Avro to arrow schema conversion fails when a field has a default type that is not string #8209
- parquet: No method named
to_ne_bytesfound for structbloom_filter::Blockfor targets390x-unknown-linux-gnu#8207 - [Variant] cast_to_variant will panic on certain
Date64or Timestamp Values values #8155 - Parquet: Avoid page-size overflows i32 #8264 [parquet] (mapleFU)
Documentation updates:
- Update docstring comment for Writer::write() in writer.rs #8267 [arrow] (YKoustubhRao)
Closed issues:
- comfy-table release 7.2.0 breaks MSRV #8243
- [Variant] Add
Variant::as_f16#8228 - Support appending raw bytes to variant objects and lists #8217
VariantArrayBuilderusesParentStatefor simpler rollbacks #8205- Make
ObjectBuilder::finishsignature infallible #8184 - Improve performance of
i256tof64#8013
Merged pull requests:
- [Variant] Support Variant to PrimitiveArrow for unsigned integer #8369 (klion26)
- [Variant] [Shredding] Support typed_access for Utf8 and BinaryView #8364 [parquet] (petern48)
- Fix casting floats to Decimal64 #8363 [arrow] (AdamGS)
- [Variant] Implement new VariantValueArrayBuilder #8360 (scovich)
- [Variant] Add constants for empty variant metadata #8359 (scovich)
- [Variant] Allow lossless casting from integer to floating point #8357 (scovich)
- [Variant] Minor code cleanups #8356 (scovich)
- [Variant] Remove unused metadata from variant ShreddingState #8355 (scovich)
- Adds Map & Enum support, round-trip & benchmark tests #8353 [arrow] (nathaniel-d-ef)
- [Variant] [Shredding] feat: Support typed_access for FixedSizeBinary #8352 (petern48)
- Add arrow-avro Reader support for Dense Union and Union resolution (Part 1) #8348 [arrow] (jecsand838)
- [Variant] feat: Support typed_access for Boolean #8346 (Weijun-H)
- [Variant] Make VariantToArrowRowBuilder an enum #8345 (scovich)
- [Variant] Rename VariantShreddingRowBuilder to VariantToArrowRowBuilder #8344 (scovich)
- [Variant] Add tests for variant_get requesting Some struct #8343 (scovich)
- [Variant] Add nullable arg to StructArrayBuilder::with_field #8342 (scovich)
- Minor: avoid an
Arc::clonein CacheOptions for Parquet PredicateCache #8338 [parquet] (alamb) - Fix
can_cast_typesfor temporal toUtf8View#8328 [arrow] (findepi) - Update
variant_integrationtest to use final approvedparquet-testingdata #8325 [parquet] (alamb) - [Variant] ParentState tracks builder-specific state in a uniform way #8324 [parquet] (scovich)
- [Variant] Remove boilerplate from make_shredding_row_builder #8322 (scovich)
- [Variant] Move VariantAsPrimitive to type_conversions.rs #8321 (scovich)
- [Variant] Remove unused output builder files #8320 (scovich)
- Add arrow-avro examples and Reader documentation #8316 [arrow] (jecsand838)
- Expose predicates from RowFilter #8315 [parquet] (yeya24)
- [Variant] Implement row builders for cast_to_variant #8299 (scovich)
- Adds additional type support to arrow-avro writer #8298 [arrow] (nathaniel-d-ef)
- Use apache/arrow-dotnet for integration test #8295 (kou)
- Add projection with default values support to
RecordDecoder#8293 [arrow] (jecsand838) - Add array/map/fixed schema resolution and default value support to arrow-avro codec #8292 [arrow] (jecsand838)
- Bump actions/labeler from 6.0.0 to 6.0.1 #8288 (dependabot[bot])
- Bump actions/github-script from 7 to 8 #8287 (dependabot[bot])
- [Variant] Add as_u* for Variant #8284 (klion26)
- [Variant] Support Shredded Objects in variant_get (take 2) #8280 (scovich)
- Bump actions/setup-node from 4 to 5 #8279 (dependabot[bot])
- Bump actions/setup-python from 5 to 6 #8278 (dependabot[bot])
- Bump actions/labeler from 5.0.0 to 6.0.0 #8276 (dependabot[bot])
- Impl
DisplayforTz#8275 [arrow] (kylebarron) - Added List and Struct Encoding to arrow-avro Writer #8274 [arrow] (jecsand838)
- Add into_builder method for WriterProperties #8272 [parquet] (corwinjoy)
- chore(parquet/record/field): dont truncate timestamps on display #8266 [parquet] (Erigara)
- [Parquet] Write row group with async writer #8262 [parquet] (lilianm)
- Parquet: Do not compress v2 data page when compress is bad quality #8257 [parquet] (mapleFU)
- Add Decimal32 and Decimal64 support to arrow-avro Reader #8255 [arrow] (jecsand838)
- [Minor] Backport changes to metadata benchmark #8251 [parquet] (etseidl)
- Update hashbrown requirement from 0.15.1 to 0.16.0 #8248 [parquet] [arrow] (dependabot[bot])
- Sort: Change lexsort comment from stable to unstable #8245 [arrow] (mapleFU)
- pin comfy-table to 7.1.2 #8244 [arrow] (zachschuermann)
- Adds Confluent wire format handling to arrow-avro crate #8242 [arrow] (nathaniel-d-ef)
- feat: gRPC compression support for flight CLI #8240 [arrow] [arrow-flight] (crepererum)
- feat:
SSLKEYLOGFILEsupport for flight CLI #8239 [arrow] [arrow-flight] (crepererum) - [Variant] Refactor
cast_to_variant#8235 (liamzwbao) - [Variant] add strict mode to cast_to_variant #8233 (codephage2020)
- [Variant] Add Variant::as_f16 #8232 (klion26)
- Unpin nightly rust version (MIRI job) #8229 (mbrobbel)
- Update apache-avro requirement from 0.14.0 to 0.20.0 #8226 [arrow] (dependabot[bot])
- Bump actions/upload-pages-artifact from 3 to 4 #8224 (dependabot[bot])
- Added arrow-avro enum mapping support for schema resolution #8223 [arrow] (jecsand838)
- Added arrow-avro schema resolution value skipping #8220 [arrow] (jecsand838)
- Fix error condition in doc comment of
Field::try_canonical_extension_type#8216 [arrow] (mbrobbel) - [Variant]: Implement
DataType::Durationsupport forcast_to_variantkernel #8215 (liamzwbao) - [Variant] feat: remove unnecessary unwraps in
Object::finish#8214 (Weijun-H) - [avro] Fix Avro decoder bitmap corruption when nullable field decoding fails #8213 [arrow] (yongkyunlee)
- Restore accidentally removed method Block::to_ne_bytes #8211 [parquet] (jhorstmann)
- [avro] Support all default types for avro schema's record field #8210 [arrow] (yongkyunlee)
- [Variant] Support read-only metadata builders #8208 (scovich)
- [Variant] VariantArrayBuilder uses MetadataBuilder and ValueBuilder #8206 (scovich)
- [Variant]: Implement DataType::List/LargeList support for cast_to_variant kernel #8201 (sdf-jkl)
- [Variant]: Implement
DataType::Unionsupport forcast_to_variantkernel #8196 (liamzwbao) - [Variant] Support typed access for numeric types in variant_get #8179 (superserious-dev)
- [Variant]: Implement
DataType::Unionsupport forcast_to_variantkernel #8195 - [Variant]: Implement
DataType::Durationsupport forcast_to_variantkernel #8194 - [Variant] Support typed access for numeric types in variant_get #8178
- [Parquet] Implement a "push style" API for decoding Parquet Metadata #8164
- [Variant] Support creating Variants with pre-existing Metadata #8152
- [Variant] Support Shredded Objects in
variant_get: typed path access (STEP 1) #8150 - [Variant] Add
variantfeature toparquetcrate #8132 - [Parquet] Concurrent writes with ArrowWriter.get_column_writers should parallelize across row groups #8115
- [Variant] Implement
VariantArray::valuefor shredded variants #8091 - [Variant] Integration tests for reading parquet w/ Variants #8084
- [Variant]: Implement
DataType::Mapsupport forcast_to_variantkernel #8063 - [Variant]: Implement
DataType::List/LargeListsupport forcast_to_variantkernel #8060
-
56.1.025 Aug 2025Release notes
Open source →Implemented enhancements:
- Implement cast and other operations on decimal32 and decimal64 #7815 #8204 [arrow]
- Speed up Parquet filter pushdown with predicate cache #8203 [parquet]
- Optionally read parquet page indexes #8070 [parquet]
- Parquet reader: add method for sync reader read bloom filter #8023 [parquet]
- [parquet] Support writing logically equivalent types to
ArrowWriter#8012 [parquet] - Improve StringArray(Utf8) sort performance #7847 [arrow]
- feat: arrow-ipc delta dictionary support #8001 [arrow] (JakeDern)
Fixed bugs:
- The Rustdocs are clean CI job is failing #8175
- [avro] Bug in resolving avro schema with named type #8045 [arrow]
- Doc test failure (test arrow-avro/src/lib.rs - reader) when verifying avro 56.0.0 RC1 release #8018 [arrow]
Documentation updates:
- arrow-row: Document dictionary handling #8168 [arrow] (alamb)
- Docs: Clarify that Array::value does not check for nulls #8065 [arrow] (alamb)
- docs: Fix a typo in README #8036 (EricccTaiwan)
- Add more comments to the internal parquet reader #7932 [parquet] (alamb)
Performance improvements:
- perf(arrow-ipc): avoid counting nulls in
RecordBatchDecoder#8127 [arrow] (rluvaton) - Use
Vecdirectly in builders #7984 [arrow] (liamzwbao) - Improve StringArray(Utf8) sort performance (~2-4x faster) #7860 [arrow] (zhuqi-lucas)
Closed issues:
- [Variant] Improve fuzz test for Variant #8199
- [Variant] Improve fuzz test for Variant #8198
VariantArrayBuildertracks starting offsets instead of (offset, len) pairs #8192- Rework
ValueBuilderAPI to work withParentStatefor reliable nested rollbacks #8188 - [Variant] Rename
ValueBufferasValueBuilder#8186 - [Variant] Refactor
ParentStateto track and rollback state on behalf of its owning builder #8182 - [Variant]
ObjectBuildershould detect duplicates at insertion time, not at finish #8180 - [Variant] ObjectBuilder does not reliably check for duplicates #8170
- [Variant] Support
StringViewandLargeStringin ´batch_json_string_to_variant` #8145 [parquet] - [Variant] Rename
batch_json_string_to_variantandbatch_variant_to_json_stringjson_to_variant #8144 [parquet] - [avro] Use
tempfilecrate rather than custom temporary file generator in tests #8143 [arrow] - [Avro] Use
Writeratherdyn Writein Decoder #8142 [arrow] - [Variant] Nested builder rollback is broken #8136
- [Variant] Add support the remaing primitive type(timestamp_nanos/timestampntz_nanos/uuid) for parquet variant #8126
- Meta: Implement missing Arrow 56.0 lint rules - Sequential workflow #8121
- ARROW-012-015: Add linter rules for remaining Arrow 56.0 breaking changes #8120
- ARROW-010 & ARROW-011: Add linter rules for Parquet Statistics and Metadata API removals #8119
- ARROW-009: Add linter rules for IPC Dictionary API removals in Arrow 56.0 #8118
- ARROW-008: Add linter rule for SerializedPageReaderState usize→u64 breaking change #8117
- ARROW-007: Add linter rule for Schema.all_fields() removal in Arrow 56.0 #8116
- [Variant] Implement
ShreddingState::AllNullvariant #8088 [parquet] - [Variant] Support Shredded Objects in
variant_get#8083 [parquet] - [Variant]: Implement
DataType::RunEndEncodedsupport forcast_to_variantkernel #8064 [parquet] - [Variant]: Implement
DataType::Dictionarysupport forcast_to_variantkernel #8062 [parquet] - [Variant]: Implement
DataType::Structsupport forcast_to_variantkernel #8061 [parquet] - [Variant]: Implement
DataType::Decimal32/Decimal64/Decimal128/Decimal256support forcast_to_variantkernel #8059 [parquet] - [Variant]: Implement
DataType::Timestamp(..)support forcast_to_variantkernel #8058 [parquet] - [Variant]: Implement
DataType::Float16support forcast_to_variantkernel #8057 [parquet] - [Variant]: Implement
DataType::Intervalsupport forcast_to_variantkernel #8056 [parquet] - [Variant]: Implement
DataType::Time32/Time64support forcast_to_variantkernel #8055 [parquet] - [Variant]: Implement
DataType::Date32 / DataType::Date64support forcast_to_variantkernel #8054 [parquet] - [Variant]: Implement
DataType::Nullsupport forcast_to_variantkernel #8053 [parquet] - [Variant]: Implement
DataType::Booleansupport forcast_to_variantkernel #8052 [parquet] - [Variant]: Implement
DataType::FixedSizeBinarysupport forcast_to_variantkernel #8051 [parquet] - [Variant]: Implement
DataType::Binary/LargeBinary/BinaryViewsupport forcast_to_variantkernel #8050 [parquet] - [Variant]: Implement
DataType::Utf8/LargeUtf8/Utf8Viewsupport forcast_to_variantkernel #8049 [parquet] - [Variant] Implement
cast_to_variantkernel #8043 [parquet] - [Variant] Support
variant_getkernel for shredded variants #7941 [parquet] - Add test for casting
Decimal128(i128::MINandi128::MAX) tof64with overflow handling #7939 [arrow]
Merged pull requests:
- [Variant] Enhance the variant fuz test to cover time/timestamp/uuid primitive type #8200 (klion26)
- [Variant] VariantArrayBuilder tracks only offsets #8193 (scovich)
- [Variant] Caller provides ParentState to ValueBuilder methods #8189 (scovich)
- [Variant] Rename ValueBuffer as ValueBuilder #8187 (scovich)
- [Variant] ParentState handles finish/rollback for builders #8185 (scovich)
- [Variant]: Implement
DataType::RunEndEncodedsupport forcast_to_variantkernel #8174 (liamzwbao) - [Variant]: Implement
DataType::Dictionarysupport forcast_to_variantkernel #8173 (liamzwbao) - Implement
ArrayBuilderforUnionBuilder#8169 [arrow] (grtlr) - [Variant] Support
LargeStringandStringViewinbatch_json_string_to_variant#8163 (liamzwbao) - [Variant] Rename
batch_json_string_to_variantandbatch_variant_to_json_string#8161 (liamzwbao) - [Variant] Add primitive type timestamp_nanos(with&without timezone) and uuid #8149 (klion26)
- refactor(avro): Use impl Write instead of dyn Write in encoder #8148 [arrow] (Xuanwo)
- chore: Use tempfile to replace hand-written utils functions #8147 [arrow] (Xuanwo)
- feat: support push batch direct to completed and add biggest coalesce batch support #8146 [arrow] (zhuqi-lucas)
- [Variant] Add human-readable impl Debug for Variant #8140 (scovich)
- [Variant] Fix broken metadata builder rollback #8135 (scovich)
- [Variant]: Implement DataType::Interval support for cast_to_variant kernel #8125 (codephage2020)
- Add schema resolution and type promotion support to arrow-avro Decoder #8124 [arrow] (jecsand838)
- Add Initial
arrow-avrowriter implementation with basic type support #8123 [arrow] (jecsand838) - [Variant] Add Variant::Time primitive and cast logic #8114 (klion26)
- [Variant] Support Timestamp to variant for
cast_to_variantkernel #8113 (abacef) - Bump actions/checkout from 4 to 5 #8110 (dependabot[bot])
- [Varaint]: add
DataType::Nullsupport to cast_to_variant #8107 (feniljain) - [Variant] Adding fixed size byte array to variant and test #8106 (abacef)
- [VARIANT] Initial integration tests for variant reads #8104 [parquet] (carpecodeum)
- [Variant]: Implement
DataType::Decimal32/Decimal64/Decimal128/Decimal256support forcast_to_variantkernel #8101 (liamzwbao) - Refactor arrow-avro
Decoderto support partial decoding #8100 [arrow] (jecsand838) - fix: Validate metadata len in IPC reader #8097 [arrow] (JakeDern)
- [parquet] further improve logical type compatibility in ArrowWriter #8095 [parquet] (albertlockett)
- [Varint] Implement ShreddingState::AllNull variant #8093 (codephage2020)
- [Variant] Minor: Add comments to tickets for follow on items #8092 (alamb)
- [VARIANT] Add support for DataType::Struct for cast_to_variant #8090 (carpecodeum)
- [VARIANT] Add support for DataType::Utf8/LargeUtf8/Utf8View for cast_to_variant #8089 (carpecodeum)
- [Variant] Implement
DataType::Booleansupport forcast_to_variantkernel #8085 (sdf-jkl) - [Variant] Implement
DataType::{Date32,Date64}=>Variant::Date#8081 (superserious-dev) - Fix new clippy lints from Rust 1.89 #8078 [parquet] [arrow] [arrow-flight] (alamb)
- Implement ArrowSchema to AvroSchema conversion logic in arrow-avro #8075 [arrow] (jecsand838)
- Implement
DataType::{Binary, LargeBinary, BinaryView}=>Variant::Binary#8074 (superserious-dev) - [Variant] Implement
DataType::Float16=>Variant::Float#8073 (superserious-dev) - create PageIndexPolicy to allow optional indexes #8071 [parquet] (kczimm)
- [Variant] Minor: use From impl to make conversion infallable #8068 [parquet] (alamb)
- Bump actions/download-artifact from 4 to 5 #8066 (dependabot[bot])
- Added arrow-avro schema resolution foundations and type promotion #8047 [arrow] (jecsand838)
- Fix arrow-avro type resolver register bug #8046 [arrow] (yongkyunlee)
- implement
cast_to_variantkernel to cast native types toVariantArray#8044 [parquet] (alamb) - Add arrow-avro
SchemaStoreand fingerprinting #8039 [arrow] (jecsand838) - Add more benchmarks for Parquet thrift decoding #8037 [parquet] (etseidl)
- Support multi-threaded writing of Parquet files with modular encryption #8029 [parquet] (rok)
- Add arrow-avro Decoder Benchmarks #8025 [arrow] (jecsand838)
- feat: add method for sync Parquet reader read bloom filter #8024 [parquet] (mapleFU)
- [Variant] Add
variant_getand ShreddedVariantArray#8021 [parquet] (alamb) - Implement arrow-avro SchemaStore and Fingerprinting To Enable Schema Resolution #8006 [arrow] (jecsand838)
- [Parquet] Add tests for IO/CPU access in parquet reader #7971 [parquet] (alamb)
- Speed up Parquet filter pushdown v4 (Predicate evaluation cache for async_reader) #7850 [parquet] (XiangpengHao)
- Implement cast and other operations on decimal32 and decimal64 #7815 [arrow] (CurtHagenlocher)
-
56.0.001 Aug 2025Release notes
Open source →Breaking changes:
- arrow-schema: Remove dict_id from being required equal for merging #7968 [arrow] (brancz)
- [Parquet] Use
u64forSerializedPageReaderState.offset&remaining_bytes, instead ofusize#7918 [parquet] (JigaoLuo) - Upgrade tonic dependencies to 0.13.0 version (try 2) #7839 [arrow] [arrow-flight] (alamb)
- Remove deprecated Arrow functions #7830 [arrow] [arrow-flight] (etseidl)
- Remove deprecated temporal functions #7813 [arrow] (etseidl)
- Remove functions from parquet crate deprecated in or before 54.0.0 #7811 [parquet] (etseidl)
- GH-7686: [Parquet] Fix int96 min/max stats #7687 [parquet] (rahulketch)
Implemented enhancements:
- [parquet] Relax type restriction to allow writing dictionary/native batches for same column #8004
- Support casting int64 to interval #7988 [arrow]
- [Variant] Add
ListBuilder::with_valuefor convenience #7951 [parquet] - [Variant] Add
ObjectBuilder::with_fieldfor convenience #7949 [parquet] - [Variant] Impl PartialEq for VariantObject #7943 #7948
- [Variant] Offer
simdutf8as an optional dependency when validating metadata #7902 [parquet] [arrow] - [Variant] Avoid collecting offset iterator #7901 [parquet]
- [Variant] Remove superfluous check when validating monotonic offsets #7900 [parquet]
- [Variant] Avoid extra allocation in
ObjectBuilder#7899 [parquet] - [Variant][Compute]
variant_getkernel #7893 [parquet] - [Variant][Compute] Add batch processing for Variant-JSON String conversion #7883 [parquet]
- Support
MapArrayin lexsort #7881 [arrow] - [Variant] Add testing for invalid variants (fuzz testing??) #7842 [parquet]
- [Variant] VariantMetadata, VariantList and VariantObject are too big for Copy #7831 [parquet]
- Allow choosing flate2 backend #7826 [parquet]
- [Variant] Tests for creating "large"
VariantObjectss #7821 [parquet] - [Variant] Tests for creating "large"
VariantLists #7820 [parquet] - [Variant] Support VariantBuilder to write to buffers owned by the caller #7805 [parquet]
- [Variant] Move JSON related functionality to different crate. #7800 [parquet]
- [Variant] Add flag in
ObjectBuilderto control validation behavior on duplicate field write #7777 [parquet] - [Variant] make
serde_jsonan optional dependency ofparquet-variant#7775 [parquet] - [coalesce] Implement specialized
BatchCoalescer::push_batchforPrimitiveArray#7763 [arrow] - Add sort_kernel benchmark for StringViewArray case #7758 [arrow]
- [Variant] Improved API for accessing Variant Objects and lists #7756 [parquet]
- Buildable reproducible release builds #7751
- Allow per-column parquet dictionary page size limit #7723 [parquet]
- [Variant] Test and implement efficient building for "large" Arrays #7699 [parquet]
- [Variant] Improve VariantBuilder when creating field name dictionaries / sorted dictionaries #7698 [parquet]
- [Variant] Add input validation in
VariantBuilder#7697 [parquet] - [Variant] Support Nested Data in
VariantBuilder#7696 [parquet] - Parquet: Incorrect min/max stats for int96 columns #7686 [parquet]
- Add
DictionaryArray::gcmethod #7683 [arrow] - [Variant] Add negative tests for reading invalid primitive variant values #7645 [parquet]
Fixed bugs:
- [Variant] Panic when appending nested objects to VariantBuilder #7907 [parquet]
- Panic when casting large Decimal256 to f64 due to unchecked
unwrap()#7886 [arrow] - Incorrect inlined string view comparison after " Add prefix compare for inlined" #7874 [parquet] [arrow]
- [Variant]
test_json_to_variant_object_very_largetakes over 20s #7872 [parquet] - [Variant] If
ObjectBuilder::finalizeis not called, the resulting Variant object is malformed. #7863 [parquet] - CSV error message has values transposed #7848 [arrow]
- Concating struct arrays with no fields unnecessarily errors #7828 [arrow]
- Clippy CI is failing on main after Rust
1.88upgrade #7796 [parquet] [arrow] [arrow-flight] - [Variant] Field lookup with out of bounds index causes unwanted behavior #7784 [parquet]
- Error verifying
parquet-variantcrate on 55.2.0 withverify-release-candidate.sh#7746 test_to_pyarrowtests fail during release verification #7736 [arrow]- [parquet_derive] Example for ParquetRecordWriter is broken. #7732
- [Variant]
Variant::Objectcan contain two fields with the same field name #7730 [parquet] - [Variant] Panic when appending Object or List to VariantBuilder #7701 [parquet]
- Slicing a single-field dense union array creates an array with incorrect
logical_nullslength #7647 [arrow] - Ensure page encoding statistics are written to Parquet file #7643 [parquet] (etseidl)
Documentation updates:
- Minor: Upate
cast_with_optionsdocs about casting integers --> intervals #8002 [arrow] (alamb) - docs: More docs to
BatchCoalescer#7891 [arrow] (2010YOUY01) - chore: fix a typo in
ExtensionType::supports_data_typedocs #7682 [arrow] (mbrobbel) - [Variant] Add variant docs and examples #7661 [parquet] (alamb)
- Minor: Add version to deprecation notice for
ParquetMetaDataReader::decode_footer#7639 [parquet] (etseidl)
Performance improvements:
RowConverteron list should only encode the sliced list values and not the entire data #7993 [arrow]- [Variant] Avoid extra allocation in list builder #7977 [parquet]
- [Variant] Convert JSON to Variant with fewer copies #7964 [parquet]
- Optimize sort kernels partition_validity method #7936 [arrow]
- Speedup sorting for inline views #7857 [arrow]
- Perf: Investigate and improve parquet writing performance #7822 [parquet] [arrow]
- Perf: optimize sort string_view performance #7790 [arrow]
- Clickbench microbenchmark spends significant time in memcmp for not_empty predicate #7766 [arrow]
- Use prefix first for comparisons, resort to data buffer for remaining data on equal values #7744 [arrow]
- Change use of
inline_valueto inline it to a u128 #7743 [arrow] - Add efficient way to upgrade keys for additional dictionary builders #7654 [arrow]
- Perf: Make sort string view fast(1.5X ~ 3X faster) #7792 [arrow] (zhuqi-lucas)
- Add specialized coalesce path for PrimitiveArrays #7772 [arrow] (alamb)
Closed issues:
- Implement full-range
i256::to_f64to replace current ±∞ saturation for Decimal256 → Float64 #7985 - [Variant]
impl FromIteratorfprVariantPath#7955 validatedandis_fully_validatedflags doesn't need to be part of PartialEq #7952 [parquet]- [Variant] remove VariantMetadata::dictionary_size #7947 [parquet]
- [Variant] Improve
VariantArrayperformance by storing the index of the metadata and value arrays #7920 - [Variant] Converting variant to JSON string seems slow #7869 [parquet]
- [Variant] Present Variant at Iceberg Summit NYC July 10, 2025 #7858
- [Variant] Avoid second copy of field name in MetadataBuilder #7814 [parquet]
- Remove APIs deprecated in or before 54.0.0 #7810 [parquet] [arrow] [arrow-flight]
- [Variant] Make it harder to forget to finish a pending parent i n ObjectBuilder #7798 [parquet]
- [Variant] Remove explicit ObjectBuilder::finish() and ListBuilder::finish and move to
Dropimpl #7780 [parquet] - Reduce repetition in tests for arrow-row/src/run.rs #7692 [arrow]
- [Variant] Add tests for invalid variant values (aka verify invalid inputs) #7681 [parquet]
- [Variant] Introduce structs for Variant::Decimal types #7660 [parquet]
Merged pull requests:
- Add benchmark for converting StringViewArray with mixed short and long strings #8015 [arrow] (ding-young)
- [Variant] impl FromIterator for VariantPath #8011 [parquet] (sdf-jkl)
- Create empty buffer for a buffer specified in the C Data Interface with length zero #8009 [arrow] (viirya)
- bench: add benchmark for converting list and sliced list to row format #8008 [arrow] (rluvaton)
- bench: benchmark interleave structs #8007 [arrow] (rluvaton)
- [Parquet] Allow writing compatible DictionaryArrays to parquet writer #8005 [parquet] (albertlockett)
- doc: remove outdated info from CONTRIBUTING doc in project root dir. #7998 (sonhmai)
- perf: only encode actual list values in
RowConverter(16-26 times faster for small sliced list) #7996 [arrow] (rluvaton) - test: add tests for converting sliced list to row based #7994 [arrow] (rluvaton)
- perf: Improve
interleaveperformance for struct (3-6 times faster) #7991 [arrow] (rluvaton) - [Variant] Avoid extra buffer allocation in ListBuilder #7987 [parquet] (klion26)
- Implement full-range
i256::to_f64to eliminate ±∞ saturation for Decimal256 → Float64 casts #7986 [arrow] (kosiew) - Minor: Restore warning comment on Int96 statistics read #7975 [parquet] (alamb)
- Add additional integration tests to arrow-avro #7974 [arrow] (nathaniel-d-ef)
- Perf: optimize actual_buffer_size to use only data buffer capacity for coalesce #7967 [arrow] (zhuqi-lucas)
- Implement Improved arrow-avro Reader Zero-Byte Record Handling #7966 [arrow] (jecsand838)
- Perf: improve sort via
partition_validityto use fast path for bit map scan (up to 30% faster) #7962 [arrow] (zhuqi-lucas) - [Variant] Revisit VariantMetadata and Object equality #7961 [parquet] (friendlymatthew)
- [Variant] Add ListBuilder::with_value for convenience #7959 [parquet] (codephage2020)
- [Variant] remove VariantMetadata::dictionary_size #7958 [parquet] (codephage2020)
- [Variant] VariantMetadata is allowed to contain the empty string #7956 [parquet] (scovich)
- Add arrow-avro support for Impala Nullability #7954 [arrow] (veronica-m-ef)
- [Test] Add tests for VariantList equality #7953 [parquet] (alamb)
- [Variant] Add ObjectBuilder::with_field for convenience #7950 [parquet] (alamb)
- [Variant] Adding code to store metadata and value references in VariantArray #7945 (abacef)
- [Variant] Add
variant_kernelsbenchmark #7944 (alamb) - [Variant] Impl
PartialEqfor VariantObject #7943 [parquet] (friendlymatthew) - [Variant] Add documentation, tests and cleaner api for Variant::get_path #7942 [parquet] (alamb)
- arrow-ipc: Remove all abilities to preserve dict IDs #7940 [parquet] [arrow] [arrow-flight] (brancz)
- Optimize partition_validity function used in sort kernels #7937 [arrow] (jhorstmann)
- [Variant] Avoid extra allocation in object builder #7935 [parquet] (klion26)
- [Variant] Avoid collecting offset iterator #7934 [parquet] (codephage2020)
- Minor: Support BinaryView and StringView builders in
make_builder#7931 [arrow] (kylebarron) - chore: bump MSRV to 1.84 #7926 [parquet] [arrow] [arrow-flight] (mbrobbel)
- Update bzip2 requirement from 0.4.4 to 0.6.0 #7924 [arrow] (mbrobbel)
- [Variant] Reserve capacity beforehand during large object building #7922 [parquet] (friendlymatthew)
- [Variant] Add
variant_getcompute kernel #7919 [parquet] (Samyak2) - Improve memory usage for
arrow-row -> String/BinaryViewwhen utf8 validation disabled #7917 [arrow] (ding-young) - Restructure compare_greater function used in parquet statistics for better performance #7916 [parquet] (jhorstmann)
- [Variant] Support appending complex variants in
VariantBuilder#7914 [parquet] (friendlymatthew) - [Variant] Add
VariantBuilder::new_with_buffersto write to existing buffers #7912 [parquet] (alamb) - Convert JSON to VariantArray without copying (8 - 32% faster) #7911 [parquet] (alamb)
- [Variant] Use simdutf8 for UTF-8 validation #7908 [parquet] [arrow] (codephage2020)
- [Variant] Avoid superflous validation checks #7906 [parquet] (friendlymatthew)
- Add
VariantArrayandVariantArrayBuilderfor constructing Arrow Arrays of Variants #7905 (alamb) - Update sysinfo requirement from 0.35.0 to 0.36.0 #7904 [parquet] (dependabot[bot])
- Fix current CI failure #7898 [arrow] (viirya)
- Remove redundant is_err checks in Variant tests #7897 [parquet] (viirya)
- [Variant] test: add variant object tests with different sizes #7896 [parquet] (odysa)
- [Variant] Define basic convenience methods for variant pathing #7894 [parquet] (scovich)
- fix:
view_typesbenchmark slice should follow by correct len array #7892 [arrow] (zhuqi-lucas) - Add arrow-avro support for bzip2 and xz compression #7890 [arrow] (jecsand838)
- Add arrow-avro support for Duration type and minor fixes for UUID decoding #7889 [arrow] (jecsand838)
- [Variant] Reduce variant-related struct sizes #7888 [parquet] (scovich)
- Fix panic on lossy decimal to float casting: round to saturation for overflows #7887 [arrow] (kosiew)
- Add tests for invalid variant metadata and value #7885 [parquet] (viirya)
- [Variant] Introduce parquet-variant-compute crate to transform batches of JSON strings to and from Variants #7884 (harshmotw-db)
- feat: support
MapArrayin lexsort #7882 [arrow] (rluvaton) - fix: mark
DataType::Mapas unsupported inRowConverter#7880 [arrow] (rluvaton) - [Variant] Speedup validation #7878 [parquet] (friendlymatthew)
- benchmark: Add StringViewArray gc benchmark with not null cases #7877 [arrow] (zhuqi-lucas)
- [ARROW-RS-7820][Variant] Add tests for large variant lists #7876 [parquet] (klion26)
- fix: Incorrect inlined string view comparison after Add prefix compar… #7875 [arrow] (zhuqi-lucas)
- perf: speed up StringViewArray gc 1.4 ~5.x faster #7873 [arrow] (zhuqi-lucas)
- [Variant] Remove superflous validate call and rename methods #7871 [parquet] (friendlymatthew)
- Benchmark: Add rich testing cases for sort string(utf8) #7867 [arrow] (zhuqi-lucas)
- chore: update link for
row_filter.rs#7866 [parquet] (haohuaijin) - [Variant] List and object builders have no effect until finalized #7865 [parquet] (scovich)
- Added number to string benches for json_writer #7864 [arrow] (abacef)
- [Variant] Introduce
parquet-variant-jsoncrate #7862 [parquet] (alamb) - [Variant] Remove dead code, add comments #7861 [parquet] (alamb)
- Speedup sorting for inline views: 1.4x - 1.7x improvement #7856 [arrow] (Dandandan)
- Fix union slice logical_nulls length #7855 [arrow] (codephage2020)
- Add
get_ref/get_mutto JSON Writer #7854 [arrow] (cetra3) - [Minor] Add Benchmark for RowConverter::append #7853 [arrow] (Dandandan)
- Add Enum type support to arrow-avro and Minor Decimal type fix #7852 [arrow] (jecsand838)
- CSV error message has values transposed #7851 [arrow] (Omega359)
- [Variant] Fuzz testing and benchmarks for vaildation #7849 [parquet] (carpecodeum)
- [Variant] Follow up nits and uncomment test cases #7846 [parquet] (friendlymatthew)
- [Variant] Make sure ObjectBuilder and ListBuilder to be finalized before its parent builder #7843 [parquet] (viirya)
- Add decimal32 and decimal64 support to Parquet, JSON and CSV readers and writers #7841 [parquet] [arrow] (CurtHagenlocher)
- Implement arrow-avro Reader and ReaderBuilder #7834 [arrow] (jecsand838)
- [Variant] Support creating sorted dictionaries #7833 [parquet] (friendlymatthew)
- Add Decimal type support to arrow-avro #7832 [arrow] (jecsand838)
- Allow concating struct arrays with no fields #7829 [arrow] (AdamGS)
- Add features to configure flate2 #7827 [parquet] (zeevm)
- make builder public under experimental #7825 [parquet] (XiangpengHao)
- Improvements for parquet writing performance (25%-44%) #7824 [parquet] [arrow] (jhorstmann)
- Use in-memory buffer for arrow_writer benchmark #7823 [parquet] (jhorstmann)
- [Variant] impl [Try]From for VariantDecimalXX types #7809 [parquet] (scovich)
- [Variant] Speedup
ObjectBuilder(62x faster) #7808 [parquet] (friendlymatthew) - [VARIANT] Support both fallible and infallible access to variants #7807 [parquet] (scovich)
- Minor: fix clippy in parquet-variant after logical conflict #7803 [parquet] (alamb)
- [Variant] Add flag in
ObjectBuilderto control validation behavior on duplicate field write #7801 [parquet] (micoo227) - Fix clippy for Rust 1.88 release #7797 [parquet] [arrow] [arrow-flight] (alamb)
- [Variant] Simplify
Builderbuffer operations #7795 [parquet] (friendlymatthew) - fix: Change panic to error in
takekernel for StringArrary/BinaryArray on overflow #7793 [arrow] (chenkovsky) - Update base64 requirement from 0.21 to 0.22 #7791 [parquet] (dependabot[bot])
- Fix RowConverter when FixedSizeList is not the last #7789 [arrow] (findepi)
- Add schema with only primitive arrays to
coalesce_kernelbenchmark #7788 [arrow] (alamb) - Add sort_kernel benchmark for StringViewArray case #7787 [arrow] (zhuqi-lucas)
- [Variant] Check pending before
VariantObject::insert#7786 [parquet] (friendlymatthew) - [VARIANT] impl Display for VariantDecimalXX #7785 [parquet] [arrow] (scovich)
- [VARIANT] Add support for the json_to_variant API #7783 [parquet] (harshmotw-db)
- [Variant] Consolidate examples for json writing #7782 [parquet] (alamb)
- Add benchmark for about view array slice #7781 [arrow] (ctsk)
- [Variant] Add negative tests for reading invalid primitive variant values #7779 [parquet] (superserious-dev)
- [Variant] Support creating nested objects and object with lists #7778 [parquet] (friendlymatthew)
- [VARIANT] Validate precision in VariantDecimalXX structs and add missing tests #7776 [parquet] (scovich)
- Add tests for
BatchCoalescer::push_batch_with_filter, fix bug #7774 [arrow] (alamb) - [Variant] Minor: make fields in
VariantDecimal*private, add examples #7770 [parquet] (alamb) - Extend the fast path in GenericByteViewArray::is_eq for comparing against empty strings #7767 [arrow] (jhorstmann)
- [Variant] Improve getter API for
VariantListandVariantObject#7757 [parquet] (friendlymatthew) - [Variant] Add Variant::as_object and Variant::as_list #7755 [parquet] (alamb)
- [Variant] Fix several overflow panic risks for 32-bit arch #7752 [parquet] (scovich)
- Add testing section to pull request template #7749 (alamb)
- Perf: Add prefix compare for inlined compare and change use of inline_value to inline it to a u128 #7748 [arrow] (zhuqi-lucas)
- Move arrow-pyarrow tests that require
pyarrowto be installed intoarrow-pyarrow-testingcrate #7742 (alamb) - [Variant] Improve write API in
Variant::Object#7741 [parquet] (friendlymatthew) - [Variant] Support nested lists and object lists #7740 [parquet] (friendlymatthew)
- feat: [Variant] Add Validation for Variant Deciaml #7738 [parquet] (Weijun-H)
- Add fallible versions of temporal functions that may panic #7737 [arrow] (adriangb)
- fix: Implement support for appending Object and List variants in VariantBuilder #7735 [parquet] (Weijun-H)
- parquet_derive: update in working example for ParquetRecordWriter #7733 (LanHikari22)
- Perf: Optimize comparison kernels for inlined views #7731 [arrow] (zhuqi-lucas)
- arrow-row: Refactor arrow-row REE roundtrip tests #7729 [arrow] (brancz)
- arrow-array: Implement PartialEq for RunArray #7727 [arrow] (brancz)
- fix: Do not add null buffer for
NullArrayin MutableArrayData #7726 [arrow] (comphead) - Allow per-column parquet dictionary page size limit #7724 [parquet] (XiangpengHao)
- fix JSON decoder error checking for UTF16 / surrogate parsing panic #7721 [arrow] (nicklan)
- [Variant] Use
BTreeMapforVariantBuilder.dictandObjectBuilder.fieldsto maintain invariants upon entry writes #7720 [parquet] (friendlymatthew) - Introduce
MAX_INLINE_VIEW_LENconstant for string/byte views #7719 [arrow] (alamb) - [Variant] Introduce new type over &str for ShortString #7718 [parquet] (friendlymatthew)
- Split out variant code into several new sub-modules #7717 [parquet] (scovich)
- add
garbage_collect_dictionarytoarrow-select#7716 [arrow] (davidhewitt) - Support write to buffer api for SerializedFileWriter #7714 [parquet] (zhuqi-lucas)
- Support
FixedSizeListRowConverter #7705 [arrow] (findepi) - Make variant iterators safely infallible #7704 [parquet] (scovich)
- Speedup
interleave_views(4-7x faster) #7695 [arrow] (Dandandan) - Define a "arrow-pyrarrow" crate to implement the "pyarrow" feature. #7694 [arrow] (brunal)
- feat: add constructor to efficiently upgrade dict key type to remaining builders #7689 [arrow] (albertlockett)
- Document REE row format and add some more tests #7680 [arrow] (alamb)
- feat: add min max aggregate support for FixedSizeBinary #7675 [arrow] (alexwilcoxson-rel)
- arrow-data: Add REE support for
build_extendandbuild_extend_nulls#7671 [arrow] (brancz) - Variant: Write Variant Values as JSON #7670 [parquet] (carpecodeum)
- Remove
lazy_staticdependency #7669 [arrow] (Expyron) - Finish implementing Variant::Object and Variant::List #7666 [parquet] (scovich)
- Add
RecordBatch::schema_metadata_mutandField::metadata_mut#7664 [arrow] (emilk) - [Variant] Simplify creation of Variants from metadata and value #7663 [parquet] (alamb)
- chore: group prost dependabot updates #7659 (mbrobbel)
- Initial Builder API for Creating Variant Values #7653 [parquet] (PinkCrow007)
- Add
BatchCoalescer::push_filtered_batchand docs #7652 [arrow] (alamb) - Optimize coalesce kernel for StringView (10-50% faster) #7650 [arrow] (alamb)
- arrow-row: Add support for REE #7649 [arrow] (brancz)
- Use approximate comparisons for pow tests #7646 [arrow] (adamreeve)
- [Variant] Implement read support for remaining primitive types #7644 [parquet] (superserious-dev)
- Add
pretty_format_batches_with_schemafunction #7642 [arrow] (lewiszlw) - Deprecate old Parquet page index parsing functions #7640 [parquet] (etseidl)
- Update FlightSQL
GetDbSchemasandGetTablesschemas to fully match the protocol #7638 [arrow] [arrow-flight] (sgrebnov) - Minor: Remove outdated FIXME from
ParquetMetaDataReader#7635 [parquet] (etseidl) - Fix the error info of
StructArray::try_new#7634 [arrow] (xudong963) - Fix reading encrypted Parquet pages when using the page index #7633 [parquet] (adamreeve)
- [Variant] Add commented out primitive test casees #7631 [parquet] (alamb)
-
55.2.026 Jun 2025Release notes
Open source →- Add a
strong_countmethod toBuffer#7568 [arrow] - Create version of LexicographicalComparator that compares fixed number of columns #7531 [arrow]
- parquet-show-bloom-filter should work with integer typed columns #7528 [parquet]
- Allow merging primitive dictionary values in concat and interleave kernels #7518 [arrow]
- Add efficient concatenation of StructArrays #7516 [arrow]
- Rename
flight-sql-experimentaltoflight-sql#7498 [arrow] [arrow-flight] - Consider moving from ryu to lexical-core for string formatting / casting floats to string. #7496
- Arithmetic kernels can be safer and faster #7494 [arrow]
- Speedup
filter_bytesby precalculating capacity #7465 [arrow] - [Variant]: Rust API to Create Variant Values #7424 [parquet] [arrow]
- [Variant] Rust API to Read Variant Values #7423 [arrow]
- Release arrow-rs / parquet Minor version
55.1.0(May 2025) #7393 [parquet] - Support create_random_array for Decimal data types #7343 [arrow]
- Truncate Parquet page data page statistics #7555 [parquet] (etseidl)
Fixed bugs:
- In arrow_json, Decoder::decode can panic if it encounters two high surrogates in a row. #7712
- FlightSQL "GetDbSchemas" and "GetTables" schemas do not fully match the protocol #7637 [arrow] [arrow-flight]
- Cannot read encrypted Parquet file if page index reading is enabled #7629 [parquet]
encoding_statsnot present in Parquet generated byparquet-rewrite#7616 [parquet]- When writing parquet plaintext footer files
footer_signing_key_metadatais not included, encryption alghoritm is always written in footer #7599 [parquet] new_null_arraypanics when constructing a struct of a dictionary #7571- Parquet derive fails to build when Result is aliased #7547
- Unable to read
Dictionary(u8, FixedSizeBinary(_))using datafusion. #7545 [parquet] - filter_record_batch panics with empty struct array. #7538 [arrow]
- Panic in
pretty_formatfunction when displaying DurationSecondsArray withi64::MIN/i64::MAX#7533 [arrow] - Record API unable to parse TIME_MILLIS when encoded as INT32 #7510 [parquet]
- The
read_record_batchfunc of theRecordBatchDecoderdoes not respect theskip_validationproperty #7508 [arrow] arrow-55.1.0breaksfilter_record_batch#7500- Files containing binary data with >=8_388_855 bytes per row written with
arrow-rscan't be read withpyarrow#7489 [parquet] - [Bug] Ingestion with Arrow Flight Sql panic when the input stream is empty or fallible #7329 [arrow] [arrow-flight]
- Ensure page encoding statistics are written to Parquet file #7643 [parquet] (etseidl)
Documentation updates:
- arrow_reader_row_filter benchmark doesn't capture page cache improvements #7460 [parquet] [arrow]
- chore: fix a typo in
ExtensionType::supports_data_typedocs #7682 [arrow] (mbrobbel) - [Variant] Add variant docs and examples #7661 [parquet] (alamb)
- Minor: Add version to deprecation notice for
ParquetMetaDataReader::decode_footer#7639 [parquet] (etseidl) - Add references for defaults in
WriterPropertiesBuilder#7558 [parquet] (etseidl) - Clarify Docs: NullBuffer::len is in bits #7556 [arrow] (alamb)
- docs: fix typo for
Decimal128Array#7525 [arrow] (burmecia) - Minor: Add examples to ProjectionMask documentation #7523 [parquet] (alamb)
- Improve documentation for Parquet
WriterProperties#7491 [parquet] (alamb)
Closed issues:
- [Variant] More efficient determination of String vs ShortString #7700
- [Variant] Improve API for iterating over values of a VariantList #7685 [parquet]
- [Variant] Consider validating variants on creation (rather than read) #7684 [parquet]
- Miri test_native_type_pow test failing #7641 [arrow]
- Improve performance of
coalesceandconcatfor views #7615 [arrow] - Bad min value in row group statistics in some special cases #7593
- Feature Request: BloomFilter Position Flexibility in
parquet-rewrite#7552 [parquet]
Merged pull requests:
- arrow-array: Implement PartialEq for RunArray #7727 [arrow] (brancz)
- fix: Do not add null buffer for
NullArrayin MutableArrayData #7726 [arrow] (comphead) - fix JSON decoder error checking for UTF16 / surrogate parsing panic #7721 [arrow] (nicklan)
- [Variant] Introduce new type over &str for ShortString #7718 [parquet] (friendlymatthew)
- Split out variant code into several new sub-modules #7717 [parquet] (scovich)
- Support write to buffer api for SerializedFileWriter #7714 [parquet] (zhuqi-lucas)
- Make variant iterators safely infallible #7704 [parquet] (scovich)
- Speedup
interleave_views(4-7x faster) #7695 [arrow] (Dandandan) - Define a "arrow-pyrarrow" crate to implement the "pyarrow" feature. #7694 [arrow] (brunal)
- Document REE row format and add some more tests #7680 [arrow] (alamb)
- feat: add min max aggregate support for FixedSizeBinary #7675 [arrow] (alexwilcoxson-rel)
- arrow-data: Add REE support for
build_extendandbuild_extend_nulls#7671 [arrow] (brancz) - Remove
lazy_staticdependency #7669 [arrow] (Expyron) - Finish implementing Variant::Object and Variant::List #7666 [parquet] (scovich)
- Add
RecordBatch::schema_metadata_mutandField::metadata_mut#7664 [arrow] (emilk) - [Variant] Simplify creation of Variants from metadata and value #7663 [parquet] (alamb)
- chore: group prost dependabot updates #7659 (mbrobbel)
- Initial Builder API for Creating Variant Values #7653 [parquet] (PinkCrow007)
- Add
BatchCoalescer::push_filtered_batchand docs #7652 [arrow] (alamb) - Optimize coalesce kernel for StringView (10-50% faster) #7650 [arrow] (alamb)
- arrow-row: Add support for REE #7649 [arrow] (brancz)
- Use approximate comparisons for pow tests #7646 [arrow] (adamreeve)
- [Variant] Implement read support for remaining primitive types #7644 [parquet] (superserious-dev)
- Add
pretty_format_batches_with_schemafunction #7642 [arrow] (lewiszlw) - Deprecate old Parquet page index parsing functions #7640 [parquet] (etseidl)
- Update FlightSQL
GetDbSchemasandGetTablesschemas to fully match the protocol #7638 [arrow] [arrow-flight] (sgrebnov) - Minor: Remove outdated FIXME from
ParquetMetaDataReader#7635 [parquet] (etseidl) - Fix the error info of
StructArray::try_new#7634 [arrow] (xudong963) - Fix reading encrypted Parquet pages when using the page index #7633 [parquet] (adamreeve)
- [Variant] Add commented out primitive test casees #7631 [parquet] (alamb)
- Improve
coalescekernel tests #7626 [arrow] (alamb) - Revert "Revert "Improve
coalesceandconcatperformance for views… #7625 [arrow] (Dandandan) - Revert "Improve
coalesceandconcatperformance for views (#7614)" #7623 [arrow] (Dandandan) - Improve coalesce_kernel benchmark to capture inline vs non inline views #7619 [arrow] (alamb)
- Improve
coalesceandconcatperformance for views #7614 [arrow] (Dandandan) - feat: add constructor to help efficiently upgrade key for GenericBytesDictionaryBuilder #7611 [arrow] (albertlockett)
- feat: support append_nulls on additional builders #7606 [arrow] (albertlockett)
- feat: add AsyncArrowWriter::into_inner #7604 [parquet] (jpopesculian)
- Move variant interop test to Rust integration test #7602 [parquet] (alamb)
- Include footer key metadata when writing encrypted Parquet with a plaintext footer #7600 [parquet] (rok)
- Add
coalescekernel andBatchCoalescerfor statefully combining selected b…atches: #7597 [arrow] (alamb) - Add FixedSizeBinary to
take_kernelbenchmark #7592 [arrow] (alamb) - Fix GenericBinaryArray docstring. #7588 [arrow] (brunal)
- fix: error reading multiple batches of
Dict(_, FixedSizeBinary(_))#7585 [parquet] (albertlockett) - Revert "Minor: remove filter code deprecated in 2023 (#7554)" #7583 [arrow] (alamb)
- Fixed a warning build build: function never used. #7577 [parquet] (JigaoLuo)
- Adding Encoding argument in
parquet-rewrite#7576 [parquet] (JigaoLuo) - feat: add
row_group_is_[max/min]_value_exactto StatisticsConverter #7574 [parquet] (CookiePieWw) - [array] Remove unwrap checks from GenericByteArray::value_unchecked #7573 [arrow] (ctsk)
- [benches/row_format] fix typo in array lengths #7572 [arrow] (ctsk)
- Add a strong_count method to Buffer #7569 [arrow] (westonpace)
- Minor: Enable byte view for clickbench benchmark #7565 [parquet] (zhuqi-lucas)
- Optimize length calculation in row encoding for fixed-length columns #7564 [arrow] (ctsk)
- Use PR title and description for commit message #7563 (kou)
- Use apache/arrow-{go,java,js} in integration test #7561 (kou)
- Implement Array Decoding in arrow-avro #7559 [arrow] (jecsand838)
- Minor: remove filter code deprecated in 2023 #7554 [arrow] (alamb)
- fix: Correct docs for
WriterPropertiesBuilder::set_column_index_truncate_length#7553 [parquet] (etseidl) - Adding Bloom Filter Position argument in parquet-rewrite #7550 [parquet] (JigaoLuo)
- Fix
Resultname collision in parquet_derive #7548 (jspaezp) - Fix: Converted feature flight-sql-experimental to flight-sql #7546 [arrow] [arrow-flight] (kunalsinghdadhwal)
- Fix CI on main due to logical conflict #7542 [arrow] (alamb)
- Fix
filter_record_batchpanics with empty struct array #7539 [arrow] (thorfour) - [Variant] Initial API for reading Variant data and metadata #7535 (mkarbo)
- fix: Panic in pretty_format function when displaying DurationSecondsA… #7534 [arrow] (zhuqi-lucas)
- Create version of LexicographicalComparator that compares fixed number of columns (~ -15%) #7530 [arrow] (Dandandan)
- Make parquet-show-bloom-filter work with integer typed columns #7529 [parquet] (adamreeve)
- chore(deps): update criterion requirement from 0.5 to 0.6 #7527 [parquet] [arrow] (mbrobbel)
- Minor: Add a parquet row_filter test, reduce some test boiler plate #7522 [parquet] (alamb)
- Refactor
build_array_readerinto a struct #7521 [parquet] (alamb) - arrow: add concat structs benchmark #7520 [arrow] (asubiotto)
- arrow-select: add support for merging primitive dictionary values #7519 [arrow] (asubiotto)
- arrow-select: add support for optimized concatenation of struct arrays #7517 [arrow] (asubiotto)
- Fix Clippy in CI for Rust 1.87 release #7514 [parquet] [arrow] [arrow-flight] (alamb)
- Simplify
ParquetRecordBatchReader::nextcontrol logic #7512 [parquet] (alamb) - Fix record API support for reading INT32 encoded TIME_MILLIS #7511 [parquet] (njaremko)
- RecordBatchDecoder: skip RecordBatch validation when
skip_validationproperty is enabled #7509 [arrow] (nilskch) - Introduce
ReadPlanto encapsulate the calculation of what parquet rows to decode #7502 [parquet] (alamb) - Update documentation for ParquetReader #7501 [parquet] (alamb)
- Improve
Fielddocs, add missingField::set_*methods #7497 [arrow] (alamb) - Speed up arithmetic kernels, reduce
unsafeusage #7493 [arrow] (Dandandan) - Prevent FlightSQL server panics for
do_putwhen stream is empty or 1st stream element is an Err #7492 [arrow] [arrow-flight] (superserious-dev) - arrow-ipc: add
StreamDecoder::schema#7488 [arrow] (lidavidm) - arrow-select: Implement concat for
RunArrays #7487 [arrow] (brancz) - [Variant] Add (empty)
parquet-variantcrate, updateparquet-testingpin #7485 (alamb) - Improve error messages if schema hint mismatches with parquet schema #7481 [parquet] [arrow] (alamb)
- Add
arrow_reader_clickbenchbenchmark #7470 [parquet] (alamb) - Speedup
filter_bytes~-20-40%,filter_nativelow selectivity (~-37%) #7463 [arrow] (Dandandan)
- Add a