NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
crates.io, #3061 most downloaded on crates.io
Apache Arrow Flight
Latest stable
60.1.0
Last release 20 days ago
28 Oct 2026
Ships fairly regularly
a new release about every 4 weeks
Nearly every release is documented
notes for 59 of the last 60 stable releases
Nothing withdrawn
no release was ever pulled
7 years old
113 releases · first in 2020
Drag across the ruler to compare, or tap a release to open it.
<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this wor
<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -->
Implemented enhancements:
DatePart::from_str API #9930 [arrow]Fixed bugs:
DeltaBitPackDecoder::skip could panic on "non-standard" miniblocks #9793 [parquet]Documentation updates:
Closed issues:
Merged pull requests:
FromStr for DatePart #9931 [arrow] (sdf-jkl)value_length name in FixedSizeBinaryArray #9905 [arrow] (alamb)FixedSizeBinaryArray i32 offset overflows (try 2) #9872 [arrow] (alamb)like::eq_ascii_ignore_case kernel #9871 [arrow] (albertlockett)UNKNOWN logical type annotation on any physical type #9855 [parquet] (etseidl)InMemoryArrayReader with PrimitiveArrayReader in tests #9847 [parquet] (HippoBaro)ArrayData::slice length overflow #9813 [arrow] (alamb)* This Changelog was automatically generated by github_changelog_generator
Your coding agent can read these notes before it upgrades. Set up the MCP server →
<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this wor
<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -->
Implemented enhancements:
RowFilter when with_limit(N) is set #9765 [parquet]RleEncoder::flush_bit_packed_run #9734 [parquet]FixedSizeList in arrow-json reader #9714 [arrow]VariantArrayBuilder::append_nulls API #9684variant_get(..., List<_>) non-Struct types support #9615Struct fast-path for variant_get(..., Struct) #9596variant_get to cast kernel for numeric/bool types #9564 [arrow]arrow_string::Predicate and its methods? #9480Fixed bugs:
unshred_variant panics on malformed bytes despite returning Result #9740DictEncoder::estimated_memory_size #9719 [parquet]Documentation updates:
Performance improvements:
GenericListViewArray::new_unchecked and refactor ListView json decoder #9646 [arrow]partition function #9640 [arrow]LIMIT / OFFSET into the last RowFilter predicate and skip unused row groups #9766 [parquet] (haohuaijin)write_list #9752 [parquet] (HippoBaro)len field from buffer builder #9750 [arrow] (cetra3)offset_index_disabled at build-time #9724 [parquet] (HippoBaro)partition and take in RunEndEncoded decoder #9658 [arrow] (liamzwbao)ArrayData with typed Array construction in json-reader #9497 [arrow] (liamzwbao)Closed issues:
skip_field #9828 [arrow]skip_field with explicit DataType handling #9821 [arrow]typed_value is extracted #9701take benchmarks #9627 [arrow]Merged pull requests:
MutableBuffer::extend_zeros #9820 [arrow] (alamb)BooleanBufferBuilder with NullBufferBuilder in arrow-json if applicable #9811 [arrow] (liamzwbao)pub #9772 [arrow] (ashdnazg)8 literals with named constants #9751 [parquet] (HippoBaro)RleEncoder::flush_bit_packed_run to make flow clearer #9735 [parquet] (etseidl)Decoder::flush_block in async reader #9726 [arrow] (mzabaluev)DictEncoder::estimated_memory_size, Interner::estimated_memory_size #9720 [parquet] (mzabaluev)FixedSizeList in json decoder #9715 [arrow] (liamzwbao)has_non_empty_nulls helper function in OffsetBuffer #9711 [arrow] (rluvaton)push_decoder benchmark for PushBuffers overhead #9696 [parquet] (HippoBaro)BooleanArray and NullBuffer::union_many #9692 [arrow] (mbutrovich)VariantArrayBuilder::append_nulls API #9685 (sdf-jkl)Field has a valid ExtensionType #9677 [parquet] [arrow] (sdf-jkl)variant_get should follow JSONPath semantics for Field path element #9676 (sdf-jkl)MAX_INLINE_VIEW_LEN from arrow_data #9665 [arrow] (rluvaton)GenericListViewArray::new_unchecked and refactor ListView json decoder #9648 [arrow] (liamzwbao)variant_get to return List<T> instead of List<Struct> #9631 (liamzwbao)MutableBuffer::clear #9622 [parquet] [arrow] (Rafferty97)object_store breakage for 0.13.2 #9612 (mzabaluev-flarion)PartialEq and Eq for CdcOptions #9602 [parquet] (kszucs)finish_preserve_values to ArrayBuilder trait #9601 [arrow] (adamreichold)Struct fast-path for variant_get(..., Struct) #9597 (sdf-jkl)UnionArray #9544 [arrow] (friendlymatthew)ListView codec in arrow-json #9503 [arrow] (liamzwbao)* This Changelog was automatically generated by github_changelog_generator
Deprecate ArrowTimestampType::make\_value in favor of from\_naive\_datetime \#9490 [arrow]
<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -->
Implemented enhancements:
variant_to_arrow Struct type support #9529unshred_variant support for Binary and LargeBinary types #9526shred_variant support for LargeUtf8 and LargeBinary types #9525variant_get tests clean up #9517unshred_variant #9513unshred_variant #9512append_nulls to MapBuilder #9431 [arrow]append_non_nulls to StructBuilder #9429 [arrow]append_value_n to GenericByteBuilder #9425 [arrow]from_bitwise_binary_op #9378 [arrow]Fixed bugs:
util::bit_mask::set_bits (soundness issue) #9543 [arrow]RowSelection #9370 [parquet]Documentation updates:
Performance improvements:
NullBuffer::try_from_unsliced to simplify array construction #9385 [parquet] [arrow]Closed issues:
partially_shredded_variant_array_gen #9492LargeList / ListView / LargeListView for VariantArray::try_new #9455Merged pull requests:
variant_to_arrow Struct type support #9572 (sdf-jkl)checked_add for bounds checks to avoid UB #9568 [arrow] (etseidl)ValueIter into own module, and add public record_count function #9557 [arrow] (Rafferty97)shred_variant support for LargeUtf8 and LargeBinary #9554 (sdf-jkl)HeaderInfo to expose OCF header #9548 [arrow] (mzabaluev)main branch with required reviews #9547 (comphead)infer_json_schema #9546 [arrow] (Rafferty97)take_fixed_size_binary For Predefined Value Lengths #9535 [arrow] (tobixdev)variant_get tests #9518 (sdf-jkl)partially_shredded_variant_array_gen #9498 (codephage2020)prettyprint tests in arrow-cast to insta inline snapshots #9472 [parquet] [arrow] (grtlr)ListLikeArray to arrow-array to be shared with json writer and parquet unshredding #9437 [arrow] (liamzwbao)claim method to recordbatch for memory accounting #9433 [arrow] (cetra3)append_nulls to MapBuilder #9432 [arrow] (Fokko)append_non_nulls to StructBuilder #9430 [arrow] (Fokko)append_value_n to GenericByteBuilder #9426 [arrow] (Fokko)NullBuffer::from_unsliced_buffer helper and refactor call sites #9411 [parquet] [arrow] (Eyad3skr)RunArray::new_unchecked and RunArray::into_parts #9376 [arrow] (rluvaton)* This Changelog was automatically generated by github_changelog_generator
Deprecate ArrowReaderOptions::with_page_index and update API \#9199 [parquet] (friendlymatthew)
<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -->
Breaking changes:
from_bitwise_unary_op #9297 [arrow] (Dandandan)BufferBuilder::new_from_buffer as unsafe #9292 [arrow] (Jefffrey)['fieldName'] in VariantPath parser #9276 (klion26)PageEncodingStats to bitmask #9051 [parquet] (WaterWhisperer)Implemented enhancements:
Vec in StructBuilder #9427from_bitwise_unary_op #9364RunEndEncoded in ord comparator #9360RunEndEncoded arrays in arrow-json #9359BinaryView in bit_length kernel #9351List types in bit_length kernel #9349ListView in parquet arrow writer #9344ListView in length kernel #9343ListView in sort kernel #9341DateTime #9337DataType::is_list and DataType::IsBinary #9326avro_custom_types round-trip + non-custom fallbacks #9290Fixed bugs:
Documentation updates:
Array trait #9314 [arrow] (alamb){Null,Boolean,}BufferBuilder #9155 [arrow] (alamb)ArrowReaderBuilder::with_row_filter documentation #9153 [parquet] (alamb)make_array #9112 [arrow] (alamb)Performance improvements:
ArrayData with direct Array construction #9338 [arrow] (liamzwbao)unsafe and allocations when creating PrimitiveArrays from Vec and from_trusted_len_iter #9299 [arrow] (alamb)BooleanArray from ArrayData #9159 [arrow] (alamb)ArrayData (1% improvement) #9120 [parquet] [arrow] (alamb)make_array for StructArray and GenericByteViewArray #9114 [arrow] (alamb)and and or), add BooleanBuffer::from_bitwise_binary_op #9090 [arrow] (alamb)GenericByteArray in row conversion #9078 [arrow] (rluvaton)Closed issues:
57.3.0 (January 2026) #9240.. and ['fieldName'] syntax in the VariantPath parser #9050Merged pull requests:
Vec in StructBuilder #9428 [arrow] (Fokko)ArrowArrayStreamReader for 0-columns record batch streams #9405 [arrow] (jonded94)ListView not being fully supported #9356 [arrow] (Jefffrey)DataType::is_list and DataType::is_binary #9327 [arrow] (AdamGS)PrimitiveArray::from_iter perf #9294 [arrow] (Dandandan)BufferBuilder::truncate doesn't overset length #9288 [arrow] (Jefffrey)Array as unsafe #9234 [arrow] (gabotechs)FilterBuilder::Optimize/BitIndexIterator/iter_set_bits_rev #9229 [parquet] [arrow] (Dandandan)RowGroupIndexReader for single row group reads #9226 [parquet] (friendlymatthew)BatchCoalescer push_batch_with_filter #9218 [arrow] (alamb)ArrowReaderOptions::with_page_index and update API #9199 [parquet] (friendlymatthew)ListArray from ArrayData #9194 [arrow] (alamb)ListViewArray from ArrayData #9193 [arrow] (alamb)MapArray from ArrayData #9192 [arrow] (alamb)NullArray from ArrayData #9191 [arrow] (alamb)PrimitiveArray from ArrayData #9190 [arrow] (alamb)RunEndArray from ArrayData #9189 [arrow] (alamb)UnionArray from ArrayData #9188 [arrow] (alamb)FixedSizeListArray from ArrayData #9187 [arrow] (alamb)FixedSizeBinaryArray from ArrayData #9186 [arrow] (alamb)DictionaryArray from ArrayData #9185 [arrow] (alamb)#[inline] to BitIterator next function #9177 [arrow] (rluvaton)arrow-row and arrow-ord #9176 [arrow] (brancz)arrow-avro Writer #9171 [arrow] (jecsand838)RecordBatchStream trait #9166 [arrow] (lidavidm)GenericByteViewArray from ArrayData #9156 [arrow] (alamb)reserve to Rows #9142 [arrow] (rluvaton)BooleanBufferBuilder::extend_trusted_len #9137 [arrow] (Dandandan)Null to list view/run encoded/union types #9134 [arrow] (Jefffrey)PrimitiveArrays directly rather than via ArrayData #9122 [parquet] (alamb)ArrayData #9121 [parquet] (alamb)bincode to maintained postcard crate (RUSTSEC-2025-0141 ) #9104 [arrow] (alamb)variant_get #9049 (liamzwbao)0 secs for empty DayTime/MonthDayNano intervals #9023 [arrow] (Jefffrey)Statistics and SizeStatistics in Parquet metadata #9008 [parquet] (etseidl)* This Changelog was automatically generated by github_changelog_generator
[57_maintenance] fix: implement ArrayData::validate_values for union types by @Jefffrey in #11814
ArrayData::validate_values for union types by @Jefffrey in #11814GenericByteArray::into_builder by @Jefffrey in #11816UnionFields when deserializing by @Jefffrey in #11813PrimitiveArray::into_builder by @Jefffrey in #11807ArrayData equality by @Jefffrey in #11811Buffer::into_mutable return error instead of panic for converting owned sliced when not start at 0 and fix returned Mutable length by @Jefffrey in #11806PrimitiveRunBuilder was buggy after finish by @Jefffrey in #11804PrimitiveDictionaryBuilder discard type params by @Jefffrey in #11805UnionFields::from_iter should validate input by @alamb in #11726<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this wor
<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -->
Fixed bugs:
Merged pull requests:
<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this wor
<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -->
Breaking changes:
Array as unsafe #9313 (alamb, gabotechs)BufferBuilder::new_from_buffer as unsafe #9312 (alamb, Jefffrey)Fixed bugs:
BufferBuilder::truncate doesn't overset length #9311 (alamb, Jefffrey)<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this wor
<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -->
Breaking changes:
Implemented enhancements:
LevelInfoBuilder::types_compatible for ArrowWriter #9098append_n to bytes view builder API #9034 [arrow]RunArray::get_physical_indices to RunEndBuffer #9025 [arrow]BinaryArrayType for &FixedSizeBinaryArrays #8992 [arrow]Arc<str> in ParquetRecordWriter derive macro #8972ignoreLeadingWhiteSpace and ignoreTrailingWhiteSpace options to the csv writer #8961 [arrow]interleave for List/LargeList #8952 [arrow]VariantPath #8946DataType::ListView and DataType::LargeListView in ArrayData::new_null #8908 [arrow]GenericListViewArray::from_iter_primitive #8906 [arrow]cmp kernel #8837 [arrow]List/LargeList/ListView/LargeListView #8830Union data types for row format #8828 [arrow]FixedSizeBinaryBuilder supports append_array #8750 [arrow]zip with scalar for Utf8View #8724 [arrow]GEOMETRY and GEOGRAPHY #8717 [parquet]Fixed bugs:
try_binary when Array is implemented incorrectly in external crate #9106Dict(_, LargeUtf8) to Utf8View (StringViewArray) panics #9101nullif kernel #9085 [parquet] [arrow]take_fixed_size_binary Does Not Consider NULL Indices #8947 [arrow]FixedSizeBinaryArray::try_new(...) Panics with Item Length of Zero #8926 [arrow]cargo test -p arrow-cast fails on main #8910 [arrow]GenericListViewArray::new_null ignores len and returns an empty array #8904 [arrow]FixedSizeBinaryArray::new_null Does Not Properly Set the Length of the Values Buffer #8900 [arrow]Documentation updates:
Performance improvements:
GenericByteArray by 8% #9054 [arrow] (rluvaton)not kernel by 50%, add BooleanBuffer::from_bitwise_unary #8996 [arrow] (alamb)Closed issues:
Merged pull requests:
DataType::is_decimal #9100 (AdamGS)ArrayVariantToArrowRowBuilder to variant_to_arrow #9094 (liamzwbao)nullif kernel #9087 [arrow] (alamb)FlightInfo::with_endpoints method #9075 [arrow] [arrow-flight] (lewiszlw)Decimal128 and Decimal256 when field is non-nullable and have non-zero null_density #9046 [arrow] (rluvaton)Rows size should use capacity and not len #9044 [arrow] (rluvaton)try_append_value_n() function to GenericByteViewBuilder #9040 [arrow] (lyang24)boolean_kernels benchmark #9035 [arrow] (alamb)RunArray documentation #9019 [arrow] (Jefffrey)StructArray #9007 [arrow] (brancz)BinaryArrayType for &FixedSizeBinaryArrays #8993 [arrow] (Jefffrey)StringView #8990 [arrow] (lichuang)interleave_kernels benchmark #8980 [arrow] (alamb)LargeList containing sliced StringViews #8979 [arrow] (fabianmurariu)xz crate dependency to use liblzma in arrow-avro #8975 [arrow] (jecsand838)Arc<str> in ParquetRecordWriter derive macro #8973 (heilhead)Time32 to Int64 #8971 [arrow] (tshauck)UnionFields elements by index #8959 [arrow] (friendlymatthew)ArrowArrayStreamReader handle schema with attached metadata + do schema checking #8944 [arrow] (jonded94)ShreddedSchemaBuilder) #8940 (XiangpengHao)test_variant_get_error_when_cast_failure... tests to uses a valid VariantArray #8921 (alamb)FixedSizeBinaryArray::new_null #8914 [arrow] (tobixdev)ArrayData::new_null for ListView / LargeListView #8909 [arrow] (dqkqd)GenericListViewArray::from_iter_primitive #8907 [arrow] (dqkqd)GenericListViewArray::new_null returns empty array #8905 [arrow] (dqkqd)UnionFields #8895 [arrow] (friendlymatthew)variant_get performance on a perfect shredding #8887 (XiangpengHao)Union types in RowConverter #8839 [arrow] (friendlymatthew)List/LargeList/ListView/LargeListView #8831 (liamzwbao)Vec<RecordBatch> wrapper for pyarrow.Table convenience #8790 (jonded94)* This Changelog was automatically generated by github_changelog_generator
\[Parquet\] Avoid copying LogicalType in ColumnOrder::get_sort_order, deprecate get_logical_type \#8789 [parquet] (alamb)
<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -->
Implemented enhancements:
unsafe #8848 [arrow]shred_variant #8795 [arrow]FilterBuilder::optimize or not #8781 [arrow]arrow-json supports encoding binary arrays, but not decoding #8736 [arrow]FilterPredicate instances to be reused for RecordBatches #8692 [arrow]ParquetRecordBatchStream (async API) in terms of the PushDecoder #8677 [parquet]FixedSizeBinary(16) shredding #8665parse_data_type for List, ListView, LargeList, LargeListView, FixedSizeList, Union, Map, RunEndCoded. #8648 [arrow]RleDecoder::reset rather than panic #8632 [parquet]BooleanBufferBuilder and MutableBuffer that mutate directly the buffer #8618 [arrow]Fixed bugs:
List(Int64) results in nullable list in 57.0.0 and a non-nullable list in 57.1.0 #8883ThriftMetadataWriter::write_column_indexes cannot handle a ColumnIndexMetaData::NONE #8815 [parquet]error: item_reader def levels are None when reading nested field with row filter #8657 [parquet]FixedSizeListArrays are not handled #8623 [arrow]Documentation updates:
MutableBuffer from Buffer #8853 [arrow] (alamb)PrimitiveArray to a Vec #8771 [arrow] (alamb)arrow_integration_test crate #8738 [arrow] (phil-opp)Performance improvements:
RowConverter::from_binary should opportunistically take ownership of the buffer #8685 [arrow]collect_bool and remove unsafe, optimize take_bits, take_native for null values #8849 [arrow] (Dandandan)BooleanBuffer::append_packed_range to use apply_bitwise_binary_op #8812 [arrow] (alamb)LogicalType in ColumnOrder::get_sort_order, deprecate get_logical_type #8789 [parquet] (alamb)ArrayIter default impl for nth, nth_back, last and count #8785 [arrow] (rluvaton)SerializedPageReader #8745 [parquet] (XiangpengHao)count, nth, nth_back, last and max for BitIterator #8696 [arrow] (rluvaton)FilterPredicate::filter_record_batch #8693 [arrow] (pepijnve)RowConverter::from_binary #8686 [arrow] (mzabaluev)apply_unary_op and apply_binary_op bitwise operations #8619 [arrow] (rluvaton)Closed issues:
Field #8784arrow-integration-test crate is linked from arrow docs #8739 [arrow]VariantArray::value to return a Result<Variant> #8672Merged pull requests:
List(non-null Int64) instead of List(nullable Int64) #8890 [parquet] [arrow] (etseidl)ArrowReaderOptions::with_virtual_columns error rather than panic on invalid input #8867 [parquet] (alamb)RowNumberReader when not all row groups are selected #8863 [parquet] (vustef)ArrayFormatters when Pretty-Printing Record Batches #8829 [arrow] (tobixdev)ListArray documentation #8803 [arrow] (liamzwbao)ArrowReaderOptions and ArrowReaderMetadata #8798 [parquet] (alamb)shred_variant #8796 (liamzwbao)VariantPath::is_empty #8791 (friendlymatthew)ArrayIter benchmarks #8774 [arrow] (rluvaton)DataType::{Binary/LargeBinary/BinaryView} #8768 [arrow] (klion26)DataType::Union, DataType::Map, DataType::RunEndEncoded #8765 [arrow] (dqkqd)test_cache_projection_excludes_nested_columns to use high level APIs #8754 [parquet] (alamb)merge and merge_n kernels #8753 [arrow] (pepijnve)arrow-integration-test crate from main arrow crate #8740 [arrow] (phil-opp)RleDecoder::reload rather than panic #8729 [parquet] (liamzwbao)ArrayIter does not report size hint correctly after advancing from the iterator back #8728 [arrow] (rluvaton)zip now treats nulls as false in provided mask regardless of the underlying bit value #8711 [arrow] (rluvaton)arrow-avro README.md version to 57 #8695 [arrow] (jecsand838)thrift_enum macro for ConvertedType #8680 [parquet] (etseidl)OffsetBuffer::from_lengths(std::iter::repeat_n(<val>, <repeat>)); with OffsetBuffer::from_repeated_length(<val>, <repeat>); #8669 [arrow] (rluvaton)shred_variant for Uuids #8666 (friendlymatthew)create_test_variant_array helper method #8664 (friendlymatthew)new_repeated to ByteArray #8659 [arrow] (rluvaton)repeat_slice_n_times to MutableBuffer #8658 [arrow] (rluvaton)rescale_decimal followup #8655 [arrow] (liamzwbao)List, ListView, LargeList, LargeListView, FixedSizeList #8649 [arrow] (dqkqd)try_new_with_length constructor to FixedSizeList #8624 [arrow] (connortsui20)variant_to_arrow for utf8 #8600 [arrow] (sdf-jkl)* This Changelog was automatically generated by github_changelog_generator
Undeprecate ArrowWriter::into_serialized_writer and add docs \#8621 [parquet] (alamb)
<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -->
Breaking changes:
Arc<FileEncryptionProperties> everywhere to be be consistent with FileDecryptionProperties #8626 [parquet] (alamb)RunEndEncoded #8596 [arrow] (Weijun-H)ArrowError::AvroError, remaining types and roundtrip tests to arrow-avro, #8595 [arrow] (jecsand838)Map display formatting in DataType #8570 [arrow] (Weijun-H)ListView and LargeListView variants #8569 [arrow] (Weijun-H)Arc<FileDecryptionProperties> to reduce size of ParquetMetadata and avoid copying when encryption is enabled #8470 [parquet] (alamb)DataType::Struct field names in Display formatting #8291 [parquet] [arrow] (emilk)Display for DataType and Field #8290 [parquet] [arrow] (emilk)Implemented enhancements:
arrow-avro crate) #4886VariantArray iterable #8612PartialEq for VariantArray #8610VariantArray #8609Variant to VariantArray #8606RecordBatch::project #8591 [arrow]convert_row_groups #8517 [parquet]thrift-remodel branch with DataFusion #8513 [parquet]UnionArray::is_dense Method Public #8503 [arrow]append_n method to FixedSizeBinaryDictionaryBuilder #8497 [arrow]serde dependency to serde_core where applicable #8451 [arrow]ParquetMetadataReader into IO/decoder state machine and thrift parsing #8439 [parquet]arrow_json lacks with_timestamp_format functions like arrow_csv had offered #8398 [arrow]shred_variant function #8361List #8337 [parquet]Struct #8336 [parquet]Time64(Microsecond) #8334 [parquet]Decimal128 #8332 [parquet]Timestamp(Microsecond, _) and Timestamp(Nanosecond, _) #8331 [parquet]Date32 #8330 [parquet]DataType::FixedSizeList support for cast_to_variant kernel #8281Fixed bugs:
max_predicate_cache_sizeis 0 #8542 [parquet]arrow::compute::interleave fails with struct arrays with no fields #8533 [arrow]ulimit #8406 [parquet]VariantArray::data_type returns StructType, causing Array::as_struct to panic #8319 [parquet]Documentation updates:
Performance improvements:
zip kernel benchmarks #8654 [arrow] (rluvaton)GenericByteBuilder::append_array to use SIMD for extending the offsets #8388 [arrow] (rluvaton)Closed issues:
metadata field should be marked is non-nullable #8410 [parquet]parquet-geospatial crate #8377parquet-geometry crate #8374Merged pull requests:
AvroBinaryFormat to AvroSoeFormat #8633 [arrow] (jecsand838)PartialEq and FromIterator<Option<..>> for VariantArray #8627 (friendlymatthew)ArrowWriter::into_serialized_writer and add docs #8621 [parquet] (alamb)BitChunks::new #8620 [arrow] (rluvaton)get_type_name #8617 (liamzwbao)parquet::format module #8615 [parquet] (etseidl)VariantArray iterable #8613 (friendlymatthew)Extend for VariantArrayBuilder #8611 (friendlymatthew)thrift module #8599 [parquet] (etseidl)parquet::format in metadata bench code #8598 [parquet] (lichuang)extension module #8597 [arrow] (mbrobbel)try_append_value implementation to ByteViewBuilder #8594 [arrow] (samueleresca)FOOTER_SIZE instead of hard code number #8585 [parquet] (lichuang)arrow and workspace to Rust 2024 #8578 [parquet] [arrow] [arrow-flight] (mbrobbel)Dict<_, ByteArray>> column #8573 [parquet] (albertlockett)interleave_struct to handle empty fields #8563 [arrow] (Weijun-H)serde_core when possible #8558 [arrow] (AdamGS)Decimal32/64/128/256 #8552 [arrow] (liamzwbao)arrow-avro to Rust 2024 #8545 [arrow] (mbrobbel)is_dense public #8544 [arrow] (Weijun-H)parquet-variant-json to Rust 2024 #8512 (mbrobbel)parquet-variant-compute to Rust 2024 #8511 (mbrobbel)parquet-variant to Rust 2024 #8510 (mbrobbel)parquet-geospatial to Rust 2024 #8509 (mbrobbel)parquet_derive_test to Rust 2024 #8508 (mbrobbel)parquet_derive to Rust 2024 #8507 (mbrobbel)parquet to Rust 2024 #8506 [parquet] (mbrobbel)append_n method to FixedSizeBinaryDictionaryBuilder #8498 [arrow] (albertlockett)doc_cfg instead of removed doc_auto_cfg #8494 [parquet] [arrow] [arrow-flight] (mbrobbel)arrow-string to Rust 2024 #8491 [arrow] (mbrobbel)arrow-select to Rust 2024 #8490 [arrow] (mbrobbel)arrow-schema to Rust 2024 #8489 [arrow] (mbrobbel)arrow-row to Rust 2024 #8488 [arrow] (mbrobbel)arrow-pyarrow-testing to Rust 2024 #8487 (mbrobbel)arrow-pyarrow-integration-testing to Rust 2024 #8486 (mbrobbel)arrow-pyarrow to Rust 2024 #8485 (mbrobbel)arrow-ord to Rust 2024 #8484 [arrow] (mbrobbel)Field::try_extension_type #8475 [arrow] (alamb)num dependency #8459 [parquet] [arrow] (crepererum)arrow-json to Rust 2024 #8458 [arrow] (mbrobbel)arrow-ipc to Rust 2024 #8457 [arrow] (mbrobbel)arrow-flight to Rust 2024 #8456 [arrow] [arrow-flight] (mbrobbel)arrow-data to Rust 2024 #8455 [arrow] (mbrobbel)arrow-csv to Rust 2024 #8454 [arrow] (mbrobbel)arrow-cast to Rust 2024 #8453 [arrow] (mbrobbel)arrow-buffer to Rust 2024 #8452 [arrow] (mbrobbel)arrow-array to Rust 2024 #8450 [arrow] (mbrobbel)arrow-arith to Rust 2024 #8449 [arrow] (mbrobbel)fields in StructBuilder #8448 [arrow] (lewiszlw)UnionArray wording to 'non-negative' #8434 [arrow] (jdockerty)CastOptions.safe when casting BinaryView → Utf8View (return null for invalid UTF‑8) #8415 [arrow] (kosiew)VariantArray to parquet with Variant LogicalType #8408 [parquet] (alamb)new with try_new for Array types #8397 [arrow] (Jefffrey)Array impl for VariantArray and ShreddedVariantFieldArray #8392 [parquet] (alamb)parquet-geospatial crate #8390 (kylebarron)parquet-geospatial crate #8375 (kylebarron)DataType::FixedSizeList support for cast_to_variant kernel #8282 (liamzwbao)* This Changelog was automatically generated by github_changelog_generator
<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this wor
<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -->
Fixed bugs:
Merged pull requests:
* This Changelog was automatically generated by github_changelog_generator
<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this wor
<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -->
Implemented enhancements:
FixedSizeBinary #8335Utf8 and BinaryView #8333Boolean #8329cast_to_variant #8310Vairant::as_u* #8283cast_to_variant #8234DataType::Union support for cast_to_variant kernel #8195DataType::Duration support for cast_to_variant kernel #8194variant_get: typed path access (STEP 1) #8150variant feature to parquet crate #8132VariantArray::value for shredded variants #8091DataType::Map support for cast_to_variant kernel #8063DataType::List/LargeList support for cast_to_variant kernel #8060Fixed bugs:
to_ne_bytes found for struct bloom_filter::Block for target s390x-unknown-linux-gnu #8207Date64 or Timestamp Values values #8155Documentation updates:
Closed issues:
Variant::as_f16 #8228VariantArrayBuilder uses ParentState for simpler rollbacks #8205ObjectBuilder::finish signature infallible #8184i256 to f64 #8013Merged pull requests:
Arc::clone in CacheOptions for Parquet PredicateCache #8338 [parquet] (alamb)can_cast_types for temporal to Utf8View #8328 [arrow] (findepi)variant_integration test to use final approved parquet-testing data #8325 [parquet] (alamb)RecordDecoder #8293 [arrow] (jecsand838)Display for Tz #8275 [arrow] (kylebarron)SSLKEYLOGFILE support for flight CLI #8239 [arrow] [arrow-flight] (crepererum)cast_to_variant #8235 (liamzwbao)Field::try_canonical_extension_type #8216 [arrow] (mbrobbel)DataType::Duration support for cast_to_variant kernel #8215 (liamzwbao)Object::finish #8214 (Weijun-H)DataType::Union support for cast_to_variant kernel #8196 (liamzwbao)variant_experimental feature to parquet crate #8133 [parquet] (alamb)VariantArray::value for shredded variants #8105 (klion26)* This Changelog was automatically generated by github_changelog_generator
ARROW-012-015: Add linter rules for remaining Arrow 56.0 breaking changes \#8120
<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -->
Implemented enhancements:
ArrowWriter #8012 [parquet]Fixed bugs:
Documentation updates:
Performance improvements:
RecordBatchDecoder #8127 [arrow] (rluvaton)Vec directly in builders #7984 [arrow] (liamzwbao)Closed issues:
VariantArrayBuilder tracks starting offsets instead of (offset, len) pairs #8192ValueBuilder API to work with ParentState for reliable nested rollbacks #8188ValueBuffer as ValueBuilder #8186ParentState to track and rollback state on behalf of its owning builder #8182ObjectBuilder should detect duplicates at insertion time, not at finish #8180StringView and LargeString in ´batch_json_string_to_variant` #8145 [parquet]batch_json_string_to_variant and batch_variant_to_json_string json_to_variant #8144 [parquet]tempfile crate rather than custom temporary file generator in tests #8143 [arrow]Write rather dyn Write in Decoder #8142 [arrow]ShreddingState::AllNull variant #8088 [parquet]variant_get #8083 [parquet]DataType::RunEndEncoded support for cast_to_variant kernel #8064 [parquet]DataType::Dictionary support for cast_to_variant kernel #8062 [parquet]DataType::Struct support for cast_to_variant kernel #8061 [parquet]DataType::Decimal32/Decimal64/Decimal128/Decimal256 support for cast_to_variant kernel #8059 [parquet]DataType::Timestamp(..) support for cast_to_variant kernel #8058 [parquet]DataType::Float16 support for cast_to_variant kernel #8057 [parquet]DataType::Interval support for cast_to_variant kernel #8056 [parquet]DataType::Time32/Time64 support for cast_to_variant kernel #8055 [parquet]DataType::Date32 / DataType::Date64 support for cast_to_variant kernel #8054 [parquet]DataType::Null support for cast_to_variant kernel #8053 [parquet]DataType::Boolean support for cast_to_variant kernel #8052 [parquet]DataType::FixedSizeBinary support for cast_to_variant kernel #8051 [parquet]DataType::Binary/LargeBinary/BinaryView support for cast_to_variant kernel #8050 [parquet]DataType::Utf8/LargeUtf8/Utf8View support for cast_to_variant kernel #8049 [parquet]cast_to_variant kernel #8043 [parquet]variant_get kernel for shredded variants #7941 [parquet]Decimal128 (i128::MIN and i128::MAX) to f64 with overflow handling #7939 [arrow]Merged pull requests:
DataType::RunEndEncoded support for cast_to_variant kernel #8174 (liamzwbao)DataType::Dictionary support for cast_to_variant kernel #8173 (liamzwbao)ArrayBuilder for UnionBuilder #8169 [arrow] (grtlr)LargeString and StringView in batch_json_string_to_variant #8163 (liamzwbao)batch_json_string_to_variant and batch_variant_to_json_string #8161 (liamzwbao)arrow-avro writer implementation with basic type support #8123 [arrow] (jecsand838)cast_to_variant kernel #8113 (abacef)DataType::Null support to cast_to_variant #8107 (feniljain)DataType::Decimal32/Decimal64/Decimal128/Decimal256 support for cast_to_variant kernel #8101 (liamzwbao)Decoder to support partial decoding #8100 [arrow] (jecsand838)DataType::Boolean support for cast_to_variant kernel #8085 (sdf-jkl)DataType::{Date32,Date64} => Variant::Date #8081 (superserious-dev)DataType::{Binary, LargeBinary, BinaryView} => Variant::Binary #8074 (superserious-dev)DataType::Float16 => Variant::Float #8073 (superserious-dev)cast_to_variant kernel to cast native types to VariantArray #8044 [parquet] (alamb)SchemaStore and fingerprinting #8039 [arrow] (jecsand838)variant_get and Shredded VariantArray #8021 [parquet] (alamb)* This Changelog was automatically generated by github_changelog_generator
One column per quarter.