NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →
crates.io, #3081 most downloaded on crates.io
Apache Arrow Flight
Latest stable
60.1.0
Last release 25 days ago
28 Oct 2026
Ships fairly regularly
a new release about every 4 weeks
Nearly every release is documented
notes for 59 of the last 60 stable releases
Nothing withdrawn
no release was ever pulled
7 years old
113 releases · first in 2020
Drag across the ruler to compare, or tap a release to open it.
chore: Update docs to refer to non deprecated function \(partition\) \#5027 [arrow] (alamb)
Breaking changes:
Implemented enhancements:
RecordBatch::remove_column() and Schema::remove_field() #4952 [arrow]arrow_json: support binary deserialization #4945 [arrow]ParquetRecordWriter proc macro in parquet_derive, but ParquetRecordReader is missing #4772 [parquet]Fixed bugs:
Documentation updates:
Merged pull requests:
Field::remove(), Schema::remove(), and RecordBatch::remove_column() APIs #4959 [arrow] (Folyd)RecordReader trait and proc macro to implement it for a struct #4773 [parquet] (Joseph-Rance)Your coding agent can read these notes before it upgrades. Set up the MCP server →
Nothing published for this version
Evaluate null\_regex for string type in csv \(now such values will be parsed as Null rather than ""\) \#4942 [arrow] (haohuaijin)
Breaking changes:
Null rather than "") #4942 [arrow] (haohuaijin)Implemented enhancements:
Null datatype #4901 [arrow]csv::WriterBuilder #4735 [arrow]Fixed bugs:
None in csv? #4939 [arrow]Utf8 for empty columns #4903 [arrow]NullArray::is_null() returns false incorrectly #4835 [arrow]infer_json_schema_from_seekable #4814 [arrow]Closed issues:
Merged pull requests:
FileWriter schema getter #4940 [arrow] (haixuanTao)SubstraitPlan structure in arrow_flight::sql (#4932) #4933 [arrow] [arrow-flight] (amartins23)FlightDataEncoder API, add examples #4916 [arrow] [arrow-flight] (alamb)FlightDataEncoder to always resend batch dictionaries #4896 [arrow] [arrow-flight] (alexwilcoxson-rel)Array::is_null #4838 [arrow] (alamb)cast_string_to_decimal #4830 [arrow] (jonahgao)More chrono deprecations \#4822 [arrow] (tustvold)
Breaking changes:
RowConverter (#4811) #4819 [arrow] [arrow-flight] (tustvold)std::io::Error to IoError and add IpcError variant #4726 [arrow] [arrow-flight] (alexandreyc)Implemented enhancements:
DynScalar abstraction (something that makes it easy to create scalar Datums) #4781 [arrow]Datum is not exported as part of arrow (it is only exported in arrow_array) #4780 [arrow]Scalar is not exported as part of arrow (it is only exported in arrow_array) #4779 [arrow]Fixed bugs:
concat_batches errors with "schema mismatch" error when only metadata differs #4799 [arrow]cmp kernels with DictionaryArrays: Option::unwrap() on a None value' #4788 [arrow]*ListArray in 46.0.0 #4746 [arrow]Box<dyn RecordBatchReader + Send> #4751 [arrow] (wjones127)Closed issues:
Merged pull requests:
Scalar #4793 [arrow] (alamb)Remove deprecated limit kernel \#4597 [arrow] (tustvold)
Breaking changes:
batches_to_flight_data forces clone #4656 [arrow]Array Logical Nullability #4691 [parquet] [arrow] (tustvold)Implemented enhancements:
concat_batches for 0 columns #4661 [arrow]filter_record_batch should support filtering record batch without columns #4647 [arrow]lexicographical_partition_ranges #4614 [arrow]DataType::List correctly #4544 [arrow]Fixed bugs:
RowInterner::size() much too low for high cardinality dictionary columns #4645 [arrow]List(FixedSizeBinary) array equality check may return wrong result #4637 [arrow]arrow::compute::nullif panics if NullArray is provided #4634 [arrow]MutableBuffer::set_null_bits can be bypassed #4620 [arrow]compressed_size, file offset. #4610 [parquet]new_null_array generates an invalid union array #4600 [arrow]arrow::compute::concat panics for dense union arrays with non-trivial type IDs #4578 [arrow]Closed issues:
Merged pull requests:
ArrowReaderBuilder construction (#4674) #4676 [parquet] (tustvold)concat_batches for 0 columns #4662 [arrow] (Dandandan)From<Vec<T>> for BufferBuilder and MutableBuffer #4650 [arrow] (mbrobbel)Bucket size in OrderPreservingInterner #4646 [arrow] (alamb)Default,Extend and FromIterator for BufferBuilder #4638 [arrow] (mbrobbel)NullArray in nullif #4635 [arrow] (kawadakk)BufferBuilder to arrow-buffer #4630 [arrow] (mbrobbel)FromIterator for MutableBuffer #4624 [arrow] (mbrobbel)MutableBuffer::set_null_bits #4621 [arrow] (kawadakk)Default for arrow_buffer::buffer::MutableBuffer #4619 [arrow] (mbrobbel)Fix timezoned timestamp arithmetic \#4546 [arrow] (alexandreyc)
Breaking changes:
Implemented enhancements:
BooleanBuilder: Add validity_slice method for accessing validity bits #4535 [arrow]FixedSizedListArray for length kernel #4517 [arrow]RowCoverter::convert that targets an existing Rows #4479 [arrow]Fixed bugs:
assertion failed: idx < self.len when casting DictionaryArrays with nulls #4576 [arrow]FixedSizeListBuilder::new #4549 [arrow]Merged pull requests:
is_empty impl for arrow::array::ArrayBuilder #4543 [arrow] (mbrobbel)FixedSizedListArray for length kernel #4520 [arrow] (Weijun-H)Port Tests from Deprecated Arithmetic Kernels \#4480 [arrow]
Breaking changes:
Implemented enhancements:
Date - Date kernel #4383 [arrow]Fixed bugs:
AsyncArrowWriter to a file corrupts the footer for large columns #4526 [parquet]Merged pull requests:
Simplify ffi import/export \#4447 [arrow] (Virgiel)
Breaking changes:
Implemented enhancements:
Row to Rows #4466 [arrow]FlightSqlService to handle custom actions and commands #4439NullBuilder #4429 [arrow]Fixed bugs:
42.0.0 : Bad parquet column indexes for All Null Columns, resulting in Parquet error: StructArrayReader out of sync on read #4459 [parquet]Documentation updates:
Merged pull requests:
NullBuilder #4430 [arrow] (izveigor).5 the same as 0.5 #4425 [arrow] (alamb)deprecate: as\_decimal\_array \#4363 [arrow] (izveigor)
Breaking changes:
Implemented enhancements:
millisecond and microsecond functions #4374 [arrow]CommandGetXdbcTypeInfo responses (XdbcInfoValue and builders) #4257 [arrow] [arrow-flight]Fixed bugs:
Closed issues:
dict_id, dict_is_ordered into DataType? #4325Merged pull requests:
take kernel performance on primitive arrays, fix bad null index handling (#4404) #4405 [arrow] (tustvold)BooleanBuffer::new_unset and BooleanBuffer::new_set and BooleanArray::new_null constructors #4402 [arrow] (tustvold)Hash impls for CastOptions and FormatOptions #4395 [arrow] (alamb)WriterProperites and ReaderProperties docs #4392 [parquet] (alamb)microsecond and millisecond kernels #4375 [arrow] (izveigor)Use page_size consistently, deprecate pagesize in parquet WriterProperties \#4313 [parquet] (alamb)
Breaking changes:
FlightData, FlightInfo, FlightEndpoint, Locaation and Ticket #4294 [arrow] [arrow-flight] (alamb)Implemented enhancements:
CommandGetCatalogs, CommandGetSchemas, and CommandGetTables requests #4295 [arrow] [arrow-flight]FlightInfo to make it easier to create new requests #4281 [arrow] [arrow-flight]CommandGetSqlInfo responses (SqlInfoValue and builders) #4256 [arrow] [arrow-flight]Intervals #4181 [arrow]Fixed bugs:
Merged pull requests:
page_size consistently, deprecate pagesize in parquet WriterProperties #4313 [parquet] (alamb)CommandGetCatalogs, CommandGetSchemas, and CommandGetTables requests #4296 [arrow] [arrow-flight] (roeap)ArrowWriter memory usage: Buffer Pages in ArrowWriter instead of RecordBatch (#3871) #4280 [parquet] (tustvold)Debug impls for ArrowWriter and SerializedFileWriter #4278 [parquet] (alamb)RecordBatchWriter to arrow crate #4277 [arrow] (alexandreyc)Intervals #4182 [arrow] (izveigor)Prefetch page index \(\#4090\) \#4216 [parquet] (tustvold)
Breaking changes:
Implemented enhancements:
Interval +/- Interval #4178 [arrow]print_column_chunk_metadata() #4172 [parquet]Fixed bugs:
Merged pull requests:
RecordBatchReader for CSV reader #4195 [arrow] (alexandreyc)Remove deprecated parquet ArrowReader \#4125 [parquet] (tustvold)
Breaking changes:
StructArray w/ FieldRef #4116 [parquet] [arrow] (crepererum)Implemented enhancements:
Interval + Timestamp and Interval + Date in addition to Timestamp + Interval and Interval + Date #4094 [arrow]Fixed bugs:
Documentation updates:
Closed issues:
Merged pull requests:
bitwise shift left/right #4148 [arrow] (Weijun-H)as_primitive #4143 [arrow] (Weijun-H)Interval + Timestamp and Interval + Date #4117 [arrow] (Weijun-H)Remove DataType from PrimitiveArray constructors \#4098 [arrow] (tustvold)
Breaking changes:
Implemented enhancements:
Timestamp +/- Interval types #3963 [arrow]Fixed bugs:
Documentation updates:
Closed issues:
Merged pull requests:
Hash trait to SortOptions. #4089 [arrow] (mustafasrepo)Deprecate Array::data \(\#3880\) \#4019 [arrow] (tustvold)
Breaking changes:
Implemented enhancements:
Fixed bugs:
Closed issues:
Merged pull requests:
arrow-flight crates #4012 [arrow] [arrow-flight] (alamb)RecordBatch by adding Serde support to RawDecoder (#3949) #3979 [arrow] (tustvold)Binary/LargeBinary and FixedSizeBinary #3961 [arrow] (Weijun-H)Use dyn Array in sort kernels \#3931 [arrow] (tustvold)
Breaking changes:
Implemented enhancements:
Fixed bugs:
Merged pull requests:
ListBuilder and GenericListBuilder #3891 [arrow] (alamb)mul_checked to avoid silent overflow in interval arithmetic #3886 [arrow] (Weijun-H)Add RunEndBuffer \(\#1799\) \#3817 [arrow] (tustvold)
Breaking changes:
FlightSqlServiceClient improvements #3788 [arrow] [arrow-flight] (crepererum)Implemented enhancements:
Fixed bugs:
Closed issues:
Merged pull requests:
Utf8 and LargeUtf8 --> Interval #3762 [arrow] (doki23)into_primitive_dict_builder to DictionaryArray #3715 [arrow] (viirya)Deprecate old JSON reader \(\#3610\) \#3718 [parquet] [arrow] (tustvold)
Breaking changes:
FlightClient::do_put #3464 [arrow] [arrow-flight] (alamb)Implemented enhancements:
take_run improvements #3701 [arrow]as_mut_any in Array trait #3655Array --> Display formatter that supports more options and is configurable #3638 [parquet] [arrow]Fixed bugs:
Merged pull requests:
Ord/PartialOrd for SortOptions #3723 [arrow] (crepererum)take_run improvements #3705 [arrow] (askoa)RunArray #3695 [arrow] (askoa)Use ArrayFormatter in Cast Kernel \#3668 [arrow] (tustvold)
Breaking changes:
Implemented enhancements:
array_value_to_string in arrow-csv #3483 [arrow]Fixed bugs:
Closed issues:
32.0.0 of arrow/arrow-flight/parquet/parquet-derive (next release after 31.0.0) #3584 [parquet] [arrow] [arrow-flight]Merged pull requests:
Field #3630 [arrow] (chunshao90)Write Deprecated Min Max Statistics When ColumnOrder Signed \#3526 [parquet]
Breaking changes:
StringArray construction with Vec<Option<String>> #3602 [arrow] (sinistersnare)RunEndEncodedArray #3553 [parquet] [arrow] (askoa)Implemented enhancements:
From<Vec<Option<String>>> impl for GenericStringArray<OffsetSize> #3599 [arrow]Error::Source for ArrowError and FlightError #3566 [arrow] [arrow-flight]External variant to ParquetError #3285 [parquet]Fixed bugs:
nullif incorrectly calculates null_count, sometimes panics with subtraction overflow error #3579 [arrow]Closed issues:
Merged pull requests:
std::error::Error::source for ArrowError and FlightError #3567 [arrow] [arrow-flight] (alamb)support RFC3339 style timestamps in arrow-json \#3449 [arrow] (JayjeetAtGithub)
Breaking changes:
arrow-json #3449 [arrow] (JayjeetAtGithub)Implemented enhancements:
'2001-01-01' to timestamp #3492 [arrow]build_compare #3465concat_batches to take non owned RecordBatch #3456 [arrow]30.0.1 (maintenance release for 30.0.0) #3455get_slice_memory_size or similar #3407 [arrow] [arrow-flight]Fixed bugs:
Merged pull requests:
31.0.0 and add changelog #3518 [parquet] [arrow] [arrow-flight] (iajoiner)concat_batches parameter to non owned reference #3480 [arrow] (askoa)parquet-rewrite CLI #3477 [parquet] (crepererum)build_compare #3466 (viirya)FlightClient::{list_flights, list_actions, do_action, get_schema} #3463 [arrow-flight] (alamb)column_name based index access for RecordBatch and StructArray #3458 [arrow] (askoa)*like_dyn, *like_utf8_scalar_dyn and *like_dict functions to macros #3411 [arrow] (askoa)FlightClient #3402 [arrow-flight] (alamb)RecordBatch <--> FlightData encode/decode + tests #3391 [arrow] [arrow-flight] (alamb)into_builder for bytearray #3326 [arrow] (viirya)Generic bytes dictionary builder \#3425 [arrow]
Implemented enhancements:
ArrowFlight Client #3371 [arrow-flight]Fixed bugs:
nullif kernel no longer exported #3454 [arrow]Document / Deprecate arrow\_flight::flight\_data\_from\_arrow\_batch \#3312 [arrow] [arrow-flight]
Breaking changes:
Implemented enhancements:
ParquetObjectReader #3381 [parquet]Fixed bugs:
try_to_type #3350Documentation updates:
Merged pull requests:
ParquetObjectReader #3382 [parquet] (kszlim)FlightClient #3378 [arrow-flight] (alamb)Deprecate limit kernel \#3200 [arrow] (tustvold)
Breaking changes:
Field::new and Field::new_with_dict to take existing String as well as &str #3288 [arrow] (alamb)&Option<T> to Option<&T> #3249 [parquet] [arrow] (Jimexist)*_dict_scalar kernels behind *_dyn kernels #3202 [arrow] (viirya)Implemented enhancements:
Fixed bugs:
compile-time-rng kills reproducible builds #3271 [parquet]nullif panics on empty array #3261 [arrow]can_cast_types returns true #3221 [arrow]arrow-cast crate #3219 [arrow]Documentation updates:
Merged pull requests:
ilike_scalar and nilike_scalar #3303 [arrow] (askoa)arrow-cast to arrow #3222 [arrow] (viirya)FixedSizeBinary in Row format #3182 [arrow] (tustvold)Clippy failing on master : error: use of deprecated associated function chrono::NaiveDate::from\_ymd: use from\_ymd\_opt\(\) instead \#3097 [parquet]…
Breaking changes:
Field::metadata to HashMap #3148 [parquet] [arrow] (crepererum)Field::metadata #3091 [parquet] [arrow] (askoa)Implemented enhancements:
Field::size and DataType::size #3147 [parquet] [arrow]SortingColumn when reading and writing parquet metadata #3090 [parquet]Fixed bugs:
concat_batches #3123 [arrow]Closed issues:
27.0.0 (next release after 26.0.0) #3045 [parquet] [arrow] [arrow-flight]Merged pull requests:
arrow-data to arrow-schema crate #3177 [arrow] (mbrobbel)RowSelection::iter(), Into<Vec<RowSelector>> and example #3173 [parquet] (alamb){Field,DataType}::size #3149 [arrow] (crepererum)bloom default feature, add reader properties #3119 [parquet] (Jimexist)SortingColumn in parquet files #3103 [parquet] (askoa)Deprecate Buffer::count\_set\_bits \#3067 [arrow]
Breaking changes:
RowConverter #3080 [arrow] (crepererum)Implemented enhancements:
TimestampNanosecondArray with timezones #3042 [arrow]RowSelection::intersection #3003 [parquet]26.0.0 (next release after 25.0.0) #2953 [parquet] [arrow] [arrow-flight]Fixed bugs:
Documentation updates:
Merged pull requests:
TimestampNanosecondArray with timezones #3088 [arrow] (src255)OwnedRow #3079 [arrow] (crepererum)intersect_row_selections from datafusion to arrow-rs. #3047 [parquet] (Ted-Jiang)arrow::compute::kernels::temporal should support nanoseconds #2996 [arrow] (comphead)RowSelection::from_selectors_and_combine to merge RowSelectors #2994 [parquet] (Ted-Jiang)lexsort_to_indices #2991 [arrow] (alamb)byte_size from datafusion::physical_expr #2965 [arrow] (avantgardnerio)Deprecate TimestampArray from\_vec and from\_opt\_vec \#2919 [parquet] [arrow] (tustvold)
Breaking changes:
from_timestamp by from_timestamp_opt #2894 [arrow] (waitingkuo)Implemented enhancements:
true and false values in a BooleanArray #2963 [arrow]freeze_with_dictionary API to MutableArrayData #2914 [arrow][+/-]hhmm and [+/-]hh as fixedoffset timezone format #2910 [arrow]from_timestamp by from_timestamp_opt #2892 [arrow]Fixed bugs:
total_cmp found for struct f16 in the current scope #2926 [arrow]and_then method #2925 [parquet]RawPtrBox::new #2882 [arrow]parquet::arrow::arrow_writer::ArrowWriter ignores page size properties #2853 [parquet]Documentation updates:
Closed issues:
25.0.0 (next release after 24.0.0) #2820 [parquet] [arrow] [arrow-flight]Merged pull requests:
WriterProperties (#2941) #2942 [parquet] (tustvold)chrono-tz feature #2879 [arrow] (viirya)Make DecimalArray as PrimitiveArray \#2857 [parquet] [arrow] (viirya)
Breaking changes:
Implemented enhancements:
ArrowNativeTypeOp #2753 [arrow]Fixed bugs:
parquet::arrow::arrow_writer::ArrowWriter ignores page size properties #2853 [parquet]the trait bound for<'de> arrow::datatypes::Schema: serde::de::Deserialize<'de> is not satisfied #2825 [arrow]Closed issues:
Merged pull requests:
snap option to README #2847 [parquet] (exyi)interleave kernel (#1523) #2838 [arrow] (tustvold)ArrowNativeTypeOp #2756 [arrow] (HaoYang670)Cleanup ArrowNativeType \(\#1918\) \#2793 [parquet] [arrow] (tustvold)
Breaking changes:
ArrowNativeType (#1918) #2793 [parquet] [arrow] (tustvold)ArrowNativeType::FromStr #2775 [arrow] (tustvold)arrow-array crate (#2594) #2769 [arrow] (tustvold)dyn_arith_dict feature flag #2760 [arrow] (tustvold)arrow-data into a separate crate #2746 [arrow] (tustvold)Implemented enhancements:
binary function should not panic on unequaled array length. #2721 [arrow]Fixed bugs:
try_unary_dict should check value type of dictionary array #2754 [arrow]Closed issues:
23.0.0 (next release after 22.0.0) #2665 [parquet] [arrow] [arrow-flight]Merged pull requests:
verify_release_candidate.sh for new arrow subcrates #2752 (alamb)Deprecate RecordBatch::concat replace with concat\_batches \(\#2594\) \#2683 [arrow] (tustvold)
Breaking changes:
Implemented enhancements:
RecordBatchOptions #2728 [arrow]CHANGELOG-old.md #2726DivideByZero error for float modulus #2720 [arrow]try_binary should not panic on unequaled array length. #2715 [arrow]DataType function to detect nested types #2704 [arrow]RecordBatch::project #2690 [arrow]min_max_helper #2657 [arrow]downcast_primitive_array in arithmetic kernels #2639 [arrow]Fixed bugs:
DivideByZero in the Divide kernel #2719 [arrow]arrow crate does not build with features = ["ffi"] and default_features = false. #2670 [arrow]RowSelector having row_count of 0 #2669 [parquet]crate::array::layout #2659 [arrow]CastOptions #2648 [arrow]flight.proto and fix schema to SchemaResult #2571 [arrow] [arrow-flight]SchemaResult in IPC deviates from other implementations #2445 [arrow] [arrow-flight]Closed issues:
Merged pull requests:
CHANGELOG-old.md #2732 (iajoiner)try_binary and checked_ops, and remove math_checked_op #2717 [arrow] (HaoYang670)numeric_cast_with_error #2661 [arrow] (liukun4515)test_fuzz_async_reader_selection #2656 [parquet] (thinkharderdev)CastOption for casting numeric #2649 [arrow] (liukun4515)downcast_primitive_array in arithmetic kernels #2640 [arrow] (viirya)Use total_cmp for floating value ordering and remove nan_ordering feature flag \#2614 [arrow] (viirya)
Breaking changes:
total_cmp for floating value ordering and remove nan_ordering feature flag #2614 [arrow] (viirya)dyn_cmp_dict #2597 [arrow] (tustvold)Implemented enhancements:
total_cmp for floating value ordering and remove nan_ordering feature flag #2613 [arrow]dyn_cmp_dict feature flag to gate dyn comparison of dictionary arrays #2596 [arrow]do_get() #2581 [arrow-flight]eq_dyn, neq_dyn, lt_dyn, lt_eq_dyn, gt_dyn, gt_eq_dyn #2569 [arrow]sum for dictionary array #2565 [arrow]lt_dyn, lt_eq_dyn, gt_dyn, gt_eq_dyn #2538 [arrow]eq_dyn and neq_dyn #2535 [arrow]like_utf8_scalar for %pat% #2519 [arrow]Fixed bugs:
ParquetFileArrowReader::get_record_reader[_by_column] batch_size overallocates #2321 [parquet]Documentation updates:
Closed issues:
Interval(DayTime) to Timestamp(Nanosecond, None) #2606 [arrow]length field for Buffer #2524 [arrow]const_generic. #2390 [arrow]const_generic #2384 [arrow]Merged pull requests:
length field for Buffer and use more Buffer in IPC reader to avoid memory copy. #2557 [arrow] [arrow-flight] (HaoYang670)Decimal) when creating array in IPC reader #2547 [arrow] (HaoYang670)eq_dyn and neq_dyn #2533 [arrow] (viirya)create_array in IPC reader. #2525 [arrow] (HaoYang670)%pat% (>3x speedup) #2521 [arrow] (Dandandan)Remove deprecated ParquetWriter \#2380 [parquet] (tustvold)
Breaking changes:
ColumnCloseResult (#2465) #2466 [parquet] (tustvold)ChunkReader into SerializedPageReader (#2463) #2464 [parquet] (tustvold)Implemented enhancements:
ChunkReader into SerializedPageReader #2463 [parquet]RowFilter withinParquetRecordBatchReader #2431 [parquet]StructBuilder::len #2429 [arrow]ParquetRecordBatchReaderBuilder) #2427 [parquet]OffsetIndex to Prune IO in ParquetRecordBatchStream #2426 [parquet]peek_next_page and skip_next_page in InMemoryPageReader #2406 [parquet]Utf8/LargeUtf8 to Binary/LargeBinary #2402 [arrow]Decimal128 and Decimal256 arrays #2375 [arrow]skip_records #2358 [parquet]read_buffer Arrow IPC API to return Result<_> #2342 [arrow]skip_records in GenericColumnReader to skip across row groups #2331 [parquet]Decimal256 #2320 [arrow]DeltaBitPackDecoder #2281 [parquet]ParquetRecordBatchStream to support row filtering in DataFusion #2270 [parquet]ArrayReader::skip_records API #2197 [parquet]Fixed bugs:
DoPutUpdateResult #2403 [arrow-flight]AsyncFileReaderNo Longer Object-Safe #2372 [parquet]Closed issues:
Merged pull requests:
MapArrayReader (#2484) (#1699) (#1561) #2500 [parquet] (tustvold)split_second function #2436 [arrow] (viirya)Decimal and DecimalArray using const_generic #2383 [parquet] [arrow] (HaoYang670)Decimal128 and Decimal256 arrays #2376 [arrow] (viirya)Box<dyn AsyncFileReader> #2368 [parquet] (tustvold)Decimal256 validation based on bytes comparison and add benchmark test #2360 [parquet] [arrow] (liukun4515)field.rs and add more tests #2345 [arrow] (HaoYang670)Add more const evaluation for GenericBinaryArray and GenericListArray: add PREFIX and data type constructor \#2327 [parquet] [arrow] (HaoYang670)
Breaking changes:
GenericBinaryArray and GenericListArray: add PREFIX and data type constructor #2327 [parquet] [arrow] (HaoYang670)safe (#2302) #2303 [arrow] (tustvold)test_utils from default features (#2298) #2299 [arrow] (tustvold)DataType::Decimal to DataType::Decimal128 #2229 [parquet] [arrow] (viirya)Decimal128Iter and Decimal256Iter and do maximum precision/scale check #2140 [arrow] (viirya)Implemented enhancements:
ListArray #2311 [arrow]FlightSqlService trait to pass session info along #2308 [arrow-flight]take_bits for non-null indices #2306 [arrow]ffi #2302 [arrow]ffi::ArrowArray::try_new is safe #2301 [arrow]JsonEqual trait #2296 [arrow]with_precision_and_scale to Decimal array traits #2291 [arrow]ArrayReader::next_batchwith ArrayReader::read_records and ArrayReader::consume_batch #2236 [parquet]DataType::Decimal to DataType::Decimal128 #2228 [arrow]append_option support to Decimal128Builder and Decimal256Builder #2224 [arrow]FixedSizeBinaryArray and FixedSizeListArray from array_binary.rs and array_list.rs #2217 [arrow]Box Values in PrimitiveDictionaryBuilder #2215 [arrow]Hash for Schema #2182 [arrow]GenericStringBuilder should use GenericBinaryBuilder #2156 [arrow]Decimal128 and Decimal256 #2139 [arrow]ArrayAccessor in Decimal128Iter and Decimal256Iter #2138 [arrow]ArrayAccessor and FromIterator in Cast Kernels #2137 [arrow]TypedDictionaryArray for more ergonomic interaction with DictionaryArray #2136 [arrow]ArrayAccessor in Comparison Kernels #2135 [arrow]peek_next_page() and skip_next_page in InMemoryColumnChunkReader #2129 [parquet]Decimal256 #2112 [arrow]skip_def_levels for ColumnLevelDecoder #2107 [parquet]Time32 / Time64 #2053 [arrow]Fixed bugs:
Decimal128 or Decimal256 #2256 [arrow]IntervalMonthDayNanoType::make_value() does not match C implementation #2234 [arrow]FlightSqlService trait does not allow impls to do handshake #2210 [arrow-flight]EnabledStatistics::None not working #2185 [parquet]Documentation updates:
Performance improvements:
Closed issues:
Merged pull requests:
Request<FlightDescriptor> to FlightSqlService impls #2309 [parquet] [arrow-flight] (avantgardnerio)with_precision_and_scale to BasicDecimalArray trait #2292 [parquet] [arrow] (viirya)fn get_data_type by const DATA_TYPE in BinaryArray and StringArray #2289 [arrow] (HaoYang670)from_list function. #2277 [arrow] (HaoYang670)like_utf8 / nlike_utf8 kernels #2258 [arrow] (daniel-martinez-maqueda-sap)FixedSizeList to array_fixed_size_list.rs #2250 [arrow] (HaoYang670)Schema::fields and Schema::metadata pub (public) #2239 [arrow] (alamb)IntervalMonthDayNanoType::make_value() to conform to specifications #2235 [arrow] (avantgardnerio)arrow CI on changes to arrow #2227 (alamb)FixedSizeBinaryArray to array_fixed_size_binary.rs #2218 [arrow] (HaoYang670)FlightSqlService trait to proxy handshake #2211 [arrow-flight] (avantgardnerio)serde_json feature name #2209 [parquet] (flisky)Hash for Schema #2183 [arrow] (crepererum)StringBuilder use BinaryBuilder #2181 [arrow] (HaoYang670)BYTE_ARRAY type #2160 [parquet] (liukun4515)peek\_next\_page and skip\_next\_page for `InMemoryColumnCh… #2155 [parquet] (thinkharderdev)arrow changes #2152 (alamb)object_store code from object_store_rs to arrow-rs #2081 (alamb)validate_utf8 performance #2048 [arrow] (tfeda)Rename DecimalArray/DecimalBuilder to Decimal128Array/Decimal128Builder \#2101 [arrow]
Breaking changes:
DecimalArray``/DecimalBuilder to Decimal128Array/Decimal128Builder #2101 [arrow]append methods to be infallible where possible #2103 [parquet] [arrow] (jhorstmann)UnionArray::child (#2035) #2099 [arrow] (tustvold)preserve_order feature from serde_json dependency (#2095) #2098 [parquet] [arrow] (tustvold)weekday and weekday0 kernels to to num_days_from_monday and num_days_since_sunday #2066 [arrow] (alamb)null_count from write_batch_with_statistics #2047 [parquet] (tustvold)Implemented enhancements:
total_cmp from std #2130 [arrow]ParquetRecordBatchStream #2110 [parquet]GenericBinaryBuilder should use buffer builders directly. #2104 [arrow]generate_decimal256_case arrow integration test #2093 [arrow]weekday and weekday0 kernels to to num_days_from_monday and days_since_sunday #2065 [arrow]filter_dict #2062 [arrow]set_bits #2060 [arrow]BooleanBuilder #2058 [arrow]BooleanArray::from_iter should omit validity buffer if all values are valid #2055 [arrow]DICTIONARY_ORDERED flag if a field's dictionary is ordered #2049 [arrow]peek_next_page() and skip_next_page in SerializedPageReader #2043 [parquet]MapType #2037 [arrow]DecimalArrayBuilder should use FixedSizedBinaryBuilder #2026 [arrow]serialized_reader read specific Page by passing row ranges. #1976 [parquet]Fixed bugs:
type_id and value_offset are incorrect for sliced UnionArray #2086 [arrow]take kernel does not handle null indices correctly #2057 [arrow]write_batch_with_statistics #2046 [parquet]WriterProperties #2014 [parquet]Documentation updates:
Closed issues:
serde_json specify the preserve_order feature in arrow package #2095 [arrow]skip_values in DictionaryDecoder #2079 [parquet]skip_values in ByteArrayColumnValueDecoder #2072 [parquet]Builder::append methods returning results even though they are infallible #2071UnionArray::child #2035Merged pull requests:
total_cmp from std #2131 [arrow] (Dandandan)match arms have incompatible types #2121 (alamb)GenericBinaryBuilder to use buffer builders directly. #2117 [arrow] (HaoYang670)DataType::Decimal256 #2094 [parquet] [arrow] (viirya)DecimalBuilder should use FixedSizeBinaryBuilder #2092 [arrow] (HaoYang670)FixedSizeBinaryArray to DecimalArray #2041 [arrow] (HaoYang670)Fix several bugs in parquet writer statistics generation, add EnabledStatistics to control level of statistics generated \#2022 [parquet] (tustvold)
Breaking changes:
EnabledStatistics to control level of statistics generated #2022 [parquet] (tustvold)Decimal256Builder and Decimal256Array; Decimal arrays now implement BasicDecimalArray trait #2000 [parquet] [arrow] (viirya)ColumnReader::read_batch #1995 [parquet] [arrow] (tustvold)PrimitiveBuilder::finish_dict (#1978) #1980 [arrow] (tustvold)NullType #1942 [arrow] (liukun4515)Implemented enhancements:
DataType::Dictionary support to subtract_scalar, multiply_scalar, divide_scalar #2019 [arrow]add_scalar kernel #2017 [arrow]FixedSizeBinaryBuilder #2007 [arrow]Decimal256Builder and Decimal256Array #1999 [arrow]DictionaryArray in unary kernel #1989 [arrow]Arrays #1987 [arrow]DictionaryArray in divide kernel #1982 [arrow]Into<ArrayData> for T: Array #1979 [arrow]DictionaryArray in multiply kernel #1972 [arrow]DictionaryArray in subtract kernel #1970 [arrow]DecimalArray::length as a constant #1967 [arrow]DictionaryArray in add kernel #1950 [arrow]Field #1934 [arrow]StringDictionaryBuilder faster #1851 [arrow]concat_elements_utf8 should accept arbitrary number of input arrays #1748 [arrow]Fixed bugs:
ColumnWriterImpl::write_batch_with_statistics incorrect distinct count in statistics #2016 [parquet]ColumnWriterImpl::write_batch_with_statistics can write incorrect page statistics #2015 [parquet]RowFormatter is not part of the public api #2008 [parquet]ColumnReader::read_batch For Corrupted Files #1997 [parquet]PrimitiveBuilder::finish_dict does not validate dictionary offsets #1978 [arrow]n_buffers in FFI_ArrowArray #1959 [arrow]DecimalArray::from_fixed_size_list_array fails when offset > 0 #1958 [arrow]Send + Sync impl for Allocation may not be sound unless Allocation is Send + Sync as well #1944 [arrow]NullType #1923 [arrow]Documentation updates:
Closed issues:
min_bytes and max_bytes return wrong size #2021 [parquet]Decimals by using constant generic. #2001DecimalArray to a new file #1985 [arrow]DictionaryArray in multiply kernel #1974null_count of DictionaryArray #1962 [arrow]TIMESTAMP in schema #1920 [parquet]Merged pull requests:
Field: with_name, with_data_type and with_nullable #2024 [arrow] (alamb)FixedSizeBinaryBuilder #2013 [arrow] (HaoYang670)Into<ArrayData> for T: Array #1992 [parquet] [arrow] (heyrutvik)FixedSizeListBuilder #1988 [arrow] (HaoYang670)DecimalArray to array_decimal.rs #1986 [arrow] (HaoYang670)const #1968 [arrow] (HaoYang670)from_fixed_size_list when offset > 0 #1964 [arrow] (HaoYang670)FixedSizeListArray::value_length #1957 [arrow] (HaoYang670)ColumnMetadata after the column chunk data, not the ColumnChunk #1947 [parquet] (liukun4515)Add validation to RecordBatch for non-nullable fields containing null values \#1890 [arrow] (andygrove)
Breaking changes:
RecordBatch for non-nullable fields containing null values #1890 [arrow] (andygrove)ArrayData::validate_dict_offsets to ArrayData::validate_values #1889 [arrow] (frolovdev)Decimal128 API and use it in DecimalArray and DecimalBuilder #1871 [parquet] [arrow] (viirya)safe (#996) (#1027) #1866 [parquet] [arrow] (tustvold)Implemented enhancements:
ArrowWriter being used with a cursor #1927 [parquet]cast to/from NULL and DataType::Decimal #1921 [arrow]Decimal256 API #1913 [arrow]DictionaryArray::key function #1911 [arrow]ListArrays in MutableArrayData #1884 [arrow]PrimitiveArray::from_iter should omit validity buffer if all values are valid #1856 [arrow]from(v: Vec<Option<&[u8]>>) and from(v: Vec<&[u8]>) for FixedSizedBInaryArray #1852 [arrow]Vec-inspired APIs to BufferBuilder #1850 [arrow]nilike support in comparison #1845 [arrow]arrow::array::builder module #1843 [arrow]quarter support in temporal kernels #1835 [arrow]ArrayData::validate_dictionary_offset to ArrayData::validate_values #1812 [arrow]substring kernel #1801 [arrow]substring_by_char kernel #1800 [arrow]Fixed bugs:
dynamic_types example does not print the projection #1902 [arrow]log2(0) panicked at 'attempt to subtract with overflow', parquet/src/util/bit_util.rs:148:5 #1901 [parquet]combine_option_bitmap needs to use bit slices #1899 [arrow]RecordBatch with null values in non-nullable fields does not cause an error #1888 [arrow]regex dependency #1874 [arrow]Documentation updates:
DictionaryArray::lookup_key #1910 [arrow] (alamb)Closed issues:
Merged pull requests:
arrow::array::builder module (#1843) #1928 [arrow] (tustvold)checked_add/sub().unwrap() with +/- #1924 [arrow] (HaoYang670)NULL to/from Decimal #1922 [arrow] (liukun4515)Decimal256 API #1914 [arrow] (viirya)DictionaryArray::key function #1912 [arrow] (alamb)bit_util of Parquet. #1905 [parquet] (HaoYang670)weekday #1894 [arrow] (alamb)MutableArrayData #1885 [arrow] (jhorstmann)regex dependency to 1.5.6 #1875 [arrow] (martin-g)BitSliceIterator and BitIndexIterator (#1864) #1865 [arrow] (tustvold)from methods for FixedSizeBinaryArray #1854 [arrow] (HaoYang670)substring kernel. #1853 [arrow] (HaoYang670)nilike support in comparison #1846 [arrow] (MazterQyou)test_command_help #1844 [parquet] (viirya)quarter support in temporal #1836 [arrow] (MazterQyou)substring_by_char by about 2.5x #1832 [arrow] (HaoYang670)Implement ChunkReader for Bytes, deprecate SliceableCursor \#1775 [parquet] (tustvold)
Breaking changes:
ArrowNativeType and OffsetSizeTrait for safety (#1028) #1819 [arrow] (tustvold)csv::infer_file_schema by removing redundant ref #1776 [arrow] (tustvold)Implemented enhancements:
ListArray #1817 [arrow]UnionArray in V4 IPC message #1793 [arrow]combine_option_bitmap should accept arbitrary number of input arrays. #1780 [arrow]substring_by_char kernels for slicing on character boundaries #1768 [arrow]PageIndex from column metadata #1761 [parquet]DataType::Utf8 to DataType::Boolean #1740 [arrow]FileWriter. #1691 [parquet]stdout #1687 [parquet]Fixed bugs:
flight_data_to_arrow_batch does not support RecordBatches with no columns #1783 [arrow-flight]features=["zstd"] #1630 [parquet]Documentation updates:
CHANGELOG-old.md #1759 (alamb)Closed issues:
DataType::Decimal Non-Compliant? #1779 [arrow]Parquet is a part of Arrow? #1715 [parquet] [arrow]Merged pull requests:
PageIterator Documentation #1831 [parquet] (Ted-Jiang)resolver v2, test more feature flag combinations in CI, fix errors (#1630) #1822 [parquet] [arrow] (tustvold)arrow/benches/string_kernels.rs to arrow/benches/substring_kernels.rs #1805 [arrow] (HaoYang670)Substring_by_char #1784 [arrow] (HaoYang670)ParquetFileArrowReader::try_new #1782 [parquet] (tustvold)ChunkReader for Bytes, deprecate SliceableCursor #1775 [parquet] (tustvold)validate_decimal_precision check in DecimalBuilder.append_value for interop test #1767 [arrow] (viirya)Utf8 to Boolean #1738 [arrow] (MazterQyou)Change ArrayDataBuilder::null_bit_buffer to accept Option rather than Buffer \#1739 [arrow] (HaoYang670)
Breaking changes:
ArrayDataBuilder::null_bit_buffer to accept Option<Buffer> rather than Buffer #1739 [arrow] (HaoYang670)null_count from ArrayData::try_new() #1721 [arrow] (HaoYang670)std:io::Write rather custom ParquetWriter trait (#1717) (#1163) #1719 [parquet] (tustvold)ProjectionMask (#1701) #1716 [parquet] (tustvold)Implemented enhancements:
string kernel to concatenate_elements #1747 [arrow]ArrayDataBuilder::null_bit_buffer should accept Option<Buffer> as input type #1737 [arrow]BinaryArray #1724 [arrow]generate_unions_case integration test #1712 [arrow]leaf_columns argument to ArrowReader::get_record_reader_by_columns #1653 [parquet]string_concat kernel #1540 [arrow]Fixed bugs:
ArrayData::try_new cannot always return expected error. #1707 [arrow]substring kernel): The null buffer is not aligned when offset != 0 #1639 [arrow]Documentation updates:
Merged pull requests:
string_concat to concat_elements_utf8 #1754 [arrow] (alamb)string kernel to concat_elements. #1752 [arrow] (HaoYang670)cargo install installs not globally #1732 [parquet] (kazuk)min_binary and max_binary aggregate kernels #1725 [arrow] (HaoYang670)null_bit_buffer in ArrayData::try_new() #1714 [arrow] (HaoYang670)generate_unions_case integration test #1713 [arrow] (viirya)offset in substring kernel. #1704 [arrow] (HaoYang670)OffsetSizeTrait to extend num::Integer #1702 [arrow] (HaoYang670)Use bytes in parquet rather than custom Buffer implementation \(\#1474\) \#1683 [parquet] (tustvold)
Breaking changes:
bytes in parquet rather than custom Buffer implementation (#1474) #1683 [parquet] (tustvold)OffsetSize::fn is_large to const OffsetSize::IS_LARGE #1664 [parquet] [arrow] (HaoYang670)StringOffsetTrait and BinaryOffsetTrait #1645 [arrow] (HaoYang670)generate_nested_dictionary_case integration test failure #1636 [arrow] [arrow-flight] (viirya)Implemented enhancements:
DataType::Duration in ffi interface #1688 [arrow]generate_unions_case integration test #1676 [arrow]DictionaryArray support for bit_length kernel #1673 [arrow]DictionaryArray support for length kernel #1672 [arrow]Field #1646 [arrow]StringOffsetTrait and BinaryOffsetTrait #1644 [arrow]UnionArray::from(data: ArrayData) #1643 [arrow]pub fn offsets_buffer, pub fn types_ids_bufferand pub fn data_buffer for ArrayDataBuilder #1640 [arrow]generate_nested_dictionary_case integration test failure for Rust cases #1635 [arrow]ArrowWriter row group flush in public API #1626 [parquet]substring support for FixedSizeBinaryArray #1618 [arrow]UnionArrays #1594 [arrow]length kernel #1489 [arrow]DecimalType support in new_null_array #1659 [arrow] (yjshen)Fixed bugs:
UnionArray::is_null incorrect #1625 [arrow]arrow::async_reader #1617 [parquet]Documentation updates:
Closed issues:
OffsetSizeTrait::IS_LARGE as a const value #1658OffsetSizeTraits? #1638UnionArray Changes DataType #1595Merged pull requests:
GenericListArray::get_type #1650 [arrow] (HaoYang670)UnionArrays #1648 [arrow] (tfeda)dict_id and dict_is_ordered from equality comparison of Field #1647 [arrow] (viirya)substring support for FixedSizeBinaryArray #1633 [arrow] (HaoYang670)async into doc features #1349 [parquet] (HaoYang670)Update parquet::LogicalType to be more idomatic \#1612 [parquet] (tfeda)
Breaking changes:
parquet::basic::LogicalType to be more idomatic #1612 [parquet] (tfeda)ArrayData, UnionArray, and MapArray #1589 [arrow] (tustvold)&Option<T> with Option<&T> in several arrow and parquet APIs #1571 [parquet] [arrow] (tfeda)Implemented enhancements:
BinaryArray in substring kernel #1593 [arrow]Clone for JSON DecoderOptions #1580 [arrow]substring kernel #1575 [arrow]DataType::Null in cast kernel #1572 [arrow] (WinkerDu)Fixed bugs:
force_validate is active) #1587 [parquet]ipc::reader::tests::projection_should_work fails validation #1548 [arrow]Documentation updates:
Closed issues:
&Option<T> with Option<&T> in some APIs #1556 [parquet] [arrow]parquet::basic::LogicalType #1554 [parquet]substring function as unsafe and rename it. #1541 [arrow]Merged pull requests:
div_ceil #1629 [arrow] (HaoYang670)substring support for binary #1608 [arrow] (HaoYang670)ParquetTypeConverter #1592 [parquet] (viirya)Clone and PartialEq for json DecoderOptions #1581 [arrow] (alamb)substring #1577 [arrow] (HaoYang670)Option<T> rather than Option<&T> for copy types in substring kernel #1576 [arrow] (tustvold)projection_should_work #1573 [arrow] (viirya)Add ArrowReaderOptions to ParquetFileArrowReader, add option to skip decoding arrow metadata from parquet \(\#1459\) \#1558 [parquet] (tustvold)
Breaking changes:
ArrowReaderOptions to ParquetFileArrowReader, add option to skip decoding arrow metadata from parquet (#1459) #1558 [parquet] (tustvold)RecordBatch with zero columns but non zero row count, add field to RecordBatchOptions (#1536) #1552 [arrow] (tustvold)DecoderOptions #1539 [arrow] (alamb)prost, prost-derive and prost-types to 0.10, tonic, and tonic-build to 0.7 #1510 [arrow-flight] (alamb)DecoderOptions and support custom format_string for each field #1451 [arrow] (sum12)Implemented enhancements:
FixedSizeBinary in the Arrow C data interface #1553 [arrow]ParquetRecordBatchReader #1537 [parquet]RecordBatch with zero columns but non zero row count #1536 [arrow]Date32/Date64<--> String/LargeString in cast kernel #1535 [arrow]Vec or String #1516 [arrow]substring kernel #1511 [arrow]Fixed bugs:
filter produces invalid sparse UnionArrays #1547 [arrow]GenericListBuilder is not exposed. #1518 [arrow]substring kernel panics when chars > U+0x007F #1478 [arrow]Documentation updates:
substring kernel #1529 [arrow] (HaoYang670)GenericListBuilder #1525 [arrow] (comath)take kernel documentation #1524 [arrow] (alamb)Closed issues:
min_max_helper #1400Merged pull requests:
into_buffers for UnionArray #1567 [arrow] (viirya)MapArray into separate module (#1483) #1562 [parquet] (tustvold)ParquetRecordBatchReader #1560 [parquet] (tustvold)StreamReader #1550 [arrow] (dispanser)cast kernel #1534 [arrow] (yjshen)remove-old-releases.sh executable #1522 (alamb)sort kernel #1519 [arrow] (HaoYang670)substring kernel by about 2x #1512 [arrow] (HaoYang670)new_from_strings to create MapArrays #1507 [arrow] (viirya)Implement size_hint and ExactSizedIterator for DecimalArray \#1505 [arrow]
Implemented enhancements:
size_hint and ExactSizedIterator for DecimalArray #1505 [arrow]StringArray #1493 [arrow]length kernel support for ListArray #1470 [arrow]BinaryArrays #1464 [arrow]DictionaryArray::try_new() #1313 [arrow]Fixed bugs:
Documentation updates:
BinaryArray and LargeBinaryArray #1455 [arrow]GenericBinaryArray::take_iter_unchecked #1454 [arrow]BinaryOffsetSizeTrait #1453 [arrow]FixedSizeBinaryArray is confusing. #1452 [arrow]BinaryArray and LargeBinaryArray #1471 [arrow] (HaoYang670)Closed issues:
packed_simd v.s. portable_simd, which should be used? #1492Merged pull requests:
size_hint and ExactSizedIterator for DecimalArray #1506 [arrow] (alamb)StringArray::num_chars for calculating number of characters #1503 [arrow] (HaoYang670)try_from_trusted_len_iter #1497 [arrow] (jhorstmann)length kernel support for List Array #1488 [arrow] (HaoYang670)Decimal data type #1487 [arrow] (yjshen)length kernel on Binary Array #1465 [arrow] (HaoYang670)Replace filter_row_groups with ReadOptions in parquet SerializedFileReader \#1389 [parquet] (yjshen)
Breaking changes:
filter_row_groups with ReadOptions in parquet SerializedFileReader #1389 [parquet] (yjshen)IPC Reader file / streams #1339 [arrow] [arrow-flight] (Dandandan)Implemented enhancements:
ListArray and LargeListArray more readable #1433if and abs in shift() #1427FixedSizeBinaryArray #1419shift #1408UnionArrays #1394 [arrow]FixedSizeListArrayss #1393 [arrow]rust-version to 1.59 #1377Fixed bugs:
GenericListArray is confusing #1424Documentation updates:
GenericListArray #1450 [arrow] (HaoYang670)Merged pull requests:
ListArray and LargeListArray #1447 [arrow] (HaoYang670)filter kernel should work with FixedSizeListArrays #1434 [arrow] (viirya)FixedSizeListArray #1426 [arrow] (HaoYang670)default-features = false for zstd in the parquet crate to support wasm32-unknown-unknown #1414 [parquet] (kylebarron)UnionArray infilter kernel #1412 [arrow] (viirya)shift #1409 [arrow] (HaoYang670)FixedSizeBinaryArray input all none #1405 [arrow] (jackwener)Remove existing has\_ methods for optional fields in ColumnChunkMetaData \#1346 [parquet] (shanisolomon)
Breaking changes:
ColumnChunkMetaData #1346 [parquet] (shanisolomon)has_ methods in ColumnChunkMetaData #1345 [parquet] (shanisolomon)Implemented enhancements:
IpcWriteOptions #1381 [arrow]MapArray in filter kernel #1378 [arrow]week temporal kernel #1375 [arrow]compare_dict_op #1371 [arrow]arrow::array::builder::MapBuilder public #1354 [arrow]StructArray::from #1351 [arrow]RecordBatch::validate_new_batch #1350 [arrow]write method to JsonWriter #1340 [arrow]Bitmap::new #1337 [arrow]compare_dict_op #1329 [arrow]as_decimal_array(arr: &dyn Array) -> &DecimalArray #1312 [arrow]eq_dyn, neq_dyn, lt_dyn, lt_eq_dyn, gt_dyn, gt_eq_dyn #1201 [arrow]Fixed bugs:
cargo clippy fails on the master branch #1362 [arrow]ArrowArray::try_from_raw should not assume the pointers are from Arc #1333 [arrow]with_datetime_format() to CSV WriterBuilder #1272 [arrow]Performance improvements:
Closed issues:
ColumnChunkMetadata #1332Merged pull requests:
day and month in temporal.rs #1388 [arrow] (Ted-Jiang)Clone for IpcWriteOptions #1382 [arrow] (matthewmturner)week in temporal.rs #1376 [arrow] (Ted-Jiang)min_max_string #1374 [arrow] (HaoYang670)take_iter_unchecked #1372 [arrow] (viirya)Option) #1367 [arrow] (viirya)RecordBatch::validate_new_batch #1361 [arrow] (HaoYang670)StructArray::from #1360 [arrow] (HaoYang670)as_decimal_array function #1356 [arrow] (liukun4515)Bitmap::new #1343 [arrow] (HaoYang670)ArrowArray::try_from_raw should not assume the pointers are from Arc #1334 [arrow] (viirya)compare_dict_op #1330 [arrow] (viirya)Exposing page encoding stats \#1321
Implemented enhancements:
DeltaBitPackDecoder #1281 [parquet]ColumnChunkMetadata #1322 [parquet] (shanisolomon)ColumnChunkMetadata #1318 [parquet] (shanisolomon)ColumnChunkMetadata #1309 [parquet] (shanisolomon)DictionaryArray::try_new() to create dictionaries from pre existing arrays #1300 [arrow] (alamb)DictionaryArray::keys_iter, and take_iter for other array types #1296 [arrow] (viirya)rle decoder public under experimental feature #1271 [parquet] (zeevm)DictionaryArray support in eq_dyn kernel #1263 [arrow] (viirya)Fixed bugs:
len is not a parameter of MutableArrayData::extend #1316data_type is private in Rust Parquet 8.0.0 #1302 [parquet]Documentation updates:
Performance improvements:
compute module (up to 10x faster) #1248 [parquet] [arrow] (tustvold)Closed issues:
DictionaryArrays from Key and Value arrays #1299DictionaryArray #1295 [arrow]BitReader #1282Merged pull requests:
MutableArrayData::extend #1336 [arrow] (HaoYang670)dead_code #1324 [arrow] (gsserge)to_thrift #1320 [parquet] (shanisolomon)clippy::type_complexity #1310 [arrow] (gsserge)clippy::float_equality_without_abs #1305 [arrow] (gsserge)clippy::vec_init_then_push lint #1303 [arrow] (gsserge)LevelEncoder variants to conform to Rust standards #1273 [parquet] (HaoYang670)DecimalArray API ergonomics: add iter(), FromIterator, with_precision_and_scale #1223 [arrow] (alamb)Add Send + Sync to DataType, RowGroupReader, FileReader, ChunkReader. \#1264
Breaking changes:
Send + Sync to DataType, RowGroupReader, FileReader, ChunkReader. #1264Bitmap::len to Bitmap::bit_len to clarify its meaning #1242 [parquet] [arrow] (HaoYang670)memory-check feature #1222 [arrow] (jhorstmann)RecordBatches before writing a parquet row group in ArrowWriter #1214 [parquet] [arrow] (tustvold)Implemented enhancements:
async arrow parquet reader #1154 [parquet] [arrow] (tustvold)Bitmap::len to Bitmap::bit_len #1233eq_dyn etc. #1202 [arrow]BinaryArray #1108impl ArrowNativeType for i128 #1098Copy trait bound from dyn scalar kernels #1243 [arrow] (matthewmturner)into_inner for IPC FileWriter #1236 [arrow] (yjshen)array::builder::make_builder to make it available for downstream #1235 [arrow] (yjshen)Fixed bugs:
Unknown configuration option rust-version when running the rust format command #1240Bitmap Length Validation is Incorrect #1231 [arrow]ListArray or MapArray ignore offsets #1226 [parquet]memory-tracking crate feature #1171parquet::data_type and parquet::arrow::schema experimental #1244 [parquet] (tustvold)Documentation updates:
Performance improvements:
simd feature enabled (except for division/modulo) #1221 [arrow] (jhorstmann)Closed issues:
UnalignedBitChunkIterator to that iterates through already aligned u64 blocks #1227ArrowArrayReader in parquet #1197 [parquet]Merged pull requests:
Return error from JSON writer rather than panic \#1205 [arrow] (Ted-Jiang)
Breaking changes:
ArrowSignedNumericType to Simplify and reduce code duplication in arithmetic kernels #1161 [arrow] (jhorstmann)Implemented enhancements:
arbitrary_precision feature flag #1174 [arrow]Err from JSON writer rather than panic! for unsupported types #1157 [arrow]scalar mathematics kernels for Array and scalar value #1153 [arrow]DecimalArray in sort kernel #1137from_iter_values for binary array #1188 [arrow] (Jimexist)MapArray in json writer #1149 [arrow] (helgikrs)Fixed bugs:
RecordBatch containing Float16 triggers a panic #1193 [arrow]GenericStringArray::from_iter_values if reported iterator upper bound is incorrect #1144 [arrow]simd feature asserts #1136 [arrow]Security fixes:
Documentation updates:
from_trusted_len_iter #1148 [arrow] (alamb)Performance improvements:
Closed issues:
Merged pull requests:
Float16Array #1194 [arrow] (helgikrs)sort and take kernels #1172 [arrow] (liukun4515)GenericStringArray::from_vec and GenericStringArray::from_opt_vec #1147 [arrow] (alamb)pretty_format_batches now returns Result rather than String: #975
Breaking changes:
pretty_format_batches now returns Result<impl Display> rather than String: #975MutableBuffer::typed_data_mut is marked unsafe: #1029UnionMode, UnionArray::new() marked unsafe: #885New Features:
Float16Array types #888UnionArray #654eq_dyn_scalar), including DictionaryArray: #1113Enhancements:
Schema::with_metadata and Field::with_metadata #1092Array for ArrayRef for easier use #1129FixedSizeBinaryArray #1097pyo3, parquet-format, prost, toniclexicographical_partition_ranges#998Fixed bugs:
049f48559f578243935b6e512d06c4c2df360bf1 Force new cargo and target caching to fix CI
rc feature of serde (#990) (#1016)7e51df015ce851a5de444ca08b57b38e7ee959a3 add more error test case and change the code style
Changes:
4037933e43cad9e4de027039ce14caa65f78300a Fix validation for offsets of StructArrays
Features / Fixes:
b42649b0088fe7762c713a41a23c1abdf8d0496d implement eq_dyn and neq_dyn
Features / Fixes:
Other:
Replace ArrayData::new() with ArrayData::try_new() and unsafe ArrayData::new_unchecked \#822 [parquet] [arrow] (alamb)
Breaking changes:
ArrayData::new() with ArrayData::try_new() and unsafe ArrayData::new_unchecked #822 [parquet] [arrow] (alamb)Implemented enhancements:
ArrowWriter #788Fixed bugs:
minute and second temporal kernels do not respect timezone #500Documentation updates:
parquet should depend on a small set of arrow features \#800
Implemented enhancements:
Fixed bugs:
Upgrade lexical-core to 0.8 \#747
Implemented enhancements:
append_nulls and append_trusted_len_iter for PrimitiveBuilder #725Fixed bugs:
unary kernel doesn't respect offset #738new_null_array creates invalid struct arrays #734Bitmap::len returns the number of bytes, not bits. #730Closed issues:
Add optimized filter kernel for regular expression matching \#697
Implemented enhancements:
Fixed bugs:
Closed issues:
Make rand an optional dependency \#671
Implemented enhancements:
value method of boolean and primitive arrays #645Fixed bugs:
Closed issues:
Make FFI\_ArrowArray empty\(\) public \#602
Implemented enhancements:
limit if sorted unstably #553Fixed bugs:
~/.cargo is in a git checkout #589array::ffi::tests::test_struct and other ffi tests #580Documentation updates:
Closed issues:
Remove lifetime from DynComparator \#543 [arrow]
Breaking changes:
Implemented enhancements:
RecordBatch::concat #537 [arrow] (silathdiir)BitReader/DeltaBitPackDecoder #325 [parquet] (kornholi)Fixed bugs:
ahash v0.7.4 depends on itself. Cycle #544default-features=false #529Documentation updates:
Merged pull requests:
StringArray creation from String Vec #522 [arrow] (silathdiir)ArrayData equality and update unit tests #472 (Jimexist)SliceableCursor to be constructed from an Arc directly #369 (crepererum)take #281 (Dandandan)Security Vulnerabilities: flatbuffers: read_scalar and read_scalar_at allow transmuting values without unsafe blocks \#476
Breaking changes:
Implemented enhancements:
Fixed bugs:
read_scalar and read_scalar_at allow transmuting values without unsafe blocks #476Security fixes:
Closed issues:
Add partitioning kernel for sorted arrays \#428 [arrow]
Implemented enhancements:
Fixed bugs:
One column per quarter.