PackageTrack
Sign in Get early access

arrow2

Unofficial implementation of Apache Arrow spec in safe Rust

0.18.0 7.3M downloads/mo #3693 most downloaded on crates.io jorgecarleitao/arrow2

What this package is like to depend on

Last release 3 years ago

no release in 18 months

Ships fairly regularly

a new release about every 6 weeks

Most releases are documented

notes for 27 of 38 stable releases

Nothing withdrawn

no release was ever pulled

5 years old

38 releases · first in 2021

0 releases in the last 12 months

see the full history below

Release timeline

38 releases · Jul 2021 to Sep 2023
2022 2023 2024 2025 2026
Release Pre-release

Releases

latest 38
  1. 0.18.0 17 Sep 2023

    Nothing published for this version

  2. 0.17.4 14 Aug 2023

    Nothing published for this version

  3. 0.17.3 15 Jul 2023

    Nothing published for this version

  4. 0.17.2 28 May 2023

    Nothing published for this version

  5. 0.17.1 08 May 2023

    Nothing published for this version

  6. 0.17.0 27 Mar 2023
    Release notes

    What's Changed

    New Contributors

    Full Changelog: v0.16.0...v0.17.0

    Open source →
    Release notes

    Full Changelog

    Breaking changes:

    New features:

    Fixed bugs:

    • parquet_read panics when working with date64s #1400
    • Round Trip [Rust -> arrow2_convert -> Arrow -> Parquet -> Arrow -> Rust] #1376
    • Parquet writes incorrect List<u32> #1368
    • Slicing nullable list arrays into multiple parquet pages doesn't work #1356
    • Reading parquet file with multiple row groups and nested nullable struct types panics #1249
    • Changed encoded float::Inf as null in json #1427 (SimonSchneider)
    • Fixed statistics writing flag and correct null_count in dictionaries #1414 (ritchie46)
    • Fixed ahash dependency for wasm #1407 (hzuo)
    • Fixed writing of sliced arrays to parquet #1397 (jorgecarleitao)
    • Fixed writing nested parquet #1390 (jorgecarleitao)

    Enhancements:

    Documentation updates:

    Open source →
  7. 0.16.0 09 Feb 2023
    Release notes

    A new release is here! Thank you everyone that contributed to it! 🙇

    Full Changelog

    Breaking changes:

    Fixed bugs:

    • invalid written parquet file of nested structures. (Mixing list with structs) #1325
    • Fix incorrect downcast in estimated_size_bytes #1351 (jleibs)
    • fix(parquet): nested struct /list writing #1347 (ritchie46)
    • Fixed csv infer_schema on empty fields #1342 (tripokey)

    Enhancements:

    Documentation updates:

    Testing updates:

    New Contributors

    Open source →
    Release notes

    Full Changelog

    Breaking changes:

    Fixed bugs:

    • invalid written parquet file of nested structures. (Mixing list with structs) #1325
    • Fix incorrect downcast in estimated_size_bytes #1351 (jleibs)
    • fix(parquet): nested struct /list writing #1347 (ritchie46)
    • Fixed csv infer_schema on empty fields #1342 (tripokey)

    Enhancements:

    Documentation updates:

    Testing updates:

    Open source →
  8. 0.15.0 18 Dec 2022
    Release notes

    A new release is here, adding a number of new features and improvements to arrow2. Thank you to everyone that contributed to it!

    This release adds support to a new format, the "record" JSON format, contributed by @AnIrishDuck, a new trait TryExtendFromSelf to efficiently concatenate an array into an existing mutable array, and multiple improvements by @sundy-li and @ritchie46 to performance. Finally, we have a new API OffsetsBuffer and Offsets proposed by @ritchie46 to allow creating variable sized-arrays without having to check for offsets.

    This release also features a number of contributions from first contributors:

    Thank you everyone for the great work this year, and happy festivities everyone!

    Full Changelog

    Breaking changes:

    New features:

    Fixed bugs:

    • Parquet writes all values of sliced arrays? #1323
    • Avro schema: Invalid record names #1269
    • Fixed writing nested/sliced arrays to parquet #1326 (ritchie46)
    • Fixed failing to accept dictionary full of nulls #1312 (ritchie46)
    • Added support for Extension types in ffi #1300 (jondo2010)
    • Fixed error in memory usage of sliced binary/list/utf8arrays #1293 (ritchie46)
    • Fixed descending ordering when specify nulls first #1286 (sandflee)
    • Added avro record names when converting arrow schema to avro #1279 (Samrose-Ahmed)

    Enhancements:

    Documentation updates:

    Open source →
    Release notes

    Full Changelog

    Breaking changes:

    New features:

    Fixed bugs:

    • Parquet writes all values of sliced arrays? #1323
    • Avro schema: Invalid record names #1269
    • Fixed writing nested/sliced arrays to parquet #1326 (ritchie46)
    • Fixed failing to accept dictionary full of nulls #1312 (ritchie46)
    • Added support for Extension types in ffi #1300 (jondo2010)
    • Fixed error in memory usage of sliced binary/list/utf8arrays #1293 (ritchie46)
    • Fixed descending ordering when specify nulls first #1286 (sandflee)
    • Added avro record names when converting arrow schema to avro #1279 (Samrose-Ahmed)

    Enhancements:

    Documentation updates:

    Open source →
  9. 0.14.2 05 Oct 2022
    Release notes

    Full Changelog

    New features:

    Fixed bugs:

    • Unnecessary println in library code #1263

    Testing updates:

    Open source →
  10. 0.14.1 27 Sep 2022
    Release notes

    A couple of backward-compatible bug fixes and improvements that everyone benefits from :)

    Thank you @cjermain, @shaeqahmed and @ozgrakkurt! 🙇

    Full Changelog

    Fixed bugs:

    Enhancements:

    Testing updates:

    Open source →
    Release notes

    Full Changelog

    Fixed bugs:

    Enhancements:

    Testing updates:

    Open source →
  11. 0.14.0 12 Sep 2022
    Release notes

    Another release of arrow2 is here!

    Besides API improvements to reading IPC and parquet, there are two main new features, the ability to memory map arrow files (check out https://jorgecarleitao.github.io/arrow2/v0.14.0/guide/io/ipc_mmap.html) and support for decimal 256.

    The following had their first time contribution to their crate:

    Thank you everyone for all the issues, PRs and ideas!

    Full Changelog

    Breaking changes:

    New features:

    Fixed bugs:

    Enhancements:

    Documentation updates:

    Testing updates:

    Open source →
    Release notes

    Full Changelog

    Breaking changes:

    New features:

    Fixed bugs:

    Enhancements:

    Documentation updates:

    Testing updates:

    Open source →
  12. 0.13.1 04 Aug 2022
    Release notes

    Full Changelog

    Thanks @daniel-martinez-maqueda-sap!

    Fixed bugs:

    Open source →
  13. 0.13.0 31 Jul 2022
    Release notes

    A new version (0.13) is now available on crates.io! 🎉🎉🎉🎉

    This is another large release of arrow2. Among the many, many changes (see below), it is worth noting:

    • Added copy-on-write API to perform operations in place, improving performance of expressions like (a + b) * 2 by a factor of 2-10x
    • Added support to read from Apache ORC format
    • Added support for projection and limit pushdown when reading from Arrow IPC format
    • Added support for f16

    Thank you to the numerous contributors, both via PRs and issues, that resulted in this fantastic release 🙇

    Breaking changes:

    New features:

    Fixed bugs:

    Enhancements:

    Documentation updates:

    Testing updates:

    Open source →
    Release notes

    Full Changelog

    Breaking changes:

    New features:

    Fixed bugs:

    Enhancements:

    Documentation updates:

    Testing updates:

    Open source →
  14. 0.12.0 05 Jun 2022
    Release notes

    A new version of arrow2 is now available in crates.io. 🎉🎉🎉

    See below all great things that were released 🚀. But before that, thank you so much to everyone that contributed to this release: 🙇

    @ahmedriza, @dexterduck, @GPSnoopy, @HaoYang670, @SimonSchneider, @TurnOfACard, @aptr322, @arxra, @b41sh, @cjermain, @dbr, @jorgecarleitao, @ritchie46

    Breaking changes:

    New features:

    Fixed bugs:

    • Writing of ListArray does not preserve all values #1008
    • Write a two-dimensional list to parquet file failed #992
    • Writing to Parquet fails for extension types that contain lists #830
    • Fixed using lower limit than size of first parquet row group #1046 (arxra)
    • Fixed error in consuming sliced FixedSizedBinary from c data interface (FFI) #1026 (jorgecarleitao)
    • Fixed lexsort limit equal or greater than row_count #1021 (b41sh)
    • Fixed error in reading nested parquet structs #1015 (jorgecarleitao)
    • Fixed panic on debug print of invalid timezones #1013 (jorgecarleitao)
    • Treat empty timezone string as no-timezone #1009 (dbr)
    • Fixed encoding of NaN to json #990 (SimonSchneider)
    • Fixed error in writing ListArray to parquet #984 (jorgecarleitao)
    • Fixed decoding Binary Plain pages with dictionary pages #982 (aptr322)

    Enhancements:

    Documentation updates:

    Testing updates:

    Open source →
    Release notes

    Full Changelog

    Breaking changes:

    New features:

    Fixed bugs:

    • Writing of ListArray does not preserve all values #1008
    • Write a two-dimensional list to parquet file failed #992
    • Writing to Parquet fails for extension types that contain lists #830
    • Fixed using lower limit than size of first parquet row group #1046 (arxra)
    • Fixed error in consuming sliced FixedSizedBinary from c data interface (FFI) #1026 (jorgecarleitao)
    • Fixed lexsort limit equal or greater than row_count #1021 (b41sh)
    • Fixed error in reading nested parquet structs #1015 (jorgecarleitao)
    • Fixed panic on debug print of invalid timezones #1013 (jorgecarleitao)
    • Treat empty timezone string as no-timezone #1009 (dbr)
    • Fixed encoding of NaN to json #990 (SimonSchneider)
    • Fixed error in writing ListArray to parquet #984 (jorgecarleitao)
    • Fixed decoding Binary Plain pages with dictionary pages #982 (aptr322)

    Enhancements:

    Documentation updates:

    Testing updates:

    Open source →
  15. 0.11.2 05 May 2022
    Release notes

    Full Changelog

    New features:

    Fixed bugs:

    Enhancements:

    Open source →
    Release notes

    Full Changelog

    New features:

    Fixed bugs:

    Enhancements:

    Open source →
  16. 0.11.1 27 Apr 2022
    Release notes

    Full Changelog

    Fixed bugs:

    Open source →
  17. 0.11.0 27 Apr 2022
    Release notes

    Full Changelog

    Breaking changes:

    New features:

    Fixed bugs:

    • Parquet regression: exceptions.ArrowErrorException: NotYetImplemented("Can't read Dictionary(UInt32, LargeUtf8, false) from parquet") #955
    • Reading Parquet binary column panics during deserialization 'attempt to subtract with overflow` #944
    • Reading Parquet file written by pyarrow with lz4 compression fails with OutOfSpec("Thrift out of range") #940
    • Issues when trying to create a parquet file with FixedSizedListArray #691
    • Fixed bug in writing csv with buffer resizing #965 (ritchie46)
    • Fixed bug in reading binary parquet #945 (jorgecarleitao)
    • Fixed error in writing fixedSizeListArray to parquet #941 (jorgecarleitao)
    • Fixed support to read dict nested binary parquet #924 (jorgecarleitao)

    Enhancements:

    Documentation updates:

    Testing updates:

    Open source →
  18. 0.10.1 16 Mar 2022
    Release notes

    Full Changelog

    New features:

    Fixed bugs:

    Enhancements:

    Testing updates:

    Open source →
  19. 0.10.0 12 Mar 2022
    Release notes

    Full Changelog

    Breaking changes:

    New features:

    Fixed bugs:

    Enhancements:

    Documentation updates:

    Testing updates:

    Open source →
  20. 0.9.2 04 Mar 2022

    Nothing published for this version

  21. 0.9.1 19 Jan 2022
    Release notes

    Full Changelog

    New features:

    Fixed bugs:

    Enhancements:

    Testing updates:

    Open source →
  22. 0.9.0 14 Jan 2022
    Release notes

    Full Changelog

    Breaking changes:

    New features:

    Fixed bugs:

    Enhancements:

    Documentation updates:

    Testing updates:

    Open source →
  23. 0.8.2 04 Mar 2022

    Nothing published for this version

  24. 0.8.1 27 Nov 2021
    Release notes

    Full Changelog

    Fixed bugs:

    • Fixed compilation with individual features activated #642 (ritchie46)
    Open source →
  25. 0.8.0 27 Nov 2021
    Release notes

    Full Changelog

    Breaking changes:

    New features:

    Fixed bugs:

    Enhancements:

    Documentation updates:

    Testing updates:

    Open source →
  26. 0.7.1 04 Mar 2022

    Nothing published for this version

  27. 0.7.0 29 Oct 2021
    Release notes

    Full Changelog

    Breaking changes:

    New features:

    Fixed bugs:

    Enhancements:

    Documentation updates:

    Testing updates:

    Open source →
  28. 0.6.2 09 Oct 2021
    Release notes

    Full Changelog

    New features:

    • Added wrapping version arithmetics for PrimitiveArray #496 (yjhmelody)

    Fixed bugs:

    Enhancements:

    Open source →
  29. 0.6.1 07 Oct 2021
    Release notes

    Full Changelog

    Breaking changes:

    • Bring MutableFixedSizeListArray to the spec used by the rest of the Mutable API #475
    • Removed ALIGNMENT invariant from [Mutable]Buffer #449
    • Un-nested compute::arithemtics::basic #461 (jorgecarleitao)
    • Added more serialization options for csv writer. #453 (ritchie46)
    • Changed validity from &Option<Bitmap> to Option<&Bitmap>. #431 (jorgecarleitao)
    • Bumped parquet2 #422 (jorgecarleitao)
    • Changed IPC FileWriter to own the writer. #420 (yjshen)
    • Made DynComparator Send+Sync #414 (yjshen)

    New features:

    Fixed bugs:

    Enhancements:

    Documentation updates:

    Testing updates:

    Open source →
  30. 0.6.0 07 Oct 2021

    Nothing published for this version

  31. 0.5.3 14 Sep 2021
    Release notes

    Full Changelog

    New features:

    • Added support to read and write extension types to and from parquet #396 (jorgecarleitao)

    Fixed bugs:

    Enhancements:

    • Added support to read dict-encoded required primitive types from parquet #402 (Dandandan)
    • Added Array::with_validity #399 (ritchie46)

    Testing updates:

    Open source →
  32. 0.5.2 09 Sep 2021

    Nothing published for this version

  33. 0.5.1 09 Sep 2021
    Release notes

    Full Changelog

    Documentation updates:

    • [0.5] The docs io module has no submodules #390
    • Made docs be compiled with feature full #391 (jorgecarleitao)
    Open source →
  34. 0.5.0 08 Sep 2021
    Release notes

    Full Changelog

    Breaking changes:

    • Added Extension to DataType #361
    • MonthDayNano added to enum IntervalUnit #360
    • Make io::parquet::write::write_* return size of file in bytes #354
    • Renamed bitmap::utils::null_count to bitmap::utils::count_zeros #342
    • Made GroupFilter optional in parquet'sRecordReader and added method to set it. #386 (jorgecarleitao)
    • Removed PartialOrd and Ord of all enums in datatypes #379 (jorgecarleitao)
    • Made cargo features not default #369 (jorgecarleitao)
    • Prepare APIs for extension types #357 (jorgecarleitao)

    New features:

    Fixed bugs:

    • Parquet read skips a few rows at the end of the page #373
    • parquet_read fails when a column has too many rows with string values #366
    • parquet_read panics with index_out_of_bounds #351
    • Fixed error in MutableBitmap::push_unchecked #384 (jorgecarleitao)
    • Fixed display of timestamp with tz. #375 (jorgecarleitao)

    Enhancements:

    Documentation updates:

    Testing updates:

    Open source →
  35. 0.4.0 24 Aug 2021
    Release notes

    Full Changelog

    Breaking changes:

    • Change dictionary iterator of values from Arrays of one element to Scalars #335
    • Align FFI API with arrow's C++ API #328
    • Make *_compare_scalar not return Result #316
    • Make io::print, get_value_display and get_display not return Result #286
    • Add MetadataVersion to IPC interfaces #282
    • Change DataType::Union to enable round trips in IPC #281
    • Removed clone requirement in StructArray -> RecordBatch #307 (jorgecarleitao)
    • Fixed error in reading a non-finished IPC stream. #302 (jorgecarleitao)
    • Generalized ZipIterator to accept a BitmapIter #296 (jorgecarleitao)

    New features:

    Fixed bugs:

    Enhancements:

    Documentation updates:

    Testing updates:

    Closed issues:

    • Make parquet_read_record support async #331
    • Panic due to SIMD comparison #312
    • Bitmap::mutable line 155 may Panic/segfault #309
    • IPC's StreamReader may abort due to excessive memory by overflowing a usized variable #301
    • Improve performance of rem_scalar/div_scalar for integer types (4x-10x) #259
    Open source →
  36. 0.3.0 11 Aug 2021
    Release notes

    Full Changelog

    Breaking changes:

    • Renamed sum to sum_primitive #273
    • Moved trait Index from array::Index to types::Index #272
    • Added optional projection to IPC FileReader #271
    • Added optional page_filter to parquet's RecordReader and get_page_iterator #270
    • Renamed parquets' CompressionCodec to Compression #269

    New features:

    Fixed bugs:

    Enhancements:

    • Improve the performance in cast kernel using AsPrimitive trait in generic dispatch #252
    • Poor performance in sort::sort_to_indices with limit option in arrow2 #245
    • Support loading Feather v2 (IPC) files with more than 1 million tables #231
    • Migrated to parquet2 v0.3 #265 (jorgecarleitao)
    • Added more tests to cast and min/max #253 (jorgecarleitao)
    • Prettytable is unmaintained. Change to comfy-table #251 (PsiACE)
    • Added IndexRange to remove checks in hot loops #247 (jorgecarleitao)
    • Make merge_sort_slices MergeSortSlices public #243 (sundy-li)

    Documentation updates:

    Closed issues:

    • Allow projection pushdown to IPC files #261
    • Add support to write dictionary-encoded pages #211
    • Make IpcWriteOptions easier to find. #120
    Open source →
  37. 0.2.0 30 Jul 2021
    Release notes

    Full Changelog

    Breaking changes:

    New features:

    Fixed bugs:

    Enhancements:

    • Avoid materialization of indices in filter_record_batch for single arrays #234
    • Add integration tests for writing to parquet #80
    • Short-circuited boolean evaluation in GrowableList #228 (ritchie46)
    • Add extra inlining to speed up take #226 (Dandandan)
    • Removed un-needed unsafe #225 (jorgecarleitao)

    Documentation updates:

    • Add documentation to guide #96
    • Add git submodule command to correct the test doc #223 (sundy-li)
    • Added badges to README #216 (sundy-li)
    • Clarified differences with arrow crate #210 (alamb)
    • Clarified differences with arrow crate #209 (alamb)

    * This Changelog was automatically generated by github_changelog_generator

    Open source →
  38. 0.1.0 20 Jul 2021

    Nothing published for this version

Every package, every release, already written down.

The archive is open and free. Watching your own project is what we are building next.

Browse the archive