pyo3-arrow

chore: Bump pyo3-arrow to 0.19 (#508)

Release with pyo3 0.29 and arrow 59.

pyo3-arrow is not part of the workspace, so we can't use -p, which stands for package I think

Release with pyo3 0.28 and arrow 59.

• ci: Fix publish of pyo3-arrow #501

chore: Bump pyo3-arrow 0.17 (#487)

Release with pyo3 0.28 and arrow 58.

chore: Bump pyo3-arrow to 0.16.1 (#485)

• fix: Preserve schema metadata in RecordBatch PyCapsule export (#474)

chore: Prepare pyo3-arrow for 0.16 (#472)

Release with pyo3 0.28 and arrow 57.

• chore: Bump to arrow 57

• update changelog

Release with pyo3 0.27 and arrow 57.

Release with pyo3 0.26 and arrow 57.

Release with pyo3 0.27 and arrow 56.

Breaking :wrench:

• Rename to_pyarrow to into_pyarrow because it consumes self. #391
• Support pyo3 0.26 by @kylebarron in https://github.com/kylebarron/arro3/pull/391

New Features :sparkles:

• Add export::Arro3ArrayReader by @kylebarron in https://github.com/kylebarron/arro3/pull/354

Bug Fixes :bug:

• fix(pyo3-arrow, arro3-core): Preserve exceptions when Arrow dunder method fails by @kylebarron in https://github.com/kylebarron/arro3/pull/374

Documentation :book:

• Add badges for pyo3-arrow by @kylebarron in https://github.com/kylebarron/arro3/pull/371

Other

• chore(pyo3-arrow): Fix clippy warnings in pyo3-arrow by @kylebarron in https://github.com/kylebarron/arro3/pull/376
• chore: Remove PyBuffer workaround with custom drop implementation by @kylebarron in https://github.com/kylebarron/arro3/pull/392
• chore: point numpy dependency at released version by @alex in https://github.com/kylebarron/arro3/pull/393

New Contributors

• @alex made their first contribution in https://github.com/kylebarron/arro3/pull/393

Full Changelog: https://github.com/kylebarron/arro3/compare/pyo3-arrow-v0.11.0...pyo3-arrow-v0.12.0

• Bump to arrow 56.
• Improved array formatting of PyArray, PyChunkedArray, PyRecordBatch, PyScalar, and PyTable in the __repr__ exposed to Python if you re-export these types in your Python library #335.

• Fix pyo3-arrow error when importing buffer protocol object. We were incorrectly validating the stride. But since we already checked the buffer to be C-contiguous, we don't need to check for strides again. (Part of #328).
• Make a workaround to the upstream regression/change in https://github.com/apache/arrow-rs/pull/7247 that caused the test that checks we can import a zero-length record batch to fail. This also makes my code cleaner by utilizing the upstream make_array now that we have a fix to https://github.com/apache/arrow-rs/issues/6151 (Part of #328).

• Bump to pyo3 0.25.

• Bump to arrow 55.
• Minimize arrow crate dependency surface (#318). Don't depend on arrow; instead depend on individual arrow crates for a smaller dependency tree and faster compile times.

Breaking Changes

• A bare DataType is now exported through __arrow_c_schema__ (such as when passing to pyarrow.field) with nullable: true. This is required to match equality semantics with pyarrow. by @kylebarron in #483

Bug Fixes

• fix!: Fix type equality between arro3 and pyarrow by @kylebarron in #483

Full Changelog: py-v0.7.0...py-v0.8.0

Breaking Changes

• A bare DataType is now exported through __arrow_c_schema__ (such as when passing to pyarrow.field) with nullable: true. This is required to match equality semantics with pyarrow. by @kylebarron in https://github.com/kylebarron/arro3/pull/483

Bug Fixes

• fix!: Fix type equality between arro3 and pyarrow by @kylebarron in https://github.com/kylebarron/arro3/pull/483

Full Changelog: https://github.com/kylebarron/arro3/compare/py-v0.7.0...py-v0.8.0

What's Changed

• Bump to pyo3 0.24.

Full Changelog: https://github.com/kylebarron/arro3/compare/pyo3-arrow-v0.7.2...pyo3-arrow-v0.8.0

What's Changed

• Add into_arro3 method https://github.com/kylebarron/arro3/pull/289

Full Changelog: https://github.com/kylebarron/arro3/compare/pyo3-arrow-v0.7.1...pyo3-arrow-v0.7.2

What's Changed

• Bump to pyo3 0.23.4 https://github.com/kylebarron/arro3/pull/287

Full Changelog: https://github.com/kylebarron/arro3/compare/pyo3-arrow-v0.7.0...pyo3-arrow-v0.7.1

Breaking changes

• feat!: Make Table.remove_column return IndexError by @surister in #442
• fix: Rename names to columns parameter in Tables.drop_columns by @surister in #453
• fix: Properly return IndexError on incorrect index in Table.add_column by @surister in #455

Improvements

• ci: Build 3.14t wheels by @kylebarron in #428
• feat: Make Table.append_column support Array by @surister in #433
• feat: Make Table.set_column and Table.add_column support Array and ArrayReader by @surister in #437
• fix: off by one when checking if an index is valid by @surister in #443
• feat: Implement Table.drop_columns by @surister in #440
• fix: Table.rename not modifying internals batches schema by @surister in #457
• fix: Improve error messaging by adding more information by @surister in #447
• feat: Match RecordBatch.from_arrays closely to pyarrow by @AlenkaF in #461

Bug Fixes

• fix: Preserve schema metadata in RecordBatch PyCapsule export by @bretttully in #474
• fix: Update license metadata in pyproject.toml by @kylebarron in #480

Other improvements

• chore: Bump to arrow 57 by @kylebarron in #424
• chore: Bump arro3 to pyo3 0.27 by @kylebarron in #425

New Contributors

• @surister made their first contribution in #433
• @AlenkaF made their first contribution in #461
• @bretttully made their first contribution in #474

Full Changelog: py-v0.6.5...py-v0.7.0

Breaking changes

• feat!: Make Table.remove_column return IndexError by @surister in https://github.com/kylebarron/arro3/pull/442
• fix: Rename names to columns parameter in Tables.drop_columns by @surister in https://github.com/kylebarron/arro3/pull/453
• fix: Properly return IndexError on incorrect index in Table.add_column by @surister in https://github.com/kylebarron/arro3/pull/455

Improvements

• ci: Build 3.14t wheels by @kylebarron in https://github.com/kylebarron/arro3/pull/428
• feat: Make Table.append_column support Array by @surister in https://github.com/kylebarron/arro3/pull/433
• feat: Make Table.set_column and Table.add_column support Array and ArrayReader by @surister in https://github.com/kylebarron/arro3/pull/437
• fix: off by one when checking if an index is valid by @surister in https://github.com/kylebarron/arro3/pull/443
• feat: Implement Table.drop_columns by @surister in https://github.com/kylebarron/arro3/pull/440
• fix: Table.rename not modifying internals batches schema by @surister in https://github.com/kylebarron/arro3/pull/457
• fix: Improve error messaging by adding more information by @surister in https://github.com/kylebarron/arro3/pull/447
• feat: Match RecordBatch.from_arrays closely to pyarrow by @AlenkaF in https://github.com/kylebarron/arro3/pull/461

Bug Fixes

• fix: Preserve schema metadata in RecordBatch PyCapsule export by @bretttully in https://github.com/kylebarron/arro3/pull/474
• fix: Update license metadata in pyproject.toml by @kylebarron in https://github.com/kylebarron/arro3/pull/480

Other improvements

• chore: Bump to arrow 57 by @kylebarron in https://github.com/kylebarron/arro3/pull/424
• chore: Bump arro3 to pyo3 0.27 by @kylebarron in https://github.com/kylebarron/arro3/pull/425

What's Changed

• Add frozen parameter to pyclass macro by @kylebarron in https://github.com/kylebarron/arro3/pull/282
• Avoid returning PyObject and add wrapper types for arro3 export by @kylebarron in https://github.com/kylebarron/arro3/pull/269
• Bump to arrow 54 by @kylebarron in https://github.com/kylebarron/arro3/pull/285

Full Changelog: https://github.com/kylebarron/arro3/compare/pyo3-arrow-v0.6.0...pyo3-arrow-v0.7.0

arro3-core

Breaking Changes :wrench:

No breaking changes.

New features :sparkles:

• Expanded dtype support in Array.from_numpy. It now additionally supports datetime64, timedelta64, fixed-width strings and bytes, variable-width strings, and object dtypes that contain str or bytes.
• Support FixedSizeBinary in Array constructor by @kylebarron in https://github.com/kylebarron/arro3/pull/358
• Add mask parameter to list/struct constructors by @kylebarron in https://github.com/kylebarron/arro3/pull/382

Bug fixes :bug:

• Preserve exceptions when Arrow PyCapsule dunder method fails by @kylebarron in https://github.com/kylebarron/arro3/pull/374

Updated dependencies :arrow_up:

• Use Rust arrow 56 by @kylebarron in https://github.com/kylebarron/arro3/pull/368. This includes all changes in the upstream Changelog.

Full Changelog: https://github.com/kylebarron/arro3/compare/py-v0.5.1...py-v0.6.0

What's Changed

• Bump pyo3-arrow to pyo3 0.23 by @kylebarron in https://github.com/kylebarron/arro3/pull/265
• Add test for segmentation fault when converting arro3.core.Array to pyarrow.Array at interpreter exit by @3ok in https://github.com/kylebarron/arro3/pull/236
• Implement FromPyObject for PyArrowBuffer by @kylebarron in https://github.com/kylebarron/arro3/pull/241
• rust constructor for PyArrowBuffer by @kylebarron in https://github.com/kylebarron/arro3/pull/242

New Contributors

Full Changelog: https://github.com/kylebarron/arro3/compare/pyo3-arrow-v0.5.1...pyo3-arrow-v0.6.0

What's Changed

• chore: Use released pyo3-file by @kylebarron in https://github.com/kylebarron/arro3/pull/341
• feat: Support serializing binary view and string view array types to numpy by @kylebarron in https://github.com/kylebarron/arro3/pull/348
• fix(arro3-core): Add typing-extensions as a dependency for arro3-core by @kylebarron in https://github.com/kylebarron/arro3/pull/350
• feat(arro3-core): Improved repr for Array and ChunkedArray by @kylebarron in https://github.com/kylebarron/arro3/pull/351
• chore: bump dev versions for geoarrow-types and pyarrow by @kylebarron in https://github.com/kylebarron/arro3/pull/352

Full Changelog: https://github.com/kylebarron/arro3/compare/py-v0.5.0...py-v0.5.1

What's Changed

• Fix no-default-features for pyo3-arrow by @kylebarron in https://github.com/kylebarron/arro3/pull/232
• Custom drop on PyBufferWrapper by @kylebarron in https://github.com/kylebarron/arro3/pull/231

Full Changelog: https://github.com/kylebarron/arro3/compare/pyo3-arrow-v0.5.0...pyo3-arrow-v0.5.1

New features :sparkles:

• feat: access value_fields and inner fields by @ion-elgreco in https://github.com/kylebarron/arro3/pull/334
• feat(arro3-core): Basic data rendering in __repr__ by @kylebarron in https://github.com/kylebarron/arro3/pull/335

Bug fixes :bug:

• feat(arro3-core): Make DataType hashable by @kylebarron in https://github.com/kylebarron/arro3/pull/336
• test(arro3-core): Check that Schema is iterable by @kylebarron in https://github.com/kylebarron/arro3/pull/338

Documentation updates :memo:

• Add sphinx interlinking for ArrowStreamExportable by @kylebarron in https://github.com/kylebarron/arro3/pull/339

New Contributors

• @ion-elgreco made their first contribution in https://github.com/kylebarron/arro3/pull/334

Full Changelog: https://github.com/kylebarron/arro3/compare/py-v0.4.6...py-v0.5.0

What's Changed

• Bump to pyo3 0.22 by @kylebarron in https://github.com/kylebarron/arro3/pull/226

Full Changelog: https://github.com/kylebarron/arro3/compare/pyo3-arrow-v0.4.0...pyo3-arrow-v0.5.0

Enhancements :magic_wand:

• Zero-copy buffer protocol data import by @kylebarron in https://github.com/kylebarron/arro3/pull/204

  • Handle multi-dimensional buffer protocol input by @kylebarron in https://github.com/kylebarron/arro3/pull/208
  • Prefer zero-copy in from_numpy by @kylebarron in https://github.com/kylebarron/arro3/pull/214

• New compute functions:

  • Add date_part by @kylebarron in https://github.com/kylebarron/arro3/pull/202
  • Aggregate functions: array min, max, sum by @kylebarron in https://github.com/kylebarron/arro3/pull/193
  • Arith functions by @kylebarron in https://github.com/kylebarron/arro3/pull/194
  • More compute functions by @kylebarron in https://github.com/kylebarron/arro3/pull/104

• Implement scalar equality by @kylebarron in https://github.com/kylebarron/arro3/pull/205

• Improved docstrings by @kylebarron in https://github.com/kylebarron/arro3/pull/198

• Add installation section to readme by @kylebarron in https://github.com/kylebarron/arro3/pull/189

Bug fixes :bug:

• Fix data type bit width by @kylebarron in https://github.com/kylebarron/arro3/pull/190
• Fix return type of RecordBatch.column by @kylebarron in https://github.com/kylebarron/arro3/pull/191

Enhancements :magic_wand:

• Zero-copy buffer protocol data import by @kylebarron in https://github.com/kylebarron/arro3/pull/204
  • Handle multi-dimensional buffer protocol input by @kylebarron in https://github.com/kylebarron/arro3/pull/208
• Put buffer protocol behind feature flag by @kylebarron in https://github.com/kylebarron/arro3/pull/215
• Implement FromPyObject for PyScalar by @kylebarron in https://github.com/kylebarron/arro3/pull/199

Full Changelog: https://github.com/kylebarron/arro3/compare/pyo3-arrow-v0.3.0...pyo3-arrow-v0.4.0

Enhancements :magic_wand:

• Wheels for pyodide are auto-built on CI and published to Github releases.
• Ensure Parquet schema metadata is added to arrow table by @kylebarron in https://github.com/kylebarron/arro3/pull/137
• Access dictionary array keys and values by @kylebarron in https://github.com/kylebarron/arro3/pull/139
• Support reading Parquet from file objects by @kylebarron in https://github.com/kylebarron/arro3/pull/142
• Implement dictionary encoding by @kylebarron in https://github.com/kylebarron/arro3/pull/136
• Support for Arrow scalars and converting to Python objects by @kylebarron in https://github.com/kylebarron/arro3/pull/159
• Move functional accessors to core module by @kylebarron in https://github.com/kylebarron/arro3/pull/151
• Support for python buffer protocol by @kylebarron in https://github.com/kylebarron/arro3/pull/156

Bug fixes :bug:

• Fix field metadata propagation by @kylebarron in https://github.com/kylebarron/arro3/pull/150
• Set strip=true for maturin builds by @kylebarron in https://github.com/kylebarron/arro3/pull/155
• Support __getitem__ with a negative index by @kylebarron in https://github.com/kylebarron/arro3/pull/171
• support f16 in from_numpy by @kylebarron in https://github.com/kylebarron/arro3/pull/154
• Fix writing to file by @kylebarron in https://github.com/kylebarron/arro3/pull/138

Full Changelog: https://github.com/kylebarron/arro3/compare/py-v0.2.1...py-v0.3.0

Enhancements :magic_wand:

• Implement casting via Arrow PyCapsule Interface. This means that pyo3-arrow now respects the requested_schema argument. If the cast is invalid, the original data is exported.
• New PyScalar class for managing Arrow scalar interop.
• PyArray and PyScalar implement Datum.
• Public API to import Arrow objects from raw PyCapsules. https://github.com/kylebarron/arro3/pull/183
• Bump to Arrow 53.
• Use thiserror for the error enum.
• New AnyDatum input object for allowing either array or scalar input.

Fixes :bug:

• Handle RecordBatch import with positive length but no columns. https://github.com/kylebarron/arro3/pull/177

Bug fixes :bug:

• Fix API docs (#131)

Enhancements :magic_wand:

• In general, high parity with pyarrow for most data access and management. Most of the Table, ChunkedArray, Array, RecordBatchReader, Schema, and Field methods and behavior should be similar to pyarrow.
• An ArrayReader, an abstraction beyond RecordBatchReader to allow a stream of arrow arrays that are not RecordBatches.
• Readers and writers for Parquet, Arrow IPC, CSV, and JSON.
• Initial compute functions.
• Initial Python tests.
• Improved documentation, both in the type stubs and in the docs website.
• Pyodide wheel support.

Enhancements :magic_wand:

• New ArrayReader. It parallels RecordBatchReader but is more general, supporting arbitrary Arrow arrays that do not have to represent a record batch.
• New AnyArray enum that supports either Array or ArrayReader input.
• Improved documentation.

Fixes :bug:

• Validate Schema/Field when constructing new Array/ChunkedArray/Table (#72)
• Convert Table::new to Table::try_new and ensure that all batches have the same schema. Similar for Array::new and ChunkedArray::new.
• Reorder args for Table::new

• Initial release of arro3-core v0.1.

• Initial release