What's Changed

• Backport data source to pyspark 3 by @lhoestq in https://github.com/huggingface/pyspark_huggingface/pull/11

Full Changelog: https://github.com/huggingface/pyspark_huggingface/compare/0.1.1...1.0.0

What's Changed

• authentication to support reading gated/private datasets on HF
• support reading specific git revisions
• improved error messages
• Reduce requirements by deferring import until when it's actually needed by @wengh in https://github.com/huggingface/pyspark_huggingface/pull/8
• Fix import for compatibility with older huggingface_hub by @wengh in https://github.com/huggingface/pyspark_huggingface/pull/9

Full Changelog: https://github.com/huggingface/pyspark_huggingface/compare/0.1.0...0.1.1

What's Changed

• Add basic HuggingFace Data Source Implementation by @allisonwang-db in https://github.com/huggingface/pyspark_huggingface/pull/1
• initial pyproject.toml by @lhoestq in https://github.com/huggingface/pyspark_huggingface/pull/2
• Add more features to huggingface reader by @allisonwang-db in https://github.com/huggingface/pyspark_huggingface/pull/3
• Enable predicate pushdown by @lhoestq in https://github.com/huggingface/pyspark_huggingface/pull/4
• Add HuggingFaceSink data source by @wengh in https://github.com/huggingface/pyspark_huggingface/pull/5
• Support custom split name by renaming files by @wengh in https://github.com/huggingface/pyspark_huggingface/pull/6
• Use same data source name for reader and writer by @wengh in https://github.com/huggingface/pyspark_huggingface/pull/7

New Contributors

• @allisonwang-db made their first contribution in https://github.com/huggingface/pyspark_huggingface/pull/1
• @lhoestq made their first contribution in https://github.com/huggingface/pyspark_huggingface/pull/2

Full Changelog: https://github.com/huggingface/pyspark_huggingface/commits/0.1.0