fastwarc

This project publishes no release notes. Between v1.0.8 and v1.0.9 there were 11 commits, 10 of them substantive:

• Add legacy Cython implementation to list
• Fix typo in docs
• Change predicate display strings
• Slightly optimise dict/tuple header conversions
• Add warc and stream_io as explicit submodules
• Don't build rlib
• Revert add submodules in Rust
• Update docs
• Update docs
• Fix slicing errors when accessing empty header values

This project publishes no release notes. Between v1.0.7 and v1.0.8 there were 16 commits, 15 of them substantive:

• Use GzipWriter::from_path
• Fix formatting
• Consistency fix
• Add uuid: prefix
• Add WarcHeader enum
• Update doc string
• Use filtered archive iterator instead of checking filters in loop
• Enforce Send for WarcRead
• Fix FastWARC Python sdist compilation
• Fix in-tree builds
• Fix build dependency not installed
• Use native dispatch for pre-defined func_filters
• Update doc string
• Make filter application slightly more efficient
• Update benchmarks

This project publishes no release notes. Between v1.0.6 and v1.0.7 there were 13 commits, 12 of them substantive:

• Add link to FastWARC Rust manual to sidebar
• Fix typo in function name
• Pre-reserve buffer space in example
• Move buffer allocation outside of loop in example
• Reserve only additionally needed bytes
• Improve usage guides
• Change parameter type for init_headers type parameter to not accept Option any more
• Add section about writing WARC files to docs
• Guarantee byte-identical serialisation
• Implement proper Debug print for HeaderMap
• Document writing with checksumming
• Fix Python tests

This project publishes no release notes. Between v1.0.5 and v1.0.6 there were 10 commits, 9 of them substantive:

• Change default buffer size for local files to 1 MiB.
• Fix RustDoc warnings and build docs on CI
• Shorten run config name
• Fix sphinx-build not being found
• Move SharedWarcRecord to record
• Add "Freezing Records" section to Python FastWARC docs
• Fix methods and properties not being documented
• Add FastWARC RustDoc user guide
• Fix typo

This project publishes no release notes. Between v1.0.4 and v1.0.5 there were 24 commits, 23 of them substantive:

• Add benchmark suite with third-party WARC parsers
• Move buffer size to global variable
• Add more benchmarks
• Move benchmarks folder up
• Add fastwarc-py benchmark
• Fix libarchive benchmark build on Linux
• Add summary to all benchmarks
• Get BUFFER_SIZE from environment
• Fix libarchive not emitting all WARC records
• Change default buffer size to 1MiB
• Add README
• Add README
• Add individual READMEs and benchmark summary
• Add headings
• Update README
• Add highlighting for code block
• Fix incorrect warcpp performance number in table
• Update BUFFER_SIZE example
• Update README
• Update benchmark section

…and 3 more.

This project publishes no release notes. Between v1.0.3 and v1.0.4 there were 28 commits, 27 of them substantive:

• Move payload_buf initialisation out of loop
• Do not shrink GzipReader decompressed buffer
• Implement consume_n using seek
• Explicitly inline stream_position
• Make max header length configurable and increase default limit
• Fix typo
• Use memchr for : scan in headers
• Slightly optimise header parsing
• Add quirks mode parsing
• Make parsed headers copy-on-write
• Move headers to separate submodule
• Move helper function
• Fix mypy error
• Mark legacy shims as deprecated
• Mark alias names as deprecated
• Fix import error on newer Python versions
• Fix header parsing when EOH marker is split across buffer boundaries
• Explicitly inline more methods
• Avoid copying of raw header buffer
• Use existing read_line method instead of take()

…and 7 more.

This project publishes no release notes. Between v1.0.2 and v1.0.3 there were 17 commits, 16 of them substantive:

• Add performance profiling binary
• Add short circuits for stream_position() and seek() in RawReaderAdapter and GzipReader for much improved performance
• Simplify IntoWarcReader trait bound for BufReader<T> and add conversion for fs::File
• Increase buffer sizes
• Use cached stream position for member boundaries
• Print byte throughput in profile binary
• Move reader adapter infrastructure to separate module
• Add TrackingBufReader to replace BufReader at the lowest level
• Serve small relative forward seeks from fill_buf()
• Update IntoWarcReader impls for BufReader, File, and Cursor
• Add constructor to RawWriterAdapter and std::fs::File conversion trait impl
• Use TrackingBufReader in decompressing readers (except Brotli) and optimise Gzip buffer growing
• Use larger buffer sizes in Python wrapper
• Minor performance improvement in read_line
• Forward inner_seek to native inner_seek
• Fix incorrect test assumptions

This project publishes no release notes. Between v1.0.1 and v1.0.2 there were 46 commits, 45 of them substantive:

• Port test_record_writer test
• Add clone_warc_file test (ported test_record_writer test)
• Port create_new_warc_record
• Specify explicit packages
• Use newer Python version (maybe that fixes the twine upload issue?)
• Rename cargo-coverage.lcov to cargo-coverage.info
• Upgrade packaging on twine upload, fix wheel matching in build-documentation
• Install Apache Beam in Python 3.14
• Install apache_beam in docs requirements.txt as a workaround for 314t requiring us to build grpcio
• Publish crates from CI
• Uncomment crates publishing guard
• Update READMEs, add RustDoc link to Sphinx sidebar
• Unhide ArchiveIteratorImpl
• Replace explicit &[u8] arguments with AsRef<[u8]>
• Ignore additional file types
• Add coverage building helper script
• Add Python bindings tests
• Fix deadlock in pickle
• Implement PartialEq and eq for WarcRecords
• Dispatch close to runtime self

…and 25 more.

This project publishes no release notes. Between v1.0.0 and v1.0.1 there were 27 commits, 26 of them substantive:

• Install cargo-llvm-cov
• Update coverage generation
• Ignore legacy path from Codecov
• Treat partials as hits and exclude test files from coveragea
• Restructure stub files, fix stubtest errors
• Exclude WarcRecordType to prevent double indexing
• Replace deprecated HTML theme setting
• Patch modules for sphinx stub detection
• Ignore missing imports in stubtest
• Remove manual type annotations from doc strings
• Make mypy happy
• Fix python command
• Modernise stub syntax
• Fix doc building with non-editable builds
• Update doc string
• Fix Sphinx warnings and make warnings errors
• Disable source tree shadowing in docs build
• Auto-detect in-tree builds and fix CI wheel installation for doc builds
• Replace deprecated license attribute
• Print error to stderr

…and 6 more.