✨ New features and improvements

• Rebase on Curated Transformers 2.0 (#19).
• Add the transformer_discriminative.v1 schedule (#27). This schedule uses a two different schedules:
  • transformer_schedule for transformer parameters;
  • default_schedule for other parameters.

• Update curated-transformers and curated-tokenizers deps

• Bump version to 2.0.0.dev1

Also:

• Set Python bound to >= 3.9, in line with spaCy 4.
• Update GHA to test 3.8 and 3.12.

Previous versions used explicit zeroed rows corresponding to whitespace tokens in spaCy. This required duplication and a number of assignments into the transformer output, which was inefficient.

Instead, whitespace tokens are now regarded as not aligning to any wordpiece tokens. If you do doc._.trf_data[i] where i is the index of a whitespace token, you'll receive an array of shape (0, n) where n is the output dimension. This is handled in Thinc's pooling operations, so the change doesn't require any update to models consuming the trf_data.

Drop the direct dependency on spaCy, to avoid requirement circles.

Specifically, we're changing models to no longer specify a spaCy version as a requirement, to allow models to be forward compatible. However, the transformer models depend on this library. If it then pulls in spaCy, we end up with spaCy in the requirements again.

✨ New features and improvements

• Make DocTransformerOutput serializable with msgpack (#29).

✨ New features and improvements

• Add init fill-curated-transformer CLI command (#16, #22).

🔴 Bug fixes

• Correctly pass hidden_width as the embedding width in non-ALBERT models (#17).
• Add vocab_size to default transformer pipe config (#20).

:sparkles: New features and improvements

• Add support for replacing listeners (#7).

:warning: Backwards incompatibilities

• Rename pipe to CuratedTransformer (#12).
• Remove quantization support until the serialization API is stabilized (#6).

✨ New features and improvements

• Switch from cutlery to curated-tokenizers for word/sentencepiecing (#4).

🔴 Bug fixes

• Do not raise an exception when encountering an empty doc (#3).

This new package contains the entry points and spaCy/Thinc wrapping for curated-transformers that originally resided in the curated-transformers package.