v1.3.1: Minor type validation improvements

Compare

Choose a tag to compare

• Refactor library to properly separate config parsing, overrides, default filling, validation and promise resolution
• Rewrite type-checking code to be more reliable

Fix pydantic v2 compat: check model_fields/model_config before fields/config, respect Strict metadata in Annotated types

• Use TypeVar self-type pattern (_ConfigSelf, _CVESelf) so Config and ConfigValidationError methods return the correct type for subclasses
• Use type(self)() instead of Config() in interpolate/copy/merge so subclass instances are preserved at runtime
• Widen input parameters from Dict to Mapping and List to Sequence where the parameter is read-only (overrides, updates, config params, section_order)
• Widen ConfigValidationError.config param to accept any Mapping
• Unify errors param type to Iterable[Mapping[str, Any]]

• Fix Config.to_str() crash on * positional arg sections (#52).
• Fix _update_from_parsed crash when a pydantic validator coerces a string to a dict (#54).
• Coerce dict arguments to BaseModel instances for registered functions (#58, #59).
• Support top-level section references in interpolation (#61).
• Fix incomplete ${ in config values crashing to_str().

Pydantic v2 is stricter than v1 and I've been unable to replicate the behaviours we had in the config.

I had also attempted to refactor the library to make it easier to port, but this seems to have also introduced regressions.

To finally resolve this I'm dropping support for Pydantic in favour of a relatively small amount of custom validation logic. This should finally unblock spaCy and Thinc.

Properly support Pydantic v2

Fix support for Pydantic v2, and greatly extend tests

Previously confection did three tree operations simultaneously (in the registry._fill method):

• Build a version of the config with defaults in-place
• Build a modified version suitable for validation
• Build an output version with promises resolved

The fact that all three were being built at once made the code quite hard to update for Pydantic 2, and just difficult to work with in general. I've therefore refactored the code so that these are separate processes that apply to the whole tree at once.

A bonus of this is that we now build a model for the whole config and validate against that, which means all the validation errors for the config should come in at once, instead of only getting validation errors for one promise at a time.

Updating to Pydantic v2 will let spaCy finally support Python 3.13.

✨ Fixes and features

• Fixes #47: Ensure config with * can be filled (#53)
• Allow typing-extensions<5.0.0 for Python < 3.8 (#63)
• Update CI (#64)

👥 Contributors

@adrianeboyd, @honnibal, @ines, @KennethEnevoldsen, @MartinBernstorff, @svlandeg

• Support use of $$ to escape a literal $ character in config strings (#49).

• Add frozen structures to __init__.__all__ to ensure they can be imported easily from confection (#43).

• Implement deepcopy() for FrozenDict and FrozenList to allow setting values in function headers (#41)

• Make catalogue a dev-only requirement (#35).
• Allow Pydantic v2 using transitional v1 support (#38).
• Format and lint with black/flake8/isort (#39).

• Add checks with warnings for single-quoted values (#22).
• Add util.SimpleFrozenDict and util.SimpleFrozenList from spaCy (#28).
• Remove recursive JSON parsing of list values to preserve string values such as ["003"] rather than parsing them as integers (#33).

• Extend support for typing_extensions to v4.4.x for Python 3.6 and 3.7.
• Add pyproject.toml to source package.

• Extend support to pydantic v1.10.x.

• Add py.typed to package.
• Switch to mypy 0.981 for fixes related to Python 3.10.7 and NumPy.
• Update README.

This is the initial release of confection, a lightweight library that offers a configuration system letting you conveniently describe arbitrary trees of objects.

If you are a user of spaCy or Thinc, you may be familiar with this configuration system already - now you can use it independently.