hypothesis

This release adds the experimental and unstable ~hypothesis.settings.backend setting. See alternative-backends for details.

---

This patch fixes 3900, a performance regression for ~hypothesis.extra.numpy.arrays due to the interaction of v6.98.12 and v6.97.1.

---

This patch improves the type annotations in hypothesis.extra.numpy, which makes inferred types more precise for both mypy and pyright, and fixes some strict-mode errors on the latter.

Thanks to Jonathan Plasse for reporting and fixing this in 3889!

---

This patch paves the way for future shrinker improvements. There is no user-visible change.

---

This release adds support for the Array API's 2023.12 release via the api_version argument in ~hypothesis.extra.array_api.make_strategies_namespace. The API additions and modifications in the 2023.12 spec do not necessitate any changes in the Hypothesis strategies, hence there is no distinction between a 2022.12 and 2023.12 strategies namespace.

---

This patch adjusts the printing of bundle values to correspond with their names when using stateful testing.

---

This patch implements filter-rewriting for ~hypothesis.strategies.text and ~hypothesis.strategies.binary with the ~re.Pattern.search, ~re.Pattern.match, or ~re.Pattern.fullmatch method of a re.compiled regex.

---

This patch implements filter-rewriting for most length filters on some additional collection types (3795), and fixes several latent bugs where unsatisfiable or partially-infeasible rewrites could trigger internal errors.

---

This patch makes stateful testing somewhat less likely to get stuck when there are only a few possible rules.

---

This patch adds a note to errors which occur while drawing from a strategy, to make it easier to tell why your test failed in such cases.

---

This patch ensures that observability outputs include an informative repr for ~hypothesis.stateful.RuleBasedStateMachine stateful tests, along with more detailed timing information.

---

This patch improves the Ghostwriter for binary operators.

---

This patch improves import-detection in the Ghostwriter (3884), particularly for ~hypothesis.strategies.from_type and strategies from hypothesis.extra.*.

---

This patch clarifies the documentation on stateful testing (3511).

---

This patch improves argument-to-json conversion for observability output. Checking for a .to_json() method on the object before a few other options like dataclass support allows better user control of the process (3880).

---

This patch updates our vendored list of top-level domains, which is used by the provisional ~hypothesis.provisional.domains strategy.

---

This patch fixes an error when generating observability reports involving large (n > 1e308) integers.

---

This patch refactors some internals. There is no user-visible change.

---

This release improves our distribution of generated values for all strategies, by doing a better job of tracking which values we have generated before and avoiding generating them again.

For example, st.lists(st.integers()) previously generated ~5 each of [] [0] in 100 examples. In this release, each of [] and [0] are generated ~1-2 times each.

---

This release deprecates use of the global random number generator while drawing from a strategy, because this makes test cases less diverse and prevents us from reporting minimal counterexamples (3810).

If you see this new warning, you can get a quick fix by using ~hypothesis.strategies.randoms; or use more idiomatic strategies ~hypothesis.strategies.sampled_from, ~hypothesis.strategies.floats, ~hypothesis.strategies.integers, and so on.

Note that the same problem applies to e.g. numpy.random, but for performance reasons we only check the stdlib random module - ignoring even other sources passed to ~hypothesis.register_random.

---

This patch updates our vendored list of top-level domains, which is used by the provisional ~hypothesis.provisional.domains strategy.

---

This patch adds some observability information about how many times predicates in ~hypothesis.assume or ~hypothesis.stateful.precondition were satisfied, so that downstream tools can warn you if some were never satisfied by any test case.

---

This patch improves formatting and adds some cross-references to our docs.

---

Internal test refactoring.

---

This patch slightly changes how we replay examples from the database: if the behavior of the saved example has changed, we now keep running the test case instead of aborting at the size of the saved example. While we know it's not the same example, we might as well continue running the test!

Because we now finish running a few more examples for affected tests, this might be a slight slowdown - but correspondingly more likely to find a bug.

We've also applied similar tricks to the |Phase.target| phase, where they are a pure performance improvement for affected tests.

---

Improves the performance of the ~hypothesis.extra.numpy.arrays strategy when generating unique values.

---

Changes the distribution of ~hypothesis.strategies.sampled_from when sampling from a ~python:enum.Flag. Previously, no-flags-set values would never be generated, and all-flags-set values would be unlikely for large enums. With this change, the distribution is more uniform in the number of flags set.

---

This patch slightly refactors some internals. There is no user-visible change.

---

This patch fixes a spurious warning about slow imports when HYPOTHESIS_EXPERIMENTAL_OBSERVABILITY was set.

---

This patch refactors some more internals, continuing our work on supporting alternative backends (3086). There is no user-visible change.

---

Fix a spurious warning seen when running pytest's test suite, caused by never realizing we got out of initialization due to imbalanced hook calls.

---

Warns when constructing a repr that is overly long. This can happen by accident if stringifying arbitrary strategies, and is expensive in time and memory. The associated deferring of these long strings in ~hypothesis.strategies.sampled_from should also lead to improved performance.

---

This release adds the ability to pass any object to ~hypothesis.note, instead of just strings. The pretty-printed representation of the object will be used.

See also 3843.

---

This release avoids creating a .hypothesis directory when using ~hypothesis.strategies.register_type_strategy (3836), and adds warnings for plugins which do so by other means or have other unintended side-effects.

---

This patch improves observability reports by moving timing information from metadata to a new timing key, and supporting conversion of additional argument types to json rather than string reprs via a .to_json() method (including e.g. Pandas dataframes).

Additionally, the ~hypothesis.HealthCheck.too_slow health check will now report which strategies were slow, e.g. for strategies a, b, c, ...:

    count | fraction |    slowest draws (seconds)
a |    3  |     65%  |      --      --      --   0.357,  2.000
b |    8  |     16%  |   0.100,  0.100,  0.100,  0.111,  0.123
c |    3  |      8%  |      --      --   0.030,  0.050,  0.200
(skipped 2 rows of fast draws)

---

This patch refactors some internals, continuing our work on supporting alternative backends (3086). There is no user-visible change.

---

The ~hypothesis.extra.lark.from_lark strategy now accepts an alphabet= argument, which is passed through to ~hypothesis.strategies.from_regex, so that you can e.g. constrain the generated strings to a particular codec.

In support of this feature, ~hypothesis.strategies.from_regex will avoid generating optional parts which do not fit the alphabet. For example, from_regex(r"abc|def", alphabet="abcd") was previously an error, and will now generate only 'abc'. Cases where there are no valid strings remain an error.

---

This patch refactors some internals, continuing our work on supporting alternative backends (3086). There is no user-visible change.

---

This patch adds a test statistics event when a generated example is rejected via assume.

This may also help with distinguishing gave_up examples in observability (3827).

---

This introduces the rewriting of length filters on some collection strategies (3791).

Thanks to Reagan Lee for implementing this feature!

---

If a test uses ~hypothesis.strategies.sampled_from on a sequence of strategies, and raises a TypeError, we now add a note asking whether you meant to use ~hypothesis.strategies.one_of.

Thanks to Vince Reuter for suggesting and implementing this hint!

---

This patch registers explicit strategies for a handful of builtin types, motivated by improved introspection in PyPy 7.3.14 triggering existing internal warnings. Thanks to Carl Friedrich Bolz-Tereick for helping us work out what changed!

---

This patch fixes an error when writing observability reports without a pre-existing .hypothesis directory.

---

This patch adds a new environment variable HYPOTHESIS_EXPERIMENTAL_OBSERVABILITY_NOCOVER, which turns on observability data collection without collecting code coverage data, which may be faster on Python 3.11 and earlier.

Thanks to Harrison Goldstein for reporting and fixing 3821.

---

This patch updates our vendored list of top-level domains, which is used by the provisional ~hypothesis.provisional.domains strategy.

---

This patch fixes a bug introduced in version 6.92.0, where using the ~hypothesis.strategies.data strategy would fail to draw a ~python:dataclasses.dataclass with a ~python:collections.defaultdict field. This was due to a bug in the standard library which was fixed in 3.12, so we've vendored the fix.

---

This release adds an experimental observability mode. You can read the docs about it here.

---

This patch refactors some more internals, continuing our work on supporting alternative backends (3086). There is no user-visible change.

---

This patch fixes an issue where ~hypothesis.strategies.builds could not be used with attrs objects that defined private attributes (i.e. attributes with a leading underscore). See also 3791.

This patch also adds support more generally for using ~hypothesis.strategies.builds with attrs' alias parameter, which was previously unsupported.

This patch increases the minimum required version of attrs to 22.2.0.

---

This release adds an optional payload argument to hypothesis.event, so that you can clearly express the difference between the label and the value of an observation. statistics will still summarize it as a string, but future observability options can preserve the distinction.

---

This patch supports assigning settings = settings(...) as a class attribute on a subclass of a .TestCase attribute of a ~hypothesis.stateful.RuleBasedStateMachine. Previously, this did nothing at all.

# works as of this release
class TestMyStatefulMachine(MyStatefulMachine.TestCase):
    settings = settings(max_examples=10000)

# the old way still works, but it's more verbose.
MyStateMachine.TestCase.settings = settings(max_examples=10000)

class TestMyStatefulMachine(MyStatefulMachine.TestCase):
    pass

Thanks to Joey Tran for reporting these settings-related edge cases in stateful testing.

---

This release makes it an error to assign settings = settings(...) as a class attribute on a ~hypothesis.stateful.RuleBasedStateMachine. This has never had any effect, and it should be used as a decorator instead:

class BadMachine(RuleBasedStateMachine):
    """This doesn't do anything, and is now an error!"""

    settings = settings(derandomize=True)

@settings(derandomize=True)
class GoodMachine(RuleBasedStateMachine):
    """This is the right way to do it :-)"""

---

This patch refactors some internals. There is no user-visible change, but we hope to improve performance and unlock support for alternative backends such as symbolic execution with crosshair in future (3086).

Thanks to Liam DeVoe for this fantastic contribution!

---

This release teaches ~hypothesis.strategies.from_type to handle constraints implied by the annotated-types package - as used by e.g. pydantic. This is usually efficient, but falls back to filtering in a few remaining cases.

Thanks to Viicos for 3780!

---

This patch adds a warning when @st.composite wraps a function annotated as returning a ~hypothesis.strategies.SearchStrategy, since this is usually an error (3786). The function should return a value, and the decorator will convert it to a function which returns a strategy.

---

This patch refactors from_type(typing.Tuple), allowing ~hypothesis.strategies.register_type_strategy to take effect for tuples instead of being silently ignored (3750).

Thanks to Nick Collins for reporting and extensive work on this issue.

---

This patch improves the speed of the explain phase on python 3.12+, by using the new sys.monitoring module to collect coverage, instead of sys.settrace.

Thanks to Liam DeVoe for 3776!

---

This patch improves ~hypothesis.strategies.register_type_strategy when used with tuple subclasses, by preventing them from being interpreted as generic and provided to strategies like st.from_type(Sequence[int]) (3767).

---