hypothesis

Tired of writing tests? Or new to Hypothesis and not sure where to start?

This release is for you! With our new Ghostwriter functions and hypothesis write ... command-line interface, you can stop writing tests entirely... or take the source code Hypothesis writes for you as a starting point.

This has been in the works for months, from 2118 to versions 5.18.3, 5.23.5, and 5.23.5 - particular thanks to the many people who reviewed pull requests or commented on demos, and to Timothy Crosley's hypothesis-auto project for inspiration.

---

This patch adds yet more internal functions to support a new feature we're working on, like version 5.18.3 and version 5.23.6. We promise it's worth the wait!

---

This release fixes a small internal bug in Hypothesis's internal automaton library. Fortunately this bug was currently impossible to hit in user facing code, so this has no user visible impact.

---

This release improves shrink quality by allowing Hypothesis to automatically learn new shrink passes for difficult to shrink tests.

The automatic learning is not currently accessible in user code (it still needs significant work on robustness and performance before it is ready for that), but this release includes learned passes that should improve shrinking quality for tests which use any of the ~hypothesis.strategies.text, ~hypothesis.strategies.floats, ~hypothesis.strategies.datetimes, ~hypothesis.strategies.emails, and ~hypothesis.strategies.complex_numbers strategies.

---

This patch updates some docstrings, without changing runtime behaviour.

---

The ~hypothesis.strategies.functions strategy has a new argument pure=True, which ensures that the same return value is generated for identical calls to the generated function (2538).

Thanks to Zac Hatfield-Dodds and Nikita Sobolev for this feature!

---

This release removes a number of Hypothesis's internal "shrink passes" - transformations it makes to a generated test case during shrinking - which appeared to be redundant with other transformations.

It is unlikely that you will see much impact from this. If you do, it will likely show up as a change in shrinking performance (probably slower, maybe faster), or possibly in worse shrunk results. If you encounter the latter, please let us know.

---

This release fixes a bug in some internal Hypothesis support code. It has no user visible impact.

---

This release improves the quality of shrunk test cases in some special cases. Specifically, it should get shrinking unstuck in some scenarios which require simultaneously changing two parts of the generated test case.

---

This release improves the performance of some internal support code. It has no user visible impact, as that code is not currently run during normal Hypothesis operation.

---

This release adds a heuristic to detect when shrinking has finished despite the fact that there are many more possible transformations to try. This will be particularly useful for tests where the minimum failing test case is very large despite there being many smaller test cases possible, where it is likely to speed up shrinking dramatically.

In some cases it is likely that this will result in worse shrunk test cases. In those cases rerunning the test will result in further shrinking.

---

This release makes some performance improvements to shrinking. They should only be noticeable for tests that are currently particularly slow to shrink.

---

This patch adds some more internal functions to support a new feature we're working on, like version 5.18.3. There is still no user-visible change... yet.

---

This release makes some changes to internal support code that is not currently used in production Hypothesis. It has no user visible effect at present.

---

This release improves shrinking quality in some special cases.

---

This release fixes 2507, where lazy evaluation meant that the values drawn from a ~hypothesis.strategies.sampled_from strategy could depend on mutations of the sampled sequence that happened after the strategy was constructed.

---

This patch fixes 2462, a bug in our handling of unittest.TestCase.subTest. Thanks to Israel Fruchter for fixing this at the EuroPython sprints!

---

This release improves the behaviour of the ~hypothesis.strategies.characters strategy when shrinking, by changing which characters are considered smallest to prefer more "normal" ascii characters where available.

---

The default print_blob setting is now smarter. It defaults to True in CI and False for local development.

Thanks to Hugo van Kemenade for implementing this feature at the EuroPython sprints!

---

The ~hypothesis.strategies.slices strategy can now generate slices for empty sequences, slices with negative start and stop indices (from the end of the sequence), and step=None in place of step=1.

Thanks to Sangarshanan for implementing this feature at the EuroPython sprints!

---

This release ensures that tests which raise RecursionError are not reported as flaky simply because we run them from different initial stack depths (2494).

---

This release improves the performance of the sample method on objects obtained from ~hypothesis.strategies.randoms when use_true_random=False. This should mostly only be noticeable when the sample size is a large fraction of the population size, but may also help avoid health check failures in some other cases.

---

This release makes some internal changes for testing purposes and should have no user visible effect.

---

This release fixes a small caching bug in Hypothesis internals that may under some circumstances have resulted in a less diverse set of test cases being generated than was intended.

Fixing this problem revealed some performance problems that could occur during targeted property based testing, so this release also fixes those. Targeted property-based testing should now be significantly faster in some cases, but this may be at the cost of reduced effectiveness.

---

This patch updates our formatting to use isort 5. There is no user-visible change.

---

The ~hypothesis.extra.numpy.basic_indices strategy can now generate bare indexers in place of length-one tuples. Thanks to Andrea for this patch!

---

This patch removes an internal use of distutils in order to avoid this setuptools warning for some users.

---

This patch contains a small internal refactoring with no user-visible impact.

Thanks to Andrea for writing this at the SciPy 2020 Sprints!

---

This release slightly improves shrinking behaviour. This should mainly only impact stateful tests, but may have some minor positive impact on shrinking collections (lists, sets, etc).

---

This release improves the ~hypothesis.strategies.randoms strategy by adding support for Random instances where Hypothesis generates the random values rather than having them be "truly" random.

---

This patch adds some internal functions to support a new feature we're working on. There is no user-visible change... yet.

---

This patch improves our docs for the ~hypothesis.settings.derandomize setting.

---

This release consists of some internal refactoring to the shrinker in preparation for future work. It has no user visible impact.

---

This release teaches Hypothesis to shorten tracebacks for |@example|, as we already do for generated examples, so that you can focus on your code rather than ours.

If you have multiple failing explicit examples, they will now all be reported. To report only the first failure, you can use the report_multiple_bugs=False setting as for generated examples.

---

This patch adds strategy inference for the Literal, NewType, Type, DefaultDict, and TypedDict types from the typing-extensions backport on PyPI.

---

This patch precomputes some of the setup logic for our external fuzzer integration and sets deadline=None in fuzzing mode, saving around 150us on each iteration.

This is around two-thirds the runtime to fuzz an empty test with @given(st.none()), and nice to have even as a much smaller fraction of the runtime for non-trivial tests.

---

This patch fixes an internal error when warning about the use of function-scoped fixtures for parametrised tests where the parametrised value contained a % character. Thanks to Bryant for reporting and fixing this bug!

---

If you pass a python:list or python:tuple where a strategy was expected, the error message now mentions ~hypothesis.strategies.sampled_from as an example strategy.

Thanks to the enthusiastic participants in the PyCon Mentored Sprints who suggested adding this hint.

---

~hypothesis.strategies.functions can now infer the appropriate returns strategy if you pass a like function with a return-type annotation. Before, omitting the returns argument would generate functions that always returned None.

---

Fix ~hypothesis.strategies.from_type with generic types under Python 3.9.

---

This patch fixes an error that happens when multiple threads create new strategies.

---

Passing min_magnitude=None to ~hypothesis.strategies.complex_numbers is now deprecated - you can explicitly pass min_magnitude=0, or omit the argument entirely.

---

This patch fixes an internal error in ~hypothesis.strategies.from_type for python:typing.NamedTuple in Python 3.9. Thanks to Michel Salim for reporting and fixing 2427!

---

This release upgrades the test statistics available via the --hypothesis-show-statistics option to include separate information on each of the ~hypothesis.settings.phases (1555).

---

This patch teaches the ~hypothesis.strategies.from_type internals to return slightly more efficient strategies for some generic sets and mappings.

---

This patch adds a # noqa comment for flake8 3.8.0, which disagrees with mypy about how to write the type of ....

---

This release limits the maximum duration of the shrinking phase to five minutes, so that Hypothesis does not appear to hang when making very slow progress shrinking a failing example (2340).

If one of your tests triggers this logic, we would really appreciate a bug report to help us improve the shrinker for difficult but realistic workloads.

---

This release improves the interaction between ~hypothesis.assume and the @example() decorator, so that the following test no longer fails with UnsatisfiedAssumption (2125):

@given(value=floats(0, 1))
@example(value=0.56789)  # used to make the test fail!
@pytest.mark.parametrize("threshold", [0.5, 1])
def test_foo(threshold, value):
    assume(value < threshold)
    ...

---

If you have Django installed but don't use it, this patch will make import hypothesis a few hundred milliseconds faster (e.g. 0.704s -> 0.271s).

Thanks to importtime-waterfall for highlighting this problem and Jake Vanderplas for the solution - it's impossible to misuse code from a module you haven't imported!

---

This patch improves the internals of ~hypothesis.strategies.builds type inference, to handle recursive forward references in certain dataclasses. This is useful for e.g. hypothesmith's forthcoming LibCST mode.

---

This release reverses the order in which some operations are tried during shrinking. This should generally be a slight performance improvement, but most tests are unlikely to notice much difference.

---

This patch fixes 2406, where use of pandas:pandas.Timestamp objects as bounds for the ~hypothesis.strategies.datetimes strategy caused an internal error. This bug was introduced in version 5.8.1.

---

This release is a small internal refactoring to how shrinking interacts with targeted property-based testing that should have no user visible impact.

---

This release improves our support for datetimes and times around DST transitions.

~hypothesis.strategies.times and ~hypothesis.strategies.datetimes are now sometimes generated with fold=1, indicating that they represent the second occurrence of a given wall-time when clocks are set backwards. This may be set even when there is no transition, in which case the fold value should be ignored.

For consistency, timezones provided by the pytz package can now generate imaginary times (such as the hour skipped over when clocks 'spring forward' to daylight saving time, or during some historical timezone transitions). All other timezones have always supported generation of imaginary times.

If you prefer the previous behaviour, ~hypothesis.strategies.datetimes now takes an argument allow_imaginary which defaults to True but can be set to False for any timezones strategy.

---

This patch fixes the rendering of ~hypothesis.strategies.binary docstring by using the proper backticks syntax.

---

Failing tests which use ~hypothesis.target now report the highest score observed for each target alongside the failing example(s), even without explicitly showing test statistics.

This improves the debugging workflow for tests of accuracy, which assert that the total imprecision is within some error budget - for example, abs(a - b) < 0.5. Previously, shrinking to a minimal failing example could often make errors seem smaller or more subtle than they really are (see the threshold problem, and 2180).

---

This patch improves the docstring of ~hypothesis.strategies.binary, the python:repr of ~hypothesis.strategies.sampled_from on an python:enum.Enum subclass, and a warning in our pytest plugin. There is no change in runtime behaviour.

---

This release (potentially very significantly) improves the performance of failing tests in some rare cases, mostly only relevant when using targeted property-based testing, by stopping further optimisation of unrelated test cases once a failing example is found.

---