hypothesis

This is a patch release for ~hypothesis.strategies.from_regex, which had a bug in handling of the python:re.VERBOSE flag (992). Flags are now handled correctly when parsing regex.

---

This patch changes a few byte-string literals from double to single quotes, thanks to an update in unify. There are no user-visible changes.

---

This fixes the repr of strategies using lambda that are defined inside decorators to include the lambda source.

This would mostly have been visible when using the statistics functionality - lambdas used for e.g. filtering would have shown up with a <unknown> as their body. This can still happen, but it should happen less often now.

---

This release updates the reported statistics so that they show approximately what fraction of your test run time is spent in data generation (as opposed to test execution).

This work was funded by Smarkets.

---

This is a documentation release, which ensures code examples are up to date by running them as doctests in CI (711).

---

This release changes the behaviour of the ~hypothesis.settings.deadline setting when used with ~hypothesis.strategies.data: Time spent inside calls to data.draw will no longer be counted towards the deadline time.

As a side effect of some refactoring required for this work, the way flaky tests are handled has changed slightly. You are unlikely to see much difference from this, but some error messages will have changed.

This work was funded by Smarkets.

---

This patch has a variety of non-user-visible refactorings, removing various minor warts ranging from indirect imports to typos in comments.

---

This release overhauls the |HealthCheck| system in a variety of small ways. It adds no new features, but is nevertheless a minor release because it changes which tests are likely to fail health checks.

The most noticeable effect is that some tests that used to fail health checks will now pass, and some that used to pass will fail. These should all be improvements in accuracy. In particular:

• New failures will usually be because they are now taking into account things like use of ~hypothesis.strategies.data and ~hypothesis.assume inside the test body.

• New failures may also be because for some classes of example the way data generation performance was measured was artificially faster than real data generation (for most examples that are hitting performance health checks the opposite should be the case).

• Tests that used to fail health checks and now pass do so because the health check system used to run in a way that was subtly different than the main Hypothesis data generation and lacked some of its support for e.g. large examples.

If your data generation is especially slow, you may also see your tests get somewhat faster, as there is no longer a separate health check phase. This will be particularly noticeable when rerunning test failures.

This work was funded by Smarkets.

---

This is a deprecation release for some health check related features.

The following are now deprecated:

• Passing HealthCheck.exception_in_generation to ~hypothesis.settings.suppress_health_check. This no longer does anything even when passed - All errors that occur during data generation will now be immediately reraised rather than going through the health check mechanism.

• Passing HealthCheck.random_module to ~hypothesis.settings.suppress_health_check. This hasn't done anything for a long time, but was never explicitly deprecated. Hypothesis always seeds the random module when running @given tests, so this is no longer an error and suppressing it doesn't do anything.

• Passing non-~hypothesis.HealthCheck values in ~hypothesis.settings.suppress_health_check. This was previously allowed but never did anything useful.

In addition, passing a non-iterable value as ~hypothesis.settings.suppress_health_check will now raise an error immediately (it would never have worked correctly, but it would previously have failed later). Some validation error messages have also been updated.

This work was funded by Smarkets.

---

This is a yak shaving release, mostly concerned with our own tests.

While ~python:inspect.getfullargspec was documented as deprecated in Python 3.5, it never actually emitted a warning. Our code to silence this (nonexistent) warning has therefore been removed.

We now run our tests with DeprecationWarning as an error, and made some minor changes to our own tests as a result. This required similar upstream updates to coverage and execnet (a test-time dependency via pytest-xdist).

There is no user-visible change in Hypothesis itself, but we encourage you to consider enabling deprecations as errors in your own tests.

---

This release adds a setting to the public API, and does some internal cleanup:

• The ~hypothesis.settings.derandomize setting is now documented (890)

• Removed - and disallowed - all 'bare excepts' in Hypothesis (953)

• Documented the strict setting as deprecated, and updated the build so our docs always match deprecations in the code.

---

This minor release supports constraining ~hypothesis.strategies.uuids to generate a particular version of ~python:uuid.UUID (721).

Thanks to Dion Misic for this feature.

---

This patch updates the documentation to suggest builds(callable) instead of just(callable()).

---

Hypothesis now emits deprecation warnings if you apply @given more than once to a target.

Applying @given repeatedly wraps the target multiple times. Each wrapper will search the space of possible parameters separately. This is equivalent but will be much more inefficient than doing it with a single call to @given.

For example, instead of @given(booleans()) @given(integers()), you could write @given(booleans(), integers())

---

This is a bugfix release:

• ~hypothesis.strategies.builds would try to infer a strategy for required positional arguments of the target from type hints, even if they had been given to ~hypothesis.strategies.builds as positional arguments (946). Now it only infers missing required arguments.

• An internal introspection function wrongly reported self as a required argument for bound methods, which might also have affected ~hypothesis.strategies.builds. Now it knows better.

---

This release supports strategy inference for more Django field types - you can now omit an argument for Date, Time, Duration, Slug, IP Address, and UUID fields. (642)

Strategy generation for fields with grouped choices now selects choices from each group, instead of selecting from the group names.

---

This patch removes the mergedb tool, introduced in Hypothesis 1.7.1 on an experimental basis. It has never actually worked, and the new Hypothesis example database is designed to make such a tool unnecessary.

---

This patch has two improvements for strategies based on enumerations.

• ~hypothesis.strategies.from_type now handles enumerations correctly, delegating to ~hypothesis.strategies.sampled_from. Previously it noted that Enum.__init__ has no required arguments and therefore delegated to ~hypothesis.strategies.builds, which would subsequently fail.

• When sampling from an python:enum.Flag, we also generate combinations of members. Eg for Flag('Permissions', 'READ, WRITE, EXECUTE') we can now generate, Permissions.READ, Permissions.READ|WRITE, and so on.

---

This changes the default value of the use_coverage setting to True when running on pypy (it was already True on CPython).

It was previously set to False because we expected it to be too slow, but recent benchmarking shows that actually performance of the feature on pypy is fairly acceptable - sometimes it's slower than on CPython, sometimes it's faster, but it's generally within a factor of two either way.

---

This patch improves the quality of strategies inferred from Numpy dtypes:

• Integer dtypes generated examples with the upper half of their (non-sign) bits set to zero. The inferred strategies can now produce any representable integer.

• Fixed-width unicode- and byte-string dtypes now cap the internal example length, which should improve example and shrink quality.

• Numpy arrays can only store fixed-size strings internally, and allow shorter strings by right-padding them with null bytes. Inferred string strategies no longer generate such values, as they can never be retrieved from an array. This improves shrinking performance by skipping useless values.

This has already been useful in Hypothesis - we found an overflow bug in our Pandas support, and as a result ~hypothesis.extra.pandas.indexes and ~hypothesis.extra.pandas.range_indexes now check that min_size and max_size are at least zero.

---

This release fixes a performance problem in tests where the use_coverage setting is True.

Tests experience a slow-down proportionate to the amount of code they cover. This is still the case, but the factor is now low enough that it should be unnoticeable. Previously it was large and became much larger in 3.30.4.

---

~hypothesis.strategies.from_type failed with a very confusing error if passed a ~typing.NewType (901). These pseudo-types are now unwrapped correctly, and strategy inference works as expected.

---

This release makes some small optimisations to our use of coverage that should reduce constant per-example overhead. This is probably only noticeable on examples where the test itself is quite fast. On no-op tests that don't test anything you may see up to a fourfold speed increase (which is still significantly slower than without coverage). On more realistic tests the speed up is likely to be less than that.

---

This release fixes some formatting and small typos/grammar issues in the documentation, specifically the page docs/settings.rst, and the inline docs for the various settings.

---

This release improves the handling of deadlines so that they act better with the shrinking process. This fixes 892.

This involves two changes:

1. The deadline is raised during the initial generation and shrinking, and then lowered to the set value for final replay. This restricts our attention to examples which exceed the deadline by a more significant margin, which increases their reliability.

2. When despite the above a test still becomes flaky because it is significantly faster on rerun than it was on its first run, the error message is now more explicit about the nature of this problem, and includes both the initial test run time and the new test run time.

In addition, this release also clarifies the documentation of the deadline setting slightly to be more explicit about where it applies.

This work was funded by Smarkets.

---

This release blocks installation of Hypothesis on Python 3.3, which :PEP:`reached its end of life date on 2017-09-29 <398>`.

This should not be of interest to anyone but downstream maintainers - if you are affected, migrate to a secure version of Python as soon as possible or at least seek commercial support.

---

This release makes several changes:

1. It significantly improves Hypothesis's ability to use coverage information to find interesting examples.

2. It reduces the default ~hypothesis.settings.max_examples setting from 200 to 100. This takes advantage of the improved algorithm meaning fewer examples are typically needed to get the same testing and is sufficiently better at covering interesting behaviour, and offsets some of the performance problems of running under coverage.

3. Hypothesis will always try to start its testing with an example that is near minimized.

The new algorithm for 1 also makes some changes to Hypothesis's low level data generation which apply even with coverage turned off. They generally reduce the total amount of data generated, which should improve test performance somewhat. Between this and 3 you should see a noticeable reduction in test runtime (how much so depends on your tests and how much example size affects their performance. On our benchmarks, where data generation dominates, we saw up to a factor of two performance improvement, but it's unlikely to be that large.

---

This release fixes some formatting and small typos/grammar issues in the documentation, specifically the page docs/details.rst, and some inline docs linked from there.

---

This release changes Hypothesis's caching approach for functions in hypothesis.strategies. Previously it would have cached extremely aggressively and cache entries would never be evicted. Now it adopts a least-frequently used, least recently used key invalidation policy, and is somewhat more conservative about which strategies it caches.

Workloads which create strategies based on dynamic values, e.g. by using |.flatmap| or ~hypothesis.strategies.composite, will use significantly less memory.

---

This release fixes a bug where when running with the use_coverage=True setting inside an existing running instance of coverage, Hypothesis would frequently put files that the coveragerc excluded in the report for the enclosing coverage.

---

This release introduces two new features:

• When a test fails, either with a health check failure or a falsifying example, Hypothesis will print out a seed that led to that failure, if the test is not already running with a fixed seed. You can then recreate that failure using either the @seed decorator or (if you are running pytest) with --hypothesis-seed.

• pytest users can specify a seed to use for @given based tests by passing the --hypothesis-seed command line argument.

This work was funded by Smarkets.

---

This release makes Hypothesis coverage aware. Hypothesis now runs all test bodies under coverage, and uses this information to guide its testing.

The use_coverage setting can be used to disable this behaviour if you want to test code that is sensitive to coverage being enabled (either because of performance or interaction with the trace function).

The main benefits of this feature are:

• Hypothesis now observes when examples it discovers cover particular lines or branches and stores them in the database for later.

• Hypothesis will make some use of this information to guide its exploration of the search space and improve the examples it finds (this is currently used only very lightly and will likely improve significantly in future releases).

This also has the following side-effects:

• Hypothesis now has an install time dependency on the coverage package.

• Tests that are already running Hypothesis under coverage will likely get faster.

• Tests that are not running under coverage now run their test bodies under coverage by default.

This feature is only partially supported under pypy. It is significantly slower than on CPython and is turned off by default as a result, but it should still work correctly if you want to use it.

---

This release is an internal change that affects how Hypothesis handles calculating certain properties of strategies.

The primary effect of this is that it fixes a bug where use of ~hypothesis.strategies.deferred could sometimes trigger an internal assertion error. However the fix for this bug involved some moderately deep changes to how Hypothesis handles certain constructs so you may notice some additional knock-on effects.

In particular the way Hypothesis handles drawing data from strategies that cannot generate any values has changed to bail out sooner than it previously did. This may speed up certain tests, but it is unlikely to make much of a difference in practice for tests that were not already failing with Unsatisfiable.

---

This is a patch release that fixes a bug in the hypothesis.extra.pandas documentation where it incorrectly referred to ~hypothesis.extra.pandas.column instead of ~hypothesis.extra.pandas.columns.

---

This is a refactoring release. It moves a number of internal uses of ~python:collections.namedtuple over to using attrs based classes, and removes a couple of internal namedtuple classes that were no longer in use.

It should have no user visible impact.

---

This release adds support for testing pandas via the hypothesis.extra.pandas module.

It also adds a dependency on attrs.

This work was funded by Stripe.

---

This release fixes some formatting and broken cross-references in the documentation, which includes editing docstrings - and thus a patch release.

---

This release introduces a ~hypothesis.settings.deadline setting to Hypothesis.

When set this turns slow tests into errors. By default it is unset but will warn if you exceed 200ms, which will become the default value in a future release.

This work was funded by Smarkets.

---

Hypothesis now emits deprecation warnings if you are using the legacy SQLite example database format, or the tool for merging them. These were already documented as deprecated, so this doesn't change their deprecation status, only that we warn about it.

---

This release fixes a bug with generating numpy datetime and timedelta types: When inferring the strategy from the dtype, datetime and timedelta dtypes with sub-second precision would always produce examples with one second resolution. Inferring a strategy from a time dtype will now always produce example with the same precision.

---

This release changes how Hypothesis shrinks and replays examples to take into account that it can encounter new bugs while shrinking the bug it originally found. Previously it would end up replacing the originally found bug with the new bug and show you only that one. Now it is (often) able to recognise when two bugs are distinct and when it finds more than one will show both.

---

This release removes the (purely internal and no longer useful) strategy_test_suite function and the corresponding strategytests module.

---

This release improves the reduction of examples involving floating point numbers to produce more human readable examples.

It also has some general purpose changes to the way the minimizer works internally, which may see some improvement in quality and slow down of test case reduction in cases that have nothing to do with floating point numbers.

---

Hypothesis now emits deprecation warnings if you use some_strategy.example() inside a test function or strategy definition (this was never intended to be supported, but is sufficiently widespread that it warrants a deprecation path).

---

This is a bugfix release for ~hypothesis.strategies.decimals with the places argument.

• No longer fails health checks (725, due to internal filtering)

• Specifying a min_value and max_value without any decimals with places places between them gives a more useful error message.

• Works for any valid arguments, regardless of the decimal precision context.

---

This is a small refactoring release that removes a now-unused parameter to an internal API. It shouldn't have any user visible effect.

---

Hypothesis no longer propagates the dynamic scope of settings into strategy definitions.

This release is a small change to something that was never part of the public API and you will almost certainly not notice any effect unless you're doing something surprising, but for example the following code will now give a different answer in some circumstances:

import hypothesis.strategies as st
from hypothesis import settings

CURRENT_SETTINGS = st.builds(lambda: settings.default)

(We don't actually encourage you writing code like this)

Previously this would have generated the settings that were in effect at the point of definition of CURRENT_SETTINGS. Now it will generate the settings that are used for the current test.

It is very unlikely to be significant enough to be visible, but you may also notice a small performance improvement.

---

This release adds a unique argument to ~hypothesis.extra.numpy.arrays which behaves the same ways as the corresponding one for ~hypothesis.strategies.lists, requiring all of the elements in the generated array to be distinct.

---

This release fixes an issue where Hypothesis would raise a TypeError when using the datetime-related strategies if running with PYTHONOPTIMIZE=2. This bug was introduced in 3.20.0. (See 822)

---

Hypothesis now transparently handles problems with an internal unicode cache, including file truncation or read-only filesystems (767). Thanks to Sam Hames for the patch.

---

This release provides what should be a substantial performance improvement to numpy arrays generated using provided numpy support, and adds a new fill_value argument to ~hypothesis.extra.numpy.arrays to control this behaviour.

This work was funded by Stripe.

---

This release fixes some extremely specific circumstances that probably have never occurred in the wild where users of ~hypothesis.strategies.deferred might have seen a python:RuntimeError from too much recursion, usually in cases where no valid example could have been generated anyway.

---

This release fixes some minor bugs in argument validation:

• hypothesis.extra.numpy dtype strategies would raise an internal error instead of an InvalidArgument exception when passed an invalid endianness specification.

• ~hypothesis.strategies.fractions would raise an internal error instead of an InvalidArgument if passed float("nan") as one of its bounds.

• The error message for passing float("nan") as a bound to various strategies has been improved.

• Various bound arguments will now raise InvalidArgument in cases where they would previously have raised an internal TypeError or ValueError from the relevant conversion function.

• streaming() would not have emitted a deprecation warning when called with an invalid argument.

---

This release fixes a bug where test failures that were the result of an @example would print an extra stack trace before re-raising the exception.

---

This release deprecates Hypothesis's strict mode, which turned Hypothesis's deprecation warnings into errors. Similar functionality can be achieved by using simplefilter('error', HypothesisDeprecationWarning).

---

This release renames the relevant arguments on the ~hypothesis.strategies.datetimes, ~hypothesis.strategies.dates, ~hypothesis.strategies.times, and ~hypothesis.strategies.timedeltas strategies to min_value and max_value, to make them consistent with the other strategies in the module.

The old argument names are still supported but will emit a deprecation warning when used explicitly as keyword arguments. Arguments passed positionally will go to the new argument names and are not deprecated.

---

This release provides a major overhaul to the internals of how Hypothesis handles shrinking.

This should mostly be visible in terms of getting better examples for tests which make heavy use of ~hypothesis.strategies.composite, ~hypothesis.strategies.data or |.flatmap| where the data drawn depends a lot on previous choices, especially where size parameters are affected. Previously Hypothesis would have struggled to reliably produce good examples here. Now it should do much better. Performance should also be better for examples with a non-zero min_size.

You may see slight changes to example generation (e.g. improved example diversity) as a result of related changes to internals, but they are unlikely to be significant enough to notice.

---