Python Hypothesis Property-Based Testing Basics

python hypothesis property based testing basics requires a clear understanding of the core syntax, runtime behavior, and practical implementation patterns demonstrated in the examples below.

Writing unit tests for a function that accepts a wide range of inputs often means manually listing edge cases. For a simple function like reverse, you might test an empty list, a single element, a list with duplicates, and a large list. Property-based testing with Python Hypothesis flips this process: you describe a property that should always hold, and Hypothesis generates hundreds of inputs to verify it. This article covers the basics of using Hypothesis to write property-based tests in Python.

The Core Idea: Properties Instead of Examples

In example-based testing, you write a test that calls the function with a specific input and asserts a specific output. Property-based testing instead asserts an invariant that must hold for a whole class of inputs. For reverse, a natural property is that reversing a list twice returns the original list. Another property is that the first element of the reversed list is the last element of the original list.

Here is how that looks with Hypothesis:

from hypothesis import given, strategies as st

@given(st.lists(st.integers()))
def test_reverse_twice_is_identity(lst):
    assert lst[::-1][::-1] == lst

Hypothesis will generate many lists of integers and run this test with each one. If any input causes the assertion to fail, Hypothesis reports the minimal failing input.

Minimal Setup: Installing Hypothesis and Writing a First Test

Install Hypothesis with pip:

pip install hypothesis

Then create a test file, for example test_reverse.py. The test above works with pytest or unittest. Hypothesis integrates with both. For a standalone script, you can call test_reverse_twice_is_identity() directly, but the typical workflow is to run it through a test runner.

The @given decorator takes one or more strategies as arguments. Each argument corresponds to a parameter of the test function. In the example, lst is generated by st.lists(st.integers()). Hypothesis will run the test with a default of 100 examples, though you can change that with max_examples.

Using Strategies to Define Input Space

Strategies describe the set of values that Hypothesis can generate. The strategies module provides a wide range of built-in strategies:

• st.integers() generates arbitrary integers.
• st.text() generates Unicode strings.
• st.floats() generates floating-point numbers.
• st.lists(elements) generates lists of values produced by elements.
• st.dictionaries(keys, values) generates dictionaries.
• st.booleans() generates True and False.

You can constrain these strategies to focus on realistic inputs. For example, st.integers(min_value=1, max_value=100) only generates integers from 1 to 100. st.lists(st.integers(), min_size=1, max_size=10) generates lists with between 1 and 10 elements. This is useful when your function has preconditions.

Strategies can also be transformed. The .map() method applies a function to each generated value, and .filter() restricts the values to those that satisfy a predicate. For example:

positive_ints = st.integers().filter(lambda x: x > 0)

However, be careful with .filter() when the predicate is too restrictive, because Hypothesis may struggle to find valid inputs.

Combining Strategies for Complex Inputs

For data structures that have multiple fields, you can use the @composite decorator to combine strategies. This is cleaner than chaining .map() and .flatmap().

Suppose you need to generate a dictionary representing a user with a name, age, and email. You can define a composite strategy:

from hypothesis import given, strategies as st, assume

@st.composite
def user_strategy(draw):
    name = draw(st.text(min_size=1, max_size=20))
    age = draw(st.integers(min_value=0, max_value=120))
    email = draw(st.text(min_size=5, max_size=50))
    return {"name": name, "age": age, "email": email}

@given(user_strategy())
def test_user(user):
    assert user["age"] >= 0
    assert len(user["name"]) > 0

The draw function inside the composite strategy pulls a value from the given strategy. This lets you build complex inputs that depend on other generated values. You can also use assume inside a composite to discard invalid combinations, but use it sparingly because it can reduce the efficiency of the search.

How Hypothesis Handles Failures: Shrinking and Reproducing

When a property fails, Hypothesis does not simply report the first failing input. It attempts to shrink the input to a minimal failing example, which makes debugging much easier. For example, if a list of integers fails a property, Hypothesis will try to remove elements and reduce the values to find the smallest list that still fails.

You can see this in action by intentionally writing a buggy function. Suppose you have a function that incorrectly handles negative numbers:

def abs_all(numbers):
    return [abs(n) for n in numbers if n >= 0]

A property test that checks abs_all returns only non-negative numbers would fail for inputs containing negative numbers. Hypothesis would shrink the failing input to something like [-1] instead of a long random list.

To reproduce a specific failure, Hypothesis prints a @reproduce_failure decorator with the version and blob. You can paste that decorator above the test to rerun the exact failing input. This is useful when you want to debug without rerunning the full search.

Runtime Cost and Controlling the Search

Property-based tests generate and execute many examples, so they are typically slower than a few hand-written example tests. The default max_examples is 100, which is enough for many cases but can be increased to 1000 if you need more thorough coverage. Conversely, you can lower it to 10 for a quick smoke test during development.

Hypothesis also supports a deadline parameter to enforce a time limit per example. If an example takes longer than the deadline, the test fails. This is useful for catching performance regressions, but it can be flaky on slow CI machines, so set it carefully.

The runtime cost is usually acceptable because Hypothesis uses a database of previously found failures to focus on edge cases. It also uses a heuristic search that adapts to the input space. For most unit tests, the added confidence outweighs the extra seconds.

Common Mistakes When Writing Property Tests

One common mistake is writing a property that is too weak. For example, testing that a function returns a list of the same length does not verify that the elements are correct. The property should capture the essential behavior of the function.

Another mistake is using strategies that are too narrow. If you restrict inputs to a small range, you may miss the edge cases that property-based testing is meant to find. Let the strategy cover the full domain unless the function has documented preconditions.

Overusing assume can also hurt. assume filters out generated inputs, and if the filter is too strict, Hypothesis may spend many examples rejecting inputs. Prefer to design strategies that only generate valid inputs in the first place.

Finally, remember that property-based tests are not a replacement for example-based tests. They complement each other. Use property tests for invariants and example tests for specific regressions.

When to Choose Property-Based Testing Over Example-Based Tests

Property-based testing shines when you have a function with a large or complex input space, and you can state a clear invariant that must hold. Sorting functions, serializers, parsers, and data transformations are good candidates. If you find yourself writing many example-based tests to cover edge cases, a property test might reduce that burden.

On the other hand, example-based tests are better when the expected behavior is known for a specific input, such as a known bug fix or a documented API contract. They are also easier to read for non-technical stakeholders because they show concrete input-output pairs.

A practical approach is to start with a few example-based tests to lock in known behavior, then add property-based tests to explore the input space. When a property test finds a bug, add that specific case as an example-based regression test.