Python Pytest Cov Configuration and Coverage Reports

When you run pytest with pytest-cov, the plugin measures which lines of your code are executed during the test run. Configuring pytest-cov properly determines not only what gets measured but also how the results are reported and enforced. This article covers the practical aspects of python pytest cov configuration and coverage reports, from basic flags to CI integration.

Installing pytest-cov

pytest-cov is a third-party plugin that integrates coverage.py with pytest. Install it with pip:

pip install pytest-cov

This also installs coverage.py as a dependency. After installation, the --cov flag becomes available in pytest. You can verify the plugin is active by running:

pytest --version

The output lists pytest-cov among the registered plugins.

Basic Configuration: --cov and --cov-report

The simplest way to enable coverage is to pass the --cov option with the package or module you want to measure:

pytest --cov=myproject

This measures all lines in the myproject package and prints a terminal report after the test run. The report shows the percentage of lines covered per module and an overall total.

To control the output format, use --cov-report. The most common values are term, html, xml, and json. You can specify multiple reports by repeating the flag:

pytest --cov=myproject --cov-report=term --cov-report=html

This prints the terminal report and writes an HTML report into the htmlcov/ directory by default. The xml report is useful for CI systems that parse coverage data, while json provides a machine-readable summary.

Configuring Coverage via .coveragerc or pyproject.toml

Instead of passing all options on the command line, you can define coverage behavior in a configuration file. Coverage.py reads .coveragerc or, if you use pyproject.toml, a [tool.coverage] section. pytest-cov respects these settings.

For example, a .coveragerc file can specify which source files to measure and which to omit:

[run]
source = myproject
omit = myproject/tests/*

[report]
show_missing = true
skip_covered = true

When using pyproject.toml, the same settings look like:

[tool.coverage.run]
source = "myproject"
omit = ["myproject/tests/*"]

[tool.coverage.report]
show_missing = true
skip_covered = true

These files also allow you to set fail_under, which is the minimum coverage percentage that must be reached for the test run to pass. You can set it in the [report] section:

[report]
fail_under = 90

Or in pyproject.toml:

[tool.coverage.report]
fail_under = 90

When the measured coverage falls below this threshold, pytest exits with a non-zero status, which is essential for CI gates.

Combining Coverage from Multiple Test Runs

In larger projects, you may run tests in parallel or across multiple processes. By default, each pytest process writes its own coverage data, and the final report only reflects the last run. To combine coverage, use the --cov-append flag:

pytest --cov=myproject --cov-append

This appends the current run's data to the existing .coverage file. Alternatively, you can run pytest with -n (pytest-xdist) and use --cov with the parallel mode. The [run] section in .coveragerc can set parallel = true to write separate data files that coverage.py can later combine with the coverage combine command.

For CI pipelines that split tests into multiple jobs, you can collect .coverage.* files from each job, combine them, and generate a single report. This is a common pattern for reducing test time while keeping accurate coverage.

Report Formats and Their Use Cases

The table below summarizes the main report formats available with pytest-cov:

You can combine any of these. For example, in CI you might generate both XML and terminal reports:

pytest --cov=myproject --cov-report=xml --cov-report=term

Enforcing Coverage Thresholds in CI

Setting a coverage threshold ensures that new code does not reduce overall coverage. Use --cov-fail-under on the command line or fail_under in the configuration file. For example, to require at least 85% coverage:

pytest --cov=myproject --cov-fail-under=85

If the actual coverage is below 85%, pytest exits with status 1, causing the CI job to fail. This is a straightforward way to enforce a minimum standard.

For more granular control, you can set per-package thresholds using the [report] section's show_missing and skip_covered options, but fail_under is global. If you need different thresholds for different parts of the codebase, you would need to split the test runs or use a custom script.

Handling Common Configuration Pitfalls

A frequent mistake is forgetting to specify the source package. Without --cov or a source setting, coverage.py measures everything imported during the test run, which often includes third-party libraries. This produces misleadingly low percentages. Always restrict coverage to your own code.

Another pitfall is omitting test files from coverage. If your tests are inside the package, they will be counted unless you explicitly omit them. Use omit in the configuration to exclude test modules:

[run]
omit = */tests/*

Also, be aware that --cov takes a path or package name. If you pass a relative path, it must be resolvable from the current working directory. In CI, this can break if the working directory changes. Prefer absolute paths or package names that are importable.

Finally, when using pyproject.toml, ensure the [tool.coverage] section is in the same file that pytest reads. If you have a setup.cfg or tox.ini, coverage.py also reads those, but the precedence can be confusing. The rule is that coverage.py looks for .coveragerc, then pyproject.toml, then setup.cfg, then tox.ini. If you have multiple files, the first one found wins.

Performance and Overhead Considerations

Coverage measurement adds overhead to every test run because the Python interpreter must track which lines are executed. The overhead is typically 20–30% for a well-optimized suite, but it can be higher if you measure many modules or use branch coverage. For large projects, this can make the difference between a fast and a slow test cycle.

To reduce overhead, you can:

• Limit coverage to the specific packages you care about with --cov.
• Use --cov-branch only when you need branch coverage; it increases overhead further.
• Run coverage only in CI, not during local development, unless you are actively investigating coverage gaps.

If you run tests in parallel with pytest-xdist, pytest-cov can merge data from each worker, but the merging itself adds a small cost. In practice, the overhead is acceptable for most projects, but it is worth measuring if your test suite is large.

Branch Coverage: When to Enable It

By default, coverage.py measures line coverage only. Branch coverage tracks whether each possible branch in a conditional statement is executed. For example, an if statement has two branches: true and false. Branch coverage tells you if both are taken.

Enable branch coverage with --cov-branch or by setting branch = true in the [run] section:

pytest --cov=myproject --cov-branch

The terminal report will then show a Branch column with the percentage of branches covered. Branch coverage is stricter and can reveal untested edge cases, but it also increases the number of coverage points and the overhead. Use it when you need to ensure that error paths and alternative conditions are exercised.

When branch coverage is enabled, fail_under applies to the combined line and branch coverage percentage. The exact calculation is based on the number of covered statements and branches out of the total. This is useful for teams that want a single metric that accounts for both dimensions.

Combining Coverage Data from Parallel Runs

In a CI pipeline that runs tests in multiple jobs, each job produces a separate .coverage file. To get a unified report, you need to combine them. Coverage.py provides the combine command:

coverage combine

This merges all .coverage.* files in the current directory into a single .coverage file. Then you can generate reports with coverage report or coverage xml. When using pytest-cov, you can also use --cov-append to accumulate data in a single process, but that is not suitable for parallel jobs because each job runs in isolation.

A common pattern is to upload coverage artifacts from each job and run a separate job that downloads them and runs coverage combine. This keeps the test jobs fast and produces an accurate overall coverage number.

Handling Coverage of Dynamic Code and Imports

Coverage.py tracks lines that are executed at runtime. Code that is imported but never called will show as missing. This is expected. However, some patterns can cause false negatives. For example, code that is executed during module import will be marked as covered even if no test explicitly calls it. This is usually fine, but it can mask missing test coverage for initialization logic.

Another edge case is code that uses exec or eval. Coverage.py cannot reliably track lines executed via exec because the source is not known at compile time. If your code relies heavily on dynamic execution, coverage reports may be incomplete. In such cases, you may need to exclude those lines using # pragma: no cover comments.

To exclude a specific line from coverage, add the comment:

def debug_only():
    if __debug__:  # pragma: no cover
        print("Debug output")

You can also exclude entire blocks with # pragma: no cover on the line before the block. This is useful for code that is intentionally not tested, such as fallback branches that are hard to trigger.

Final Configuration Example

A complete configuration that works for many projects can be set in pyproject.toml:

[tool.coverage.run]
source = "myproject"
omit = ["myproject/tests/*"]
branch = true

[tool.coverage.report]
show_missing = true
skip_covered = true
fail_under = 90

With this in place, you can run:

pytest --cov

and pytest-cov will use the configuration from pyproject.toml. The terminal report will show missing lines, and the run will fail if coverage drops below 90%. You can still override the report format on the command line, for example to generate an HTML report:

pytest --cov --cov-report=html

The configuration file provides a stable baseline, while command-line flags allow ad-hoc adjustments without editing files. This separation keeps the setup predictable across local and CI environments.