Only set dynamic context on coveragepy cores supporting it, perhaps?

[webknjaz]

Summary

Long story short, I run python -Werror pytest --pytest-cov which turns warnings into errors. A few days ago coverage==7.15.3 started emitting a warning in my typical CI setup, failing the jobs loudly (as I'd expect in such cases).

So it appears like some of the core settings in coveragepy behave subtly wrong when the dynamic context feature is in use. It was going unnoticed for quite a while, especially with coveragepy changing defaults depending on the current CPython runtime but now it emits a warning, which is a good thing.

The change log entry's here https://coverage.readthedocs.io/en/latest/changes.html#version-7-15-3-2026-08-02. The patch that broke educated me is coveragepy/coveragepy#2234. And some context about the underlying buggy behavior in the lib is coveragepy/coveragepy#2200.

Obviously, I can find a way to suppress it or find a workaround, whatever. But let's see if there's anything pytest-cov could be clever about...

Expected vs actual result

I wish that the pytest plugin could either detect this incompatibility and show a better error message with a cleaner guidance on its level or switch over to using a different core that is indeed compatible w/ dynamic contexts.

Reproducer

Pass --cov-context=test, I suppose. Python 3.14+ too, maybe. Or an explicitly set core through an env var or a config setting. In my case, it's in pytest.ini's addopts and so finding out what was causing the problem was an advanture. I guess it's sorta documented here https://coverage.readthedocs.io/en/latest/contexts.html#dynamic-contexts.

Versions

Here's what's in the tox env, but the main culprit is coveragepy released on Aug 2.

py: anyio==4.14.2,argon2-cffi==25.1.0,argon2-cffi-bindings==25.1.0,attrs==26.1.0,build==1.5.0,cachetools==7.1.7,certifi==2026.7.22,cffi==2.1.1,charset-normalizer==3.4.9,check-manifest==0.51,colorama==0.4.6,covdefaults==2.3.0,coverage==7.15.3,defusedxml==0.7.1,devpi-client==7.3.0,devpi-common==4.1.1,devpi-process==1.2.0,devpi-server==6.20.3,distlib==0.4.3,execnet==2.1.2,filelock==3.32.2,h11==0.16.0,httpcore==1.0.9,httpx==0.28.1,hupper==1.12.1,idna==3.18,iniconfig==2.3.0,itsdangerous==2.2.0,lazy==2.0,legacy-cgi==2.6.4,packaging==26.2,packaging-legacy==23.0.post0,passlib==1.7.4,PasteDeploy==3.1.0,pip==26.2,pkginfo==1.12.1.2,plaster==1.1.2,plaster-pastedeploy==1.0.1,platformdirs==4.11.0,pluggy==1.6.0,py==1.11.0,pycparser==3.0,Pygments==2.20.0,pyproject-api==1.11.0,pyproject_hooks==1.2.0,pyramid==2.1,pytest==9.1.1,pytest-cov==7.1.0,pytest-mock==3.15.1,pytest-xdist==3.8.0,python-dateutil==2.9.0.post0,python-discovery==1.5.1,repoze-lru==0.8,requests==2.34.2,ruamel.yaml==0.19.1,setuptools==81.0.0,six==1.17.0,strictyaml==1.7.3,tomli_w==1.2.0,tox==4.58.0,tox-pre-commit @ file:///home/runner/work/tox-pre-commit/tox-pre-commit/dist/tox_pre_commit-0.0.1a1.dev5%2Bg32376c46f-py3-none-any.whl#sha256=2b4127f94ecab75b0144d68efa2fd9a1275c70a64ab8748340bd66830828c850,translationstring==1.4,urllib3==2.7.0,venusian==3.1.1,virtualenv==21.7.1,waitress==3.0.2,WebOb==1.8.11,zope.deprecation==6.0,zope.interface==8.5

Config

Any relevant config can be located in this repo snapshot: https://github.com/tox-dev/tox-pre-commit/tree/32376c46fdea0d11264d3efdbe11b7faba52f92f.

Trace

Here's a paste from https://github.com/tox-dev/tox-pre-commit/actions/runs/30888801268/job/91926043047#step:16:553 for history (GHA logs expire in a few months):

$ python -Xutf8 -Im tox --parallel auto --parallel-live --skip-missing-interpreters false --skip-pkg-install --quiet  -- --cov-report=xml:.tox/.tmp/.test-results/pytest-3.14/cobertura.xml --junitxml=.tox/.tmp/.test-results/pytest-3.14/test.xml
  shell: /usr/bin/bash -e {0}
  env:
    COLOR: yes
    FORCE_COLOR: 1
    MYPY_FORCE_COLOR: 1
    PIP_DISABLE_PIP_VERSION_CHECK: 1
    PIP_NO_PYTHON_VERSION_WARNING: 1
    PIP_NO_WARN_SCRIPT_LOCATION: 1
    PRE_COMMIT_COLOR: always
    PY_COLORS: 1
    PYTHONIOENCODING: utf-8
    PYTHONUTF8: 1
    TOX_PARALLEL_NO_SPINNER: 1
    TOX_TESTENV_PASSENV: CI COLOR FORCE_COLOR GITHUB_* MYPY_FORCE_COLOR NO_COLOR PIP_DISABLE_PIP_VERSION_CHECK PIP_NO_PYTHON_VERSION_WARNING PIP_NO_WARN_SCRIPT_LOCATION PRE_COMMIT_COLOR PY_COLORS PYTEST_THEME PYTEST_THEME_MODE PYTHONDONTWRITEBYTECODE PYTHONIOENCODING PYTHONLEGACYWINDOWSSTDIO PYTHONTRACEMALLOC PYTHONUTF8
    TOXENV: py
    pythonLocation: /opt/hostedtoolcache/Python/3.14.6/arm64
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.14.6/arm64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.14.6/arm64
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.14.6/arm64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.14.6/arm64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.14.6/arm64/lib
============================= test session starts ==============================
platform linux -- Python 3.14.6, pytest-9.1.1, pluggy-1.6.0
cachedir: .tox/py/.pytest_cache
rootdir: /home/runner/work/tox-pre-commit/tox-pre-commit
configfile: pytest.ini
testpaths: tests/
plugins: cov-7.1.0, mock-3.15.1, anyio-4.14.2, xdist-3.8.0
collected 5 items

tests/importable_test.py EE                                              [ 20%]
tests/integration_test.py EEEEEE                                         [ 80%]
tests/smoke_test.py EE                                                   [100%]

==================================== ERRORS ====================================
_ ERROR at setup of test_entry_points_exposed[tox-pre-commit=tox_plugins.pre_commit._plugin@tox] _

self = <pytest_cov.plugin.TestContextPlugin object at 0xffb6c5827a10>
item = <Function test_entry_points_exposed[tox-pre-commit=tox_plugins.pre_commit._plugin@tox]>

    def pytest_runtest_setup(self, item):
>       self.switch_context(item, 'setup')

item       = <Function test_entry_points_exposed[tox-pre-commit=tox_plugins.pre_commit._plugin@tox]>
self       = <pytest_cov.plugin.TestContextPlugin object at 0xffb6c5827a10>

.tox/py/lib/python3.14/site-packages/pytest_cov/plugin.py:440: 
        if slug in self._no_warn_slugs:
            # Don't issue the warning
            return
    
        self._warnings.append(msg)
        if slug:
            msg = f"{msg} ({slug}); see {__url__}/messages.html#warning-{slug}"
        if self._debug.should("pid"):
            msg = f"[{os.getpid()}] {msg}"
>       warnings.warn(msg, category=CoverageWarning, stacklevel=2)
E       coverage.exceptions.CoverageWarning: Dynamic contexts aren't supported with core=sysmon; context data may be incomplete (no-sysmon-context); see https://coverage.readthedocs.io/en/7.15.3/messages.html#warning-no-sysmon-context

msg        = "Dynamic contexts aren't supported with core=sysmon; context data may be incomplete (no-sysmon-context); see https://coverage.readthedocs.io/en/7.15.3/messages.html#warning-no-sysmon-context"
once       = True
self       = <Coverage @0xffb6c66c4c20 core=SysMonitor data_file='/home/runner/work/tox-pre-commit/tox-pre-commit/.coverage.runnervma9114.pid2310.XBUfNXyx'>
slug       = 'no-sysmon-context'

.tox/py/lib/python3.14/site-packages/coverage/control.py:495: CoverageWarning
- generated xml file: /home/runner/work/tox-pre-commit/tox-pre-commit/.tox/.tmp/.test-results/pytest-3.14/test.xml -
============================= slowest 10 durations =============================

(10 durations < 0.005s hidden.  Use -vv to show these durations.)
=========================== short test summary info ============================
ERROR tests/importable_test.py::test_entry_points_exposed[tox-pre-commit=tox_plugins.pre_commit._plugin@tox] - coverage.exceptions.CoverageWarning: Dynamic contexts aren't supported with core=sysmon; context data may be incomplete (no-sysmon-context); see https://coverage.readthedocs.io/en/7.15.3/messages.html#warning-no-sysmon-context
ERROR tests/importable_test.py::test_entry_points_exposed[tox-pre-commit=tox_plugins.pre_commit._plugin@tox] - coverage.exceptions.CoverageWarning: Dynamic contexts aren't supported with core=sysmon; context data may be incomplete (no-sysmon-context); see https://coverage.readthedocs.io/en/7.15.3/messages.html#warning-no-sysmon-context
ERROR tests/integration_test.py::test_pre_commit_env_registered - coverage.exceptions.CoverageWarning: Dynamic contexts aren't supported with core=sysmon; context data may be incomplete (no-sysmon-context); see https://coverage.readthedocs.io/en/7.15.3/messages.html#warning-no-sysmon-context
ERROR tests/integration_test.py::test_pre_commit_env_registered - coverage.exceptions.CoverageWarning: Dynamic contexts aren't supported with core=sysmon; context data may be incomplete (no-sysmon-context); see https://coverage.readthedocs.io/en/7.15.3/messages.html#warning-no-sysmon-context
ERROR tests/integration_test.py::test_pre_commit_default_command - coverage.exceptions.CoverageWarning: Dynamic contexts aren't supported with core=sysmon; context data may be incomplete (no-sysmon-context); see https://coverage.readthedocs.io/en/7.15.3/messages.html#warning-no-sysmon-context
ERROR tests/integration_test.py::test_pre_commit_default_command - coverage.exceptions.CoverageWarning: Dynamic contexts aren't supported with core=sysmon; context data may be incomplete (no-sysmon-context); see https://coverage.readthedocs.io/en/7.15.3/messages.html#warning-no-sysmon-context
ERROR tests/integration_test.py::test_pre_commit_posargs_replace_default - coverage.exceptions.CoverageWarning: Dynamic contexts aren't supported with core=sysmon; context data may be incomplete (no-sysmon-context); see https://coverage.readthedocs.io/en/7.15.3/messages.html#warning-no-sysmon-context
ERROR tests/integration_test.py::test_pre_commit_posargs_replace_default - coverage.exceptions.CoverageWarning: Dynamic contexts aren't supported with core=sysmon; context data may be incomplete (no-sysmon-context); see https://coverage.readthedocs.io/en/7.15.3/messages.html#warning-no-sysmon-context
ERROR tests/smoke_test.py::test_smoke - coverage.exceptions.CoverageWarning: Dynamic contexts aren't supported with core=sysmon; context data may be incomplete (no-sysmon-context); see https://coverage.readthedocs.io/en/7.15.3/messages.html#warning-no-sysmon-context
ERROR tests/smoke_test.py::test_smoke - coverage.exceptions.CoverageWarning: Dynamic contexts aren't supported with core=sysmon; context data may be incomplete (no-sysmon-context); see https://coverage.readthedocs.io/en/7.15.3/messages.html#warning-no-sysmon-context
============================== 10 errors in 0.43s ==============================
py: exit 1 (1.52 seconds) /home/runner/work/tox-pre-commit/tox-pre-commit> .tox/py/bin/python -bb -E -s -I -Werror -m pytest --cov-report=xml:.tox/.tmp/.test-results/pytest-3.14/cobertura.xml --junitxml=.tox/.tmp/.test-results/pytest-3.14/test.xml pid=2310
Exception ignored while finalizing database connection <sqlite3.Connection object at 0xff9f2b412e30>:
ResourceWarning: unclosed database in <sqlite3.Connection object at 0xff9f2b412e30>
  py: FAIL code 1 (2.11 seconds)
  evaluation failed :( (2.13 seconds)

Error: Process completed with exit code 1.

[webknjaz]

Search string for people from search engines:

Dynamic contexts aren't supported with core=sysmon; context data may be incomplete (no-sysmon-context); see https://coverage.readthedocs.io/en/7.15.3/messages.html#warning-no-sysmon-context

[ewjoachim]

(fancy seeing you here :) )

So if I understand correctly:

Until coveragepy/coveragepy#2150 is fixed, we should either:

• Disable contexts
• Silence warnings (disable_warnings = ["no-sysmon-context"] or filterwarnings BUT this means coverage is wrong)
• Disable sysmon ([tool.coverage.run] core = "ctrace") but slower and nothing will tell us when the issue is fixed and we can switch it back

Have I got it right ?

[webknjaz]

Hey, haven't heard from you in a while! Yeah, this seems about right per my understanding. I haven't yet gotten to having a standardized solution across my standardized scaffolding, but it looks like I'll probably go for setting in, but maybe do something dynamic via tox/coverage plugins and/or GHA/env vars. Will see.