init

.github/workflows/bench.yaml

@@ -0,0 +1,67 @@
+name: bench
+
+# The bench runs when a human triggers it, against whatever repo+ref they name. No schedules, no push triggers -
+# this is an on-demand measurement instrument, not a monitor.
+
+on:
+  workflow_dispatch:
+    inputs:
+      repo:
+        description: Git repository holding the code under test
+        default: 'https://github.com/Mantisus/crawlee-python'
+      ref:
+        description: Branch, tag or SHA to install
+        default: 'container-limits'
+      scenarios:
+        description: Space-separated scenario names, or "all"
+        default: 'all'
+      cpu_budgets:
+        description: 'Cores for the CPU-limited scenarios, one run per value. Runner has 2 cores (private repo) or 4 (public repo)'
+        default: '[1, 2]'
+      image:
+        description: 'Base image the container scenarios run in. Any glibc distro works - uv is mounted in'
+        default: 'ghcr.io/astral-sh/uv:python3.13-bookworm'
+
+run-name: 'bench: ${{ inputs.ref }} / ${{ inputs.scenarios }} / cpus ${{ inputs.cpu_budgets }}'
+
+permissions:
+  contents: read
+
+jobs:
+  bench:
+    # One pass per CPU budget, each on its own runner, so a smaller budget's result is still there when a bigger
+    # one turns out to exceed the runner.
+    strategy:
+      fail-fast: false
+      matrix:
+        cpus: ${{ fromJSON(inputs.cpu_budgets) }}
+    name: bench (${{ matrix.cpus }} cpu)
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: astral-sh/setup-uv@v5
+
+      - name: Run scenarios
+        env:
+          CRAWLEE_REPO: ${{ inputs.repo }}
+          CRAWLEE_REF: ${{ inputs.ref }}
+          BENCH_CPUS: ${{ matrix.cpus }}
+          IMG: ${{ inputs.image }}
+        run: ./run.sh ${{ inputs.scenarios }} results/
+
+      - name: Report
+        if: always()
+        run: python3 report.py results/ >> "$GITHUB_STEP_SUMMARY"
+
+      - name: Check
+        # The only gate: a probe that produced no data. Skips and observed values are for humans.
+        if: always()
+        run: python3 report.py results/ --check >/dev/null
+
+      - name: Upload raw results
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: results-${{ matrix.cpus }}cpu
+          path: results/

.gitignore

@@ -215,4 +215,4 @@ marimo/_lsp/
 __marimo__/
 
 # Streamlit
-.streamlit/secrets.toml
+.streamlit/secrets.toml

README.md

@@ -1,2 +1,97 @@
 # cgroups-sensor
+
 Utility functions to measure resource limits from cgroups in scenarios where psutils is not sufficient.
+
+Today the repository is a bench for that measurement: it runs a library's cgroup detection inside real
+environment shapes — systemd scopes, containers, kubernetes pods — and prints what the library saw there. It
+runs when a human triggers it and reports to a human; nothing is scheduled and nothing asserts on values.
+
+## Running it
+
+On GitHub: **Actions → bench → Run workflow**. Locally, on any cgroup-v2 Linux box with `uv`:
+
+```bash
+./run.sh                             # every scenario, results into results/
+./run.sh bare docker-private         # only these two
+BENCH_CPUS=2 IMG=fedora:41 ./run.sh  # a different budget, in another distro's userspace
+python3 report.py results/           # the tables
+python3 report.py results/ --check   # same, exit 1 if a probe failed
+```
+
+| knob | what it sets |
+| --- | --- |
+| `CRAWLEE_REPO`, `CRAWLEE_REF` | what to measure; installed as a source tarball, so the image needs no `git` |
+| `BENCH_CPUS` | cores the CPU-limited scenarios ask for, default 1. They derive cpuset and quota from it, so run it twice: once below the host's core count and once equal to it, where a cpuset stops restricting anything |
+| `IMG` | base image for the container scenarios, default the uv one. Any glibc distro works — uv and the interpreter are mounted in |
+| `BENCH_PYTHON` | interpreter to measure on, default 3.13, so the image cannot change it |
+
+Scenarios whose prerequisites are missing (docker, passwordless sudo, a systemd user manager, kind, enough
+cores) are recorded as `skipped` with the reason; only a crashed probe reddens a run. The `k8s-*` scenarios
+bring up a kind cluster named `cgroups-bench` and leave it running — `kind delete cluster --name cgroups-bench`.
+
+## Files
+
+| file | what it does |
+| --- | --- |
+| [run.sh](run.sh) | Runs each scenario and always writes a result, so a crash is a recorded row rather than a missing one. |
+| [scenario-helpers.sh](scenario-helpers.sh) | The vocabulary scenarios are written in: prerequisite probes, cpuset derivation, the container and pod launchers. |
+| [scenarios/](scenarios/) | One file per environment shape; every file here is a scenario. |
+| [probe.py](probe.py) | Runs inside the prepared environment and prints one JSON object with what Crawlee sees there. |
+| [wrap.py](wrap.py) | Folds the probe's output and the scenario's configuration into one result file. |
+| [report.py](report.py) | Turns a results directory into the tables. `--check` is the only gate. |
+| [.github/workflows/bench.yaml](.github/workflows/bench.yaml) | Manual trigger only, one job per CPU budget. |
+
+## Scenarios
+
+A scenario declares what it needs, what it configures, and how to run the probe in the environment it prepares.
+The commands use the declared values rather than repeating them, so the report's `set` column is literally what
+was applied.
+
+```bash
+SCENARIO_DESC="private cgroupns (docker's default), all three axes at once"
+REQUIRES="engine min${BENCH_CPUS}cpu"   # unmet -> skipped, with this reason in the table
+
+QUOTA=$(awk "BEGIN{print $BENCH_CPUS - 0.5}")
+
+SET_MEMORY_BYTES=$((512 * 1024 * 1024))  # an axis left undeclared means the scenario restricts nothing
+SET_CPU_CORES=$QUOTA                     # there, so the sensor reporting nothing is the right answer
+SET_CPUSET_CORES=$BENCH_CPUS
+
+scenario_exec() { container_probe "$1" -m "$SET_MEMORY_BYTES" --cpus "$QUOTA" --cpuset-cpus "$(cpuset_list)"; }
+```
+
+`scenario_exec` must keep its stdout pure probe JSON, so setup noise goes to stderr; an optional
+`scenario_cleanup` runs in a trap. A new scenario is one such file — it is picked up and reported automatically.
+
+| scenario | shape it exercises |
+| --- | --- |
+| `bare` | no limits at all, so every reading should fall back to host values |
+| `systemd-own` | deep chain, limits on the process's own cgroup, cpuset delegated |
+| `systemd-ancestor` | limit on an ancestor while the leaf carries no memory controller files |
+| `docker-private` | private cgroupns (docker's default), all three axes at once |
+| `docker-memory-only` | memory alone, CPU falls back to host |
+| `docker-cpuset-only` | cpuset alone, memory falls back to host |
+| `docker-host-ns` | `--cgroupns=host`, where the container sees the full chain instead of its own root |
+| `docker-cgroup-parent` | limit on a parent cgroup, set outside the container (needs docker's systemd driver) |
+| `k8s-limits` | a pod with container limits, as kubelet writes them |
+| `k8s-no-limits` | a pod with nothing set, so every reading should fall back to the node's values |
+
+## Results
+
+One `results/<scenario>.json` per scenario with the scenario's `configured` values, its `status` (`ok`,
+`probe_failed`, `skipped`) and everything the probe printed:
+
+| section | contents |
+| --- | --- |
+| `sensor` | what `crawlee._utils.cgroup` reports: memory limit and working set, cpu quota, cpuset size, cpu time, and the levels it walked |
+| `derived` | what `crawlee._utils.system` makes of that in `get_memory_info()` and `get_cpu_info()` |
+| `evidence` | verbatim contents of the cgroup control files at every level, never interpreted |
+| `host` | kernel, cores and RAM of the machine |
+| `errors` | readings that raised; the value becomes `null` and the probe carries on |
+
+`report.py` renders two tables. In the first, each limit has a `set` column next to a `read` one: a value under
+`set` with a dash under `read` is a limit the sensor missed, and dashes under both mean the scenario restricts
+nothing there and the sensor agrees. `mem in use` is the memory charged against the limit excluding reclaimable
+page cache — what `docker stats` shows. The second table is the same run seen through `get_memory_info()` and
+`get_cpu_info()`, where a limit either reaches the caller or falls back to host values. When a reading looks
+wrong, `evidence` says who is at fault: the limit is in the control files, or it never got there.

probe.py

@@ -0,0 +1,169 @@
+from __future__ import annotations
+
+import json
+import os
+import sys
+from importlib.metadata import version
+from pathlib import Path
+from typing import Any
+
+SCHEMA = 1
+
+_EVIDENCE_FILES = (
+    'memory.max',
+    'memory.current',
+    'cpu.max',
+    'cpu.weight',
+    'cpuset.cpus.effective',
+    'cgroup.controllers',
+)
+
+_MAX_LEVELS = 20
+"""How far up the cgroup chain the evidence dump walks. Deeper than any real hierarchy, so it only bounds a loop."""
+
+_FILE_CAP_CHARS = 200
+
+errors: list[str] = []
+
+
+def _guard(where: str, read: Any, default: Any = None) -> Any:
+    """Run a reading that touches the library under test, recording rather than raising when it breaks.
+
+    The probe imports private APIs from a moving branch. A rename there should degrade one column to null, not
+    take the whole measurement down with it.
+    """
+    try:
+        return read()
+    except Exception as exc:
+        errors.append(f'{where}: {type(exc).__name__}: {exc}')
+        return default
+
+
+def _host() -> dict[str, Any]:
+    """Host facts, read directly - a cpuset only means something next to the machine's core count."""
+    mem_total = None
+    try:
+        for line in Path('/proc/meminfo').read_text().splitlines():
+            if line.startswith('MemTotal:'):
+                mem_total = int(line.split()[1]) * 1024
+                break
+    except OSError:
+        pass
+    return {'kernel': os.uname().release, 'ncpu': os.cpu_count(), 'mem_total_bytes': mem_total}
+
+
+def _read_control_file(path: Path) -> str | None:
+    """Read one cgroup control file, or None when it does not exist at this level."""
+    try:
+        return path.read_text()[:_FILE_CAP_CHARS].strip()
+    except OSError:
+        return None
+
+
+def _evidence() -> dict[str, Any]:
+    """Dump raw cgroup file contents along the chain, verbatim - receipts, not a verdict, never interpreted."""
+    evidence: dict[str, Any] = {'proc_cgroup': None, 'levels': []}
+    try:
+        evidence['proc_cgroup'] = Path('/proc/self/cgroup').read_text().strip()
+    except OSError:
+        return evidence
+
+    own_path = next((line[3:] for line in evidence['proc_cgroup'].splitlines() if line.startswith('0::')), None)
+    if own_path is None:
+        return evidence
+
+    mount = Path('/sys/fs/cgroup')
+    directory = mount / own_path.lstrip('/') if '..' not in own_path else mount
+    for _ in range(_MAX_LEVELS):
+        files = {name: content for name in _EVIDENCE_FILES if (content := _read_control_file(directory / name))}
+        evidence['levels'].append({'path': str(directory), 'files': files})
+        if directory in (mount, directory.parent):
+            break
+        directory = directory.parent
+    return evidence
+
+
+def _sensor() -> dict[str, Any]:
+    """Collect everything `crawlee._utils.cgroup` reports for this process."""
+    # Imported here rather than at module level: a rename on the branch under test must be recorded as an entry
+    # in `errors`, not kill the probe before it can report anything at all.
+    from crawlee._utils import cgroup
+
+    readings: dict[str, Any] = {
+        'is_v2': None,
+        'memory_levels': None,
+        'memory_limit_bytes': None,
+        'working_set_bytes': None,
+        'cpu_quota_cores': _guard('get_cpu_quota', cgroup.get_cpu_quota),
+        'cpu_set_cores': _guard('get_cpu_set_size', cgroup.get_cpu_set_size),
+        'cpu_usage_seconds': _guard('get_cpu_usage', cgroup.get_cpu_usage),
+    }
+
+    memory_limit = _guard('get_memory_limit', cgroup.get_memory_limit)
+    if memory_limit is not None:
+        readings['memory_limit_bytes'] = memory_limit.limit
+        readings['working_set_bytes'] = memory_limit.working_set
+
+    def discovery() -> dict[str, Any]:
+        memory = cgroup._get_controllers().memory
+        return {
+            'is_v2': memory.is_v2 if memory is not None else None,
+            'memory_levels': [str(directory) for directory in memory.dirs] if memory is not None else None,
+        }
+
+    readings.update(_guard('_get_controllers', discovery, default={}))
+    return readings
+
+
+def _derived() -> dict[str, Any]:
+    """Collect what `crawlee._utils.system` makes of those readings - the figures a crawler acts on.
+
+    `get_memory_info()` substitutes the cgroup limit and its working set for the host totals whenever a limit
+    applies, and `get_cpu_info()` measures utilization against the cores the cgroup allows instead of the whole
+    machine. Where no limit is found both fall back to host-wide values, so these fields also show what a missed
+    limit would cost.
+    """
+    from crawlee._utils.system import get_cpu_info, get_memory_info
+
+    memory = _guard('get_memory_info', get_memory_info)
+    cpu = _guard('get_cpu_info', get_cpu_info)
+
+    def allowed_cores() -> float | None:
+        from crawlee._utils.system import _get_allowed_cpu_cores
+
+        return _get_allowed_cpu_cores()
+
+    return {
+        'total_size_bytes': memory.total_size.bytes if memory is not None else None,
+        'system_wide_used_bytes': memory.system_wide_used_size.bytes if memory is not None else None,
+        'current_size_bytes': memory.current_size.bytes if memory is not None else None,
+        'allowed_cpu_cores': _guard('_get_allowed_cpu_cores', allowed_cores),
+        'cpu_used_ratio': round(cpu.used_ratio, 3) if cpu is not None else None,
+    }
+
+
+def main() -> None:
+    """Print one JSON object describing what this environment looks like to Crawlee."""
+    source = sys.argv[sys.argv.index('--source') + 1] if '--source' in sys.argv else None
+
+    report = {
+        'schema': SCHEMA,
+        'target': {
+            'name': 'crawlee-python',
+            'version': _guard('version', lambda: version('crawlee')),
+            'source': source,
+            'python': sys.version.split()[0],
+        },
+        'host': _host(),
+        'evidence': _guard('evidence', _evidence, default={}),
+        'sensor': _sensor(),
+        'derived': _guard('derived', _derived, default={}),
+        'errors': errors,
+    }
+
+    json.dump(report, sys.stdout, indent=1)
+    sys.stdout.write('\n')
+
+
+if __name__ == '__main__':
+    main()