Skip to content

metadata: add cpu_quota (Linux cgroup CPU bandwidth limit) - #265

Open
manunicholasjacob wants to merge 1 commit into
psf:mainfrom
manunicholasjacob:metadata-cpu-quota
Open

manunicholasjacob wants to merge 1 commit into
psf:mainfrom
manunicholasjacob:metadata-cpu-quota

Conversation

@manunicholasjacob

Copy link
Copy Markdown

A benchmark run inside a container or a systemd unit with a CPU limit (docker run --cpus=2, a Kubernetes CPU limit, systemd CPUQuota=) gets throttled by the kernel once it uses its quota in a period, but nothing in the metadata shows it: cpu_count is still the host count and cpu_affinity is not set, because a bandwidth limit does not change the affinity mask. Two results from the same machine can then differ for a reason compare_to cannot show.

This PR adds a cpu_quota metadata entry on Linux, read from the process's own cgroup:

  • cgroup v2: cpu.max of the process cgroup (from /proc/self/cgroup) and of each parent, keeping the smallest limit, since a parent limit also applies.
  • cgroup v1: cpu.cfs_quota_us / cpu.cfs_period_us, falling back to the mount root because the cgroup path is not visible inside a container.

The value is formatted like the other CPU entries, e.g. 2 CPUs or 1.5 CPUs, and the key is only set when a limit is set, like cpu_affinity. The new entry is documented in doc/api.rst.

Tests: three unit tests with mocked /proc and /sys files (v2 leaf, tighter parent and container root, v1 in a container and -1 for no limit, no limit at all). ruff check (0.15.22 as pinned in pre-commit) passes. The rest of the suite gives the same result with and without this change on my machine.

Checked on Linux (WSL2, 20 logical CPUs, cgroup v2), Python 3.14 and 3.11:

run cpu_count cpu_affinity cpu_quota
no limit 20 (not set) (not set)
systemd-run --user --scope -p CPUQuota=150% 20 (not set) 1.5 CPUs
docker run --cpus=2 20 (not set) 2 CPUs

I used an AI assistant (Claude) to help write this change. I ran the checks above myself.

cpu_count and cpu_affinity do not show a cgroup CPU bandwidth limit
(docker run --cpus, Kubernetes CPU limits, systemd CPUQuota=), so a
benchmark run under such a limit records cpu_count=20 even when the
process only gets 2 CPUs worth of time per period. Record the limit as
cpu_quota, read from the process cgroup (cgroup v2, including parent
cgroups, or cgroup v1). The key is only set when a limit is set.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant