[matbrgz]

Closes #86.

The gap this closes

ulimit -v caps address space, and the JVM and .NET reserve gigabytes of it before running a line of submitted code. Measured on this image at a 256 MB problem limit, the smallest barrier each runtime will even boot under:

runtime	address space needed to start
java 21	2048 MB
kotlin	4096 MB
C# (.NET 8)	3072 MB

That is why config/autojudge.php gives them no barrier at all. The peak-RSS verdict from #108 gave those languages an MLE, but only after the fact: a Kotlin program allocating 640 MB took 640 MB off the machine and was judged afterwards.

A cgroup memory.max caps resident memory, which those runtimes tolerate fine.

Measured, at a 256 MB cap under bwrap

	trivial program	allocating program
Java (-Xmx256m)	rc 0, peak 30 MB	rc 137, peak 256 MB, oom_kill 1
Java (no -Xmx)	—	rc 137, peak 256 MB, oom_kill 1
Kotlin (java -jar)	rc 0, peak 27 MB	rc 137, peak 256 MB, oom_kill 1
C# (.NET 8)	rc 0, peak 7 MB	rc 137, peak 256 MB, oom_kill 1
C / Python	peak 0 / 3 MB	rc 137, peak 256 MB, oom_kill 1

The languages that need 2–4 GB of address space to boot cost ~30 MB of resident memory.

Controlled experiment through the real judge, judging actual submissions end to end:

config	Python hog	Kotlin hog
cgroup on, peak-RSS off	MLE	MLE
cgroup off, peak-RSS on	MLE	MLE
both off	RE	WA — allocated 640 MB and ran to completion

Bottom-right is the issue. Top-right proves the verdict comes from the cgroup and not the fallback.

How it works

• docker/judge/entrypoint.sh delegates /sys/fs/cgroup/judge to uid 1000 as root, then drops to judge with setpriv. Dockerfile.judge trades USER judge for an ENTRYPOINT.
  • The judge process is moved into the subtree first. cgroup v2 only permits a migration when the caller can write the cgroup.procs of the common ancestor of source and destination; from the container's cgroup root that ancestor is the root, so every per-run migration would fail EPERM. It cannot sit in judge/ directly either — "no internal processes" — hence judge/main.
  • Only the four files the kernel's delegation interface names are chowned. Not memory.max: owning that would let the judge raise the ceiling it is capped under. Verified negatively — the judge user cannot write judge/memory.max, and touch inside the subtree is refused.
• CgroupMemoryLimiter creates one cgroup per run, writes memory.max + memory.swap.max=0, and wraps the command so a shell joins the cgroup and then execs. bwrap and everything under it is inside the cap before the runtime starts — the difference between bounding and measuring.
• MLE on oom_kill > 0, since memory.max does not guarantee a kill (a refused allocation can just return NULL). memory.peak is the backstop, but only for a run that failed: memory.peak counts page cache, so a run that merely read a large input can touch the ceiling without ever being denied a byte. Note >=, not > — memory.peak is clamped by memory.max, so a capped run reports the ceiling exactly and > could never fire.

Degradation is the default

No cgroup v2, an unprivileged container, no memory controller, a developer's laptop — every step becomes a no-op and judging carries on with the peak-RSS verdict, kept unchanged as the fallback. Refusing to start would turn a missing kernel feature into a dead contest. The unprivileged suite is green with 57 skips; nothing refuses to start.

ulimit -v is dropped where a cgroup is live, not stacked under it

Measured rather than assumed: a Python hog at 256 MB is OOM-killed at exactly 256 MB with ulimit -v 393216 and without it alike. The barrier sits at limit+grace while memory.max sits at the limit, so it can never bind first — redundant, not harmful. But it retains the ability to fail a runtime that reserves address space it never touches, which is the exact failure the null rows exist to avoid. The grace table stays for deployments with no cgroup to be had.

Review fix on top of the original branch

The per-run cgroup was released only on the happy path. Both ways out of executeProgram() that are not a return throw — Process::run() raises ProcessTimedOutException when the backstop timeout fires (confirmed against the installed Laravel, not assumed), and assertSandboxStarted() raises when bwrap never started. Those are exactly the runs most likely to still have a process inside the cgroup, and a cgroup with a live process in it is holding the memory the cap exists to bound. create() reuses a stale directory only when the same name recurs, and the name carries the pid, so once a worker is recycled it never gets that chance.

Now released in a finally, with a regression test that goes red when the finally is removed.

Verification

• Full suite in the image, --privileged --fail-on-skipped: 923 tests, OK (on the pre-rebase branch).
• Full suite unprivileged after rebase onto master: 854 tests, 3996 assertions, OK, 16 skipped.
• Tests added: 10 unit (CgroupMemoryLimiterTest), 5 integration against a real cgroup (including delegation containment and cgroup removal), 2 E2E Kotlin (MLE, plus a regression guard that a valid Kotlin solution still passes under the cap), 1 on the grace change, 1 on the release-on-throw fix.
• Pint clean on both added files; the entrypoint passes sh -n and shellcheck (one informational SC2013 on a pid loop).

Known limits, stated plainly

1. The cap is on resident memory, so untouched virtual memory is not caught — and should not be. A C# hog allocating 640 MB of byte[] but touching one byte each completes at peak 57 MB, correctly. Faulting every page gives the OOM kill in the table. The peak-RSS fallback has the identical property; only ulimit -v bounded untouched reservations, and that is exactly what these runtimes cannot tolerate.
2. Compilation is not cgroup-capped, only the run. Out of scope for Judge sandbox: cap resident memory with cgroups #86, but it is the remaining unbounded path — worth its own issue.
3. The JVM is container-aware and bwrap hiding /sys is load-bearing. Outside bwrap the JVM reads memory.max, sizes its heap to ¼ of it, and throws OutOfMemoryError (rc 1, peak 247 MB) below the cap — which would be RE, not MLE. Inside bwrap /sys is absent, so it sizes from host RAM and the kernel wins. If anyone ever adds /sys to sandbox_paths, JVM MLE detection silently degrades. There is a test that submitted code cannot reach /sys/fs/cgroup at all, which guards the security property and would also catch this.
4. CI is the one thing that could not be verified locally: if delegation fails on GitHub's runners, the 4 integration cgroup tests skip and --fail-on-skipped turns the job red. That is the honest signal — an uncapped judge should not pass as green — but it may need a follow-up if the runners' cgroup layout differs from Docker Desktop's LinuxKit VM.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Pvf2R8xtgbWQm1wRvjBqQY

[matbrgz]

Verification after the rebase, run locally

The branch was rebased onto master (now carrying #112 and #113), so I re-ran the checks rather than relying on the pre-rebase numbers in the description.

Full suite in the judge image, --privileged --fail-on-skipped:

OK (960 tests, 4518 assertions)
Time: 01:14.477

Zero skipped. That is the number that matters here — it means the cgroup delegation actually happened and all 5 integration cgroup tests plus the sandbox suite really executed, rather than skipping themselves into a green build.

The security property is not vacuous

Submitted code runs as the same uid 1000 the subtree is delegated to, so the only thing stopping a submission from raising its own memory.max is that /sys is absent inside bwrap. I checked that the test guarding this can actually fail, by adding /sys to sandbox_paths and re-running:

run	result
/sys exposed in sandbox_paths	test_sandboxed_code_cannot_reach_the_cgroup_hierarchy_at_all — 1 failure
unchanged	OK (16 tests, 38 assertions)

So if anyone ever adds /sys to the allowlist, CI says so.

The release-on-throw fix is not vacuous either

Moving release() back out of the finally (keeping the try and a rethrowing catch, so the only thing that changes is where the release happens) turns test_the_cgroup_is_released_even_when_the_run_throws red. And Process::run() throwing on timeout was confirmed against the installed Laravel rather than assumed:

THROWS: Illuminate\Process\Exceptions\ProcessTimedOutException

Still outstanding

Point 4 in the description — whether GitHub's runners can delegate the subtree — is the one thing a local run cannot answer. The judge-image job on this PR is the test. If it comes back red with skips, that is the honest signal and not a reason to loosen --fail-on-skipped.