pa-risc Context retains memory without bound on adversarial input

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

A long-lived Context for pa-risc:BE:32:default grows until it exhausts the
process, on synthetic input, where every other p-code language tested plateaus.
I can reproduce it deterministically but I cannot explain it, so this is a
report rather than a pull request — see the "what I could not establish"
section, which is the reason.

Reproducer

One Context, 65,536 distinct 5000-byte windows at distinct addresses, window
i being the four bytes i << 16 big-endian repeated:

import os, resource, sys, pypcode

WINDOW, MAXI = 5000, 99999
LANG = "pa-risc:BE:32:default"
ctx = pypcode.Context(LANG)
_s = os.open("/proc/self/statm", os.O_RDONLY)
rss = lambda: int(os.pread(_s, 128, 0).split()[1]) * 4096
_, hard = resource.getrlimit(resource.RLIMIT_AS)
with open("/proc/self/status") as f:
    vsz = [int(l.split()[1]) * 1024 for l in f if l.startswith("VmSize:")][0]
resource.setrlimit(resource.RLIMIT_AS, (vsz + 400 * 1024 * 1024, hard))

base, nbad = rss(), 0
for i in range(65536):
    pat = (i << 16).to_bytes(4, "big")
    try:
        del ctx.translate((pat * (WINDOW // 4 + 1))[:WINDOW],
                          base_address=0x40000000 + i * 0x10000,
                          max_instructions=MAXI, max_bytes=WINDOW,
                          flags=pypcode.TranslateFlags.BB_TERMINATING)
    except MemoryError:
        nbad += 1
    except BaseException:
        pass
print("cum=%d nbad=%d" % (rss() - base, nbad))

On 559aacd this prints cum=419827712 nbad=13207 — the Context consumes the
entire 400 MB budget and then raises MemoryError('std::bad_alloc') on 13,207
of the remaining windows. It is stable across reruns and across two independent
builds; raising the cap to 900 MB just moves the numbers to 943,919,104 bytes
and 7,671. Retention is flat at about 6 MB through window 51,200 and then runs
away from 0xC8010000 onward.

Controls

• SuperH4:BE:32:default 14.3 MB, 68000:BE:32:default 12.8 MB and
  Loongarch:LE:64:lp64d 6.8 MB all plateau over the same 65,536 windows with
  zero bad_alloc. Only pa-risc runs away.
• Real PA-RISC code does not trigger it: 543,231 windows taken from compiled
  hppa binaries retain 7.9 MB with zero bad_alloc. Adversarial input is
  required.
• Constructing a fresh Context every 128 windows gives zero bad_alloc with a
  worst transient of 2.85 MB, so the growth is retained in the Context and not
  in the returned translation.
• A single bounded call is not the problem. Across roughly 1.6M windows the
  worst single translate under a 5000-byte bound was 21 MB (SuperH4:BE,
  77,500 ops), and work is linear in max_bytes.
• Not the same defect as Fix heap corruption on instructions that outgrow the parse tree #289. With Fix heap corruption on instructions that outgrow the parse tree #289 applied the run gives
  cum=419868672 nbad=13208, i.e. unchanged.
• Window size matters: at 400 bytes rather than 5000, the same sweep retains
  154,824,704 bytes and then stops growing, with zero bad_alloc under the same
  400 MB cap.

What I could not establish

The mechanism. Replaying only the tail windows (51,200-52,400) on a fresh
Context costs zero bytes, and three narrower probes — sampled hot-range
patterns, repeated hot patterns, and truncated history — showed no growth at
all and cannot be reconciled with the two deterministic full-sweep runs. So
"a long-lived pa-risc Context grows to whatever ceiling exists, on adversarial
input" is solid; anything about why is not, and I did not want to dress a guess
up as a diagnosis in a patch.

Why it matters

angr keeps one Context per architecture for the life of the process
(angr/engines/pcode/lifter.py), so retention here accumulates across an entire
CFG recovery. In practice angr clamps each block to 400 bytes, which is the
milder of the two regimes above, and real code does not trigger it — so this is
not currently causing the p-code out-of-memory aborts I was chasing, which have
a separate and well-understood cause. It is a sharp edge on untrusted input.

Measured on 559aacd, Linux x86-64, CPython 3.12.13.