Cover object placement in an address space narrower than a granule

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Problem

A 16-bit p-code architecture has a 64 KiB address space, narrower than the
default 1 MiB rebase granularity, so Loader._find_safe_rebase_addr aligned the
extern object past the end of memory and a z80 blob did not load at all. Against
45c6509~1, the commit before #735:

$ cle.Loader(BytesIO(b'\0' * 0x1500), main_opts={'backend': 'blob', 'base_addr': 0,
                                                 'entry_point': 0, 'arch': ArchPcode('z80:LE:16:default')})
  File "cle/loader.py", line 1097, in _find_safe_rebase_addr
    raise CLEOperationError("Ran out of room in address space")
cle.errors.CLEOperationError: Ran out of room in address space

#735 has since fixed that, so this branch no longer changes cle/loader.py and
only the coverage is left. What is missing is a guard: nothing else in the suite
loads an architecture narrower than 32 bits.

Root cause

#735 replaced the single granule-aligned placement with an alignment ladder that
falls back when a gap is too tight:

alignments = [self._rebase_granularity]
alignments += [a for a in (0x1000, 1) if a < self._rebase_granularity]

A 16-bit space is the extreme case of that: it does not contain even one default
granule, so it holds no object at all unless the ladder reaches the bottom rung.
This branch's earlier one-byte special case for arch.bits < 32 was dropped
rather than rebased, because the ladder subsumes it.

Fix

Two tests in tests/test_rebase.py. test_address_space_narrower_than_the_granularity
loads a 0x1500-byte z80 blob and asserts the extern object lands above the image
and below 2**16. test_narrow_address_space_holds_more_objects_than_granules
then dynamic_loads 24 further objects into the same 64 KiB, asserting they are
disjoint, in bounds, and each findable by find_object_containing. At the merge
base both pass:

main_object  = [0x0:0x14ff]
extern_object= [0x2000:0x21ff]   (address space ends at 0xffff)

Testing

Delete the one-byte rung from the ladder and master's tests/test_rebase.py
stays green at 3 passed, while this branch's file reports 1 failed, 4 passed --
test_narrow_address_space_holds_more_objects_than_granules, with
CLEOperationError: Ran out of room in address space at cle/loader.py:1095.
That one test is the whole guard; the other four, including everything #735
added, cannot see the removal. Against 45c6509~1 all five fail.

angr/angr#6793 works around this with rebase_granularity=0x100 and can drop it.

Validation: #717 (comment)

sync: angr/angr#6793

session: sharpen

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head 5b0405bbbf5cff72fdf1cc338a5c0d92f8e56e2e against baseline 46a37333f4f59b0facf8774ee743ebc4cc074e9b, plain master now that #735 has landed. This replaces the record for 8a792e9; the rebase changed what there is to validate.

Rebase: the conflict was in Loader._find_safe_rebase_addr, which #735 rewrote whole. Resolved by keeping master's implementation and dropping this branch's alignment = 1 if arch.bits < 32 hack, because master's alignment ladder already ends at one byte and so already places every object this branch was placing. What survives is the coverage.

• Reproducer: cle.Loader(io.BytesIO(b"\0" * 0x1500), main_opts={"backend": "blob", "base_addr": 0, "entry_point": 0, "arch": archinfo.ArchPcode("z80:LE:16:default")}) — CLEOperationError: Ran out of room in address space on b58ea02, this branch's old base. On 45c6509 it loads, with the extern object at 0x2000.
• Regression against the old base: both new tests fail on b58ea02 with CLEOperationError: Ran out of room in address space.
• Regression against the current base: there is no production change left to withhold, so the tests were checked against the merged code they guard instead. Removing the one-byte step from the ladder in cle/loader.py (alignments += [a for a in (0x1000,) if a < self._rebase_granularity]) fails test_narrow_address_space_holds_more_objects_than_granules with the same error, while every test Place rebased objects in the free space, not one granule at a time #735 added still passes — that step has no other guard. Cutting the ladder back to the granularity alone fails all five tests in tests/test_rebase.py.
• Focused suites: python -m pytest tests/test_rebase.py tests/test_blob.py tests/test_extern.py tests/test_elfcore.py tests/test_overlap.py tests/test_preload.py — 12 passed
• Full suite: python -m pytest tests — 227 passed, 9 skipped
• Lint/type: pylint and pyright, scored per changed file against merge base 45c6509 the way the hosted Lint and Typecheck jobs score them — tests/test_rebase.py 10.00 -> 10.00 and badness 0.0 -> 0.0
• Hooks: pre-commit run --all-files — every hook passes and leaves the tree unchanged
• Test inputs: check-test-inputs.py --repository cle passes. The blob is raw zero bytes with no header, not an assembled container.
• Downstream: Support p-code architectures whose address width is not 32 or 64 bits angr#6793 loads a 16-bit shellcode with rebase_granularity=0x100 because the default did not fit. On master the default works — the extern object lands at 0x1000 — so that argument can be dropped there.
• Environment: Python 3.12.13, pytest 9.1.1, pypcode 4.0.1

Caveats: the tests need pypcode, which is in cle's testing dependency group, and skip without it. The z80 blob only exercises a 16-bit address space; 24-bit p-code architectures share the code path but are not covered here.

---

Corpus measurement of the open queue, 2026-08-15

The open pull-request queue was scored against 733 objects drawn from a sweep's own failing units (35 error classes, 49 architectures, 16 containers), with each repository's current master as the baseline rather than the revisions the sweep pinned. Each object is loaded with auto_load_libs=False, use_sim_procedures=False and then run through CFGFast(normalize=True, data_references=False, resolve_indirect_jumps=True, force_complete_scan=False) with a 120-second timeout.

Four of the narrow-p-code-architecture changes form a chain. Applied alone, each one clears part of its own class and leaves the rest of it standing on the next change's defect:

Applied alone	Its own failure class	Objects that complete CFGFast	Where the remainder lands
archinfo#363	ValueError: negative shift count constructing ArchPcode — 1,673 units	18 of 30	RecursionError from state creation — angr#6807's defect
angr#6807	RecursionError from state creation — 1,125 units	20 of 30	KeyError: 24 at lift — angr#6793's defect
angr#6793	KeyError: 24 at lift — 1,423 units	20 of 30	struct.error in pack_word — cle#721's defect
cle#721	struct.error in pack_word — 772 units	1 of 24	KeyError: 24 at lift — back to angr#6793

Applied together — angr#6793 + cle#721 + cle#717 + archinfo#363 — the KeyError: 24 class goes from 20 of 30 to 30 of 30. Under the whole open queue all four classes are complete: 30 of 30, 30 of 30, 30 of 30 and 24 of 24, with no object lost in any class. Merging any one of them on its own therefore moves a fraction of the corpus units its own validation record describes, and the remainder only looks like a new failure.

One qualification about cle#717, which is included in the combination above: since its rebase onto cle#735 it carries no production change, only regression coverage, so nothing in the movement here is attributable to it. It guards the placement rule the other three depend on rather than supplying it.

Recorded here because this PR is part of the combination that was measured, not because the measurement credits it with anything: as the description says, the production change it was opened for landed as cle#735, and what is left is the guard for that ladder's one-byte step. Nothing else in the queue covers it.

Re-keyed 2026-08-28. The figures above were measured at 27f641b0045cde23890bc27679d461de753c7e19 on baseline 45c6509c753d07f740099035cd41f7f473dc6f31, which is the head the opening line named until now; the branch is at 5b0405bbbf5cff72fdf1cc338a5c0d92f8e56e2e on 46a37333f4f59b0facf8774ee743ebc4cc074e9b. git range-diff 45c6509c753d07f740099035cd41f7f473dc6f31..27f641b0045cde23890bc27679d461de753c7e19 46a37333f4f59b0facf8774ee743ebc4cc074e9b..5b0405bbbf5cff72fdf1cc338a5c0d92f8e56e2e reports every commit unchanged and git diff 27f641b0045cde23890bc27679d461de753c7e19 5b0405bbbf5cff72fdf1cc338a5c0d92f8e56e2e differs only by master's own advance (12 files changed, 353 insertions(+), 44 deletions(-)). The second commit at this head, 5b0405bb, is empty: its tree is identical to its parent's, so the tree under test is the one measured. Master touched none of the files this change touches between the two baselines. The opening line's baseline was 45c6509c, master with #735 in it; #735 has since landed, so the new baseline 46a37333 is plain master.

[angr-bot]

Corpus decompilation diffs can be found at angr/dec-snapshots@master...angr/cle_717

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

This branch adds no production change, so the thing to show is what the suite catches. Both runs below use the same loader — cle/loader.py with the one-byte step deleted from the alignment ladder cle#735 added, alignments += [a for a in (0x1000,) if a < self._rebase_granularity] — and differ only in which tests/test_rebase.py is run against it.

Before — master's tests/test_rebase.py is green against that loader, so nothing in the suite notices:

cle master's tests, ladder stopping at 0x1000

$ cd wt/h717noladder && PYTHONPATH=. pytest ../base/tests/test_rebase.py -q

...                                                                      [100%]
3 passed in 0.70s

After — this branch's tests/test_rebase.py fails against the same loader, and passes against the unmodified one:

this branch's tests, same loader and then the real one

$ cd wt/h717noladder && PYTHONPATH=. pytest tests/test_rebase.py -q

>       raise CLEOperationError("Ran out of room in address space")
E       cle.errors.CLEOperationError: Ran out of room in address space

cle/loader.py:1095: CLEOperationError
=========================== short test summary info ============================
FAILED tests/test_rebase.py::test_narrow_address_space_holds_more_objects_than_granules
1 failed, 4 passed in 0.84s

$ cd wt/h717 && PYTHONPATH=. pytest tests/test_rebase.py -q

.....                                                                    [100%]
5 passed in 0.43s

For reference, the failure the branch was opened for, against 45c6509~1, the commit before cle#735 landed — and the same load at this branch's merge base:

the original z80 failure, and its fix

########## 45c6509~1  (before cle#735)
$ cle.Loader(BytesIO(b'\0' * 0x1500), main_opts={'backend': 'blob', 'base_addr': 0, 'entry_point': 0, 'arch': ArchPcode('z80:LE:16:default')})
    File "cle/loader.py", line 1097, in _find_safe_rebase_addr
      raise CLEOperationError("Ran out of room in address space")
  cle.errors.CLEOperationError: Ran out of room in address space

########## 46a3733  (this branch's merge base, after cle#735)
$ cle.Loader(BytesIO(b'\0' * 0x1500), main_opts={'backend': 'blob', 'base_addr': 0, 'entry_point': 0, 'arch': ArchPcode('z80:LE:16:default')})
  main_object  = [0x0:0x14ff]
  extern_object= [0x2000:0x21ff]   (address space ends at 0xffff)