Skip to content

PE: Read the ARM64 and ARMNT exception directory - #756

Open
zardus wants to merge 1 commit into
masterfrom
feature/fix-cle-pe-arm-unwind-hints
Open

PE: Read the ARM64 and ARMNT exception directory#756
zardus wants to merge 1 commit into
masterfrom
feature/fix-cle-pe-arm-unwind-hints

Conversation

@zardus

@zardus zardus commented Aug 17, 2026

Copy link
Copy Markdown
Member

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

_handle_seh reads the exception directory through pefile, which parses it for
x86-64 and Itanium only, so an ARM64 or ARMNT image gives CFGFast no function
hints even though its ABI requires an unwind entry for every non-leaf function.
The entries are in the file; only the parse was missing.

CLE now reads the eight-byte ARM64 and ARMNT record itself, taking the length
from the packed word or the .xdata header and skipping records that describe a
fragment of a function beginning elsewhere. An ARMNT start address keeps its
Thumb bit, as an ARM function address does everywhere else in CLE, because that
bit selects the decoder; masking it off measurably costs recovery.

tests/test_pe_function_hints.py covers both, and checks that the x86-64 hints
still match pefile's own parse.

Fixes #753. Needs angr/binaries#183 for the fixtures. Validation: #756 (comment)

@zardus

zardus commented Aug 17, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head e4169b696b49d700c8116217fe2720b465a673ac against baseline 45c6509c753d07f740099035cd41f7f473dc6f31.

  • Regression: pytest tests/test_pe_function_hints.py — 4 tests; against baseline cle the three ARM tests fail with an empty hint list and test_x86_64_matches_pefile passes; all 4 pass on head
  • Focused: pytest tests/ in cle — 229 passed, 9 skipped on head; 225 passed, 9 skipped on baseline
  • Lint/type: ruff check, ruff format --check, and run-ci-diff-checks.py --repository cle (the merge-base pylint and pyright comparison the hosted jobs apply) — all changed files improve or hold
  • Workspace gate: run-all-tests.sh --jobs 2 in an ANGR_FEATURE shell with cle, binaries and angr adopted — all selected suites passed: workspace checks, the test-inputs fixture check, every configured pre-commit hook, the per-feature instancing suite, cle 235 passed 9 skipped, angr 2,471 passed 46 skipped 2 xfailed 260 subtests, angr Rust 35 passed. archinfo, pypcode, pyvex, claripy and angr-management are skipped as unadopted and untouched. A first run at the default four workers lost tests/procedures/libc/test_strtol.py to SIGKILL; memory.events recorded exactly one new oom_kill, and the test passes alone in 140s, so the run was repeated at two workers

Corpus A/B, 1,557 PE objects, one process per object per side, with the
exception directory also parsed straight out of each file's bytes as the oracle:

machine objects entries in the file fragments hints on baseline hints on head
AMD64 469 228,581 3,688 228,291 228,291
ARM64 306 123,866 0 0 123,866
ARMNT 176 29,737 84 0 29,653
  • Every difference: 119 ARM64 and 57 ARMNT objects gain function hints; nothing else changes. Loaded memory, relocations, symbols and entry points are byte-identical on all 1,557 objects, x86-64 hints are byte-identical, no load newly fails, no run times out.
  • Load errors: 11 on both sides, unchanged — 10 ArchNotFound on LOONGARCH64 and one IndexError in _meta_iat that Stop indexing PE and ELF header tables past their declared length #732 fixes.
  • The addresses are not merely counted. Over the 176 objects with an exception directory: no entry is out of order, none overlaps its predecessor, and capstone decodes an instruction at every claimed start in the mode the Thumb bit selects. 121,614 of 123,866 ARM64 and 29,443 of 29,737 ARMNT ranges decode as a clean stream ending exactly on the declared length; the rest stop early on an encoding capstone does not know, which is a limit of the check.
  • Every one of the 29,737 ARMNT entries has the Thumb bit set on its start address.

The issue proposed masking that bit off. Measured, keeping it is better on every
axis, because angr reads bit 0 of an address as "decode this as Thumb" and its
mode-switch retry does not run for a job seeded from a function hint. On the two
ARMNT drivers with the most entries, CFGFast with defaults:

2,043 entries, masked kept 1,457 entries, masked kept
hints reaching a CFG node 1,230 2,038 722 1,452
of those, decoded as ARM 349 0 406 0
blocks 67,815 69,765 68,414 71,459
functions 7,396 3,470 6,294 2,656
functions at an even address 800 218 716 217
bytes covered 1,098,638 1,141,938 807,874 860,954
seconds 314 299 92 87

Both images are Thumb only, so an ARM-mode node at a hint address is a
mis-decode; masking produced 349 and 406 of them and about twice as many
functions, most of them fragments of mis-decoded code. CLE therefore keeps the
bit, which is also how it already names an ARM function from an ELF symbol
table.

CFG effect of the hints themselves. CFGFast with defaults, one process per
object per side, over the 109 objects of a 247-object ARM population that gain
hints and change nothing else:

Baseline Head
objects 109 109
objects whose block set changed 22
blocks 253,960 278,731
functions 21,312 24,411
bytes covered 6,693,968 7,360,316
errors 0 0
timeouts 0 0

That gain is ARM64, and it concentrates: the four largest are ARM64 images whose
sha256 begins 315c2b6e9d9971dc, 6e0719160f4bc825, 35189997a98437d1 and
f7293a5cf2e69dde, with 1,147, 834, 327 and 252 exception directory entries
and 230,128, 147,668, 100,260 and 85,068 more bytes of recovered code each.

On ARMNT the hints are close to neutral for CFGFast with
defaults: on the two drivers with 2,043 and 1,457 entries it recovers 69,760
and 71,457 blocks without them and 69,765 and 71,459 with them. The same two
images under the masked spelling recover 67,815 and 68,414, which is why the
Thumb bit is kept — the wrong spelling is worse than no hints at all.

Nine of the 22 lose at most four blocks or six functions, which is a hint at a
real function entry splitting a block that spanned two functions. The other 138
objects of the 247 are ones #755 moves or that gain nothing; they
are excluded above so that this table describes only the hints.

CFGFast is deterministic on this population: two independent runs of the same
revision over all 247 objects agreed on every block address, function address,
block count and byte count, so every difference above is the change.

Caveats: the corpus is a private dataset, so objects are named by machine type and count rather than by path; angr/binaries#183 reproduces both architectures publicly. angr.Project on an ARM Windows image raises KeyError: 'Win32' until angr/angr#6794 lands, so the CFG measurement registers a Windows syscall calling convention for those architectures to reach CFGFast; CFGFast never reads it. On x86-64, entries whose UNWIND_INFO sets UNW_FLAG_CHAININFO are fragments too — 3,688 of 228,581 — and are still emitted as hints; changing that reaches every Windows x64 target and is left for its own change.

CI on this PR: every job that resolves the referenced sibling pull request is
green — ci / Build, ci / Lint, ci / Typecheck, ci / Decompiler Snapshot Testing and all ten ci / Test shards. Test macos-15 fails and
Test windows-2022 is cancelled with it, because cle's own matrix job checks
out angr/binaries with no ref: and so gets master, which does not yet carry
tests/aarch64/windows/pe_reloc_arm64.exe or
tests/armel/windows/pe_reloc_armnt.exe. The failure is
CLEFileNotFoundError naming those two paths; the Windows job reports
cancelled rather than failure because the matrix does not set
fail-fast: false. Both clear when angr/binaries#183 merges, or when #738
teaches that matrix the sibling resolution the ci / * jobs already do. The
local workspace gate cannot see this: it runs against the adopted binaries
worktree, which has the fixtures.

@angr-bot

Copy link
Copy Markdown
Member

Corpus decompilation diffs can be found at angr/dec-snapshots@master...angr/cle_756

_handle_seh took the exception directory from pefile, which parses it for x86-64
and Itanium only, so an ARM64 or ARMNT image produced no function hints at all
even though its ABI requires an unwind entry for every non-leaf function. Read
the eight-byte record for those two machines directly: the function's start RVA,
then either an .xdata RVA or unwind data packed into the second word, which is
where the function length comes from. Records that describe a fragment of a
function beginning elsewhere are not function starts and are skipped, and an
ARMNT start address keeps its Thumb bit, which is what selects the decoder.
@zardus
zardus force-pushed the feature/fix-cle-pe-arm-unwind-hints branch from e4169b6 to b205ac2 Compare August 18, 2026 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PE function hints from the exception directory are x86-64 only, so an ARM64 or ARMNT image gives CFGFast none

2 participants