ELFCore: Keep the permissions of the mappings it turns into blobs

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

A core dump records what every mapping was allowed to do, and ELFCore reads it
correctly. The mappings it cannot match to a child object are then handed to
Blob, which builds a plain Segment whose base Region answers True to
every permission query, so the heap, the stack and the ---p guard pages all
reach the loader as executable memory.

Blob now takes an optional permissions triple describing the memory it came
from, and ELFCore passes each mapping's flags through. A blob loaded from a
raw image is unchanged, because a raw image records nothing to the contrary.

The regression compares every blob segment of the core-dump fixture against the
mapping it was cut from; on master the heap blob reports rwx where the program
header says p_flags 0x6.

Fixes #743. angr/angr#6864 is the other half, where CFGFast's
executable map learns to read these permissions.

Validation: #758 (comment)

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head 6e74474182bbda0c5b8e179dee1a7d1ead9425fc against baseline 45c6509c753d07f740099035cd41f7f473dc6f31, with angr at 503b1be0663e65de225194eeb8e2cee6fa20d8d3 plus angr/angr#6864.

• Regression: pytest tests/test_elfcore.py::test_blob_children_keep_the_mapping_permissions — on the baseline the heap blob at 0x55555575b220 reports rwx where the core's program header says p_flags 0x6; passes on head
• Focused: pytest tests/test_elfcore.py tests/test_blob.py — 5 passed
• Full suite: cle 226 passed, 9 skipped; angr 2480 passed, 46 skipped, 2 xfailed, 260 subtests; angr Rust 35 passed
• Lint/type: run-ci-diff-checks.py --repository cle — pylint and pyright unchanged or better on every changed file
• Workspace gate: cle and angr adopted in one feature instance; archinfo, pypcode, pyvex, claripy and angr-management skipped as unadopted and untouched — pass

Corpus A/B over 225 ELF core dumps, sampled 25 per (collection, architecture) from a private corpus of 5,250, one object per forked child and one side per child. 169 have all four runs clean; 56 fail identically on every side and are unaffected. The executable map scored against is the core's own PT_LOAD table read with pyelftools, never CLE's view of it.

	baseline	this change only	with angr/angr#6864
CFG blocks	2,114,564	2,105,167	2,105,191
Blocks outside every PF_X PT_LOAD	10,062	7	7
Total seconds	3,062	3,122	2,028
Median seconds	0.92	0.64	0.08
Peak RSS	1,682 MB	1,680 MB	1,686 MB

• Nothing that was decoded inside executable memory is lost: the head covers a superset of the baseline's executable bytes on every one of the 169 objects, 32,082,235 to 32,082,748 bytes. 64 objects have bit-identical block lists.
• The 848 blocks inside executable memory that no longer exist as blocks are re-partitionings of bytes still covered; the 1,530 new blocks are all inside PF_X ranges, and on ARM core 8f47f173386a557b the 46 new ones there are the zero-filled tail of the r-x page at 0x10000, which angr scans on both sides.
• The 7 blocks still outside are all on ARM core ad3397abc774be92, at 0xfafb-0xffff, immediately below its r-x page at 0x10000 and reached from it rather than scanned.
• Three cores that exceeded the 600 s budget on the baseline and with this change alone complete with both halves: 621a8bb8244237e3, 0c46e3e8b635c3db, 3740f861518da260. Nothing times out that did not before.
• Nondeterminism: repeating the head side changes one object, a69bc3dea27b1972, by one block — a 27-byte block against a 15-byte and a 12-byte one at 0x7fb421c34dd1. That is the only block-count difference not attributable to the change.
• The head side was re-measured on 57 of the core dumps after the final revision of the angr half, which infers the same thing from the segments rather than from a new CLE attribute: bit-identical block lists and executable maps on all 57.
• Control: 223 objects loaded with the blob backend across 63 architectures, 154 scored and 69 failing identically on both sides. Every one of the 154 has a bit-identical block list, function count and derived executable map — 924,527 blocks compared.

Caveats: corpus objects are named by architecture, container and digest because the dataset is not public; the regression uses binaries/tests/x86_64/coredump/true-libc.so.6-ld-linux-x86-64.so.2.core. One repeat run hit MDB_MAP_RESIZED from a shared RTDB directory, a harness artefact, and its object is excluded from the 169.

Corpus impact, measured after this pull request was opened

A CFGFast sweep scores every recovered block against the file's own evidence and
counts blocks that land outside the executable memory the file describes. Across
the whole run that category holds 472,106 instances, and ELF core dumps
contribute 137,411 of them, on 2,073 of the 2,118 core dumps scored — 97.9%.
A category that fires on almost every member of a population is a systematic
modelling difference rather than thousands of independent defects, and this is
that difference.

The five heaviest contributors were fetched and their recorded sample addresses
located in each core's own program headers:

digest	arch	instances	PT_LOADs	permissions of the segment each sampled address lands in
99701519c3f9eda9	x86	1,620	21	rw- ×32
4ca4a0e10dad513a	arm	393	6	rw- ×32
77039c204547bfbf	arm	393	6	rw- ×32
6d9bfb41c905f99a	arm	392	6	rw- ×32
88f3579d5c5220de	arm	392	6	rw- ×32

160 of 160 sampled addresses sit in a PT_LOAD the core itself marks
read-write and not executable. None is in an executable segment. That is the
behaviour this change corrects: ELFCore turns each mapping into a Blob, a
Blob reports every segment readable, writable and executable, and CFGFast
therefore decodes the process's data pages as code.

The sweep pins a CLE from before this pull request, so those figures describe the
unfixed behaviour. The corpus also puts the change in proportion against the rest
of the category: Mach-O contributes 329,861 instances and is addressed by #762,
which has merged; with that and this change accounted for, 99.0% of the
category is explained, leaving 4,834 instances on ELF, CGC, COFF and
Universal2 as the genuinely unexplained remainder.

Reproducing needs only a core dump with writable data mappings: every block
CFGFast recovers inside one is a block recovered from data.

The other side of that comparison

The figures above are the unfixed behaviour. Re-running the same three core dumps
with this change applied, together with its companion angr/angr#6864 (sync: on
that pull request), and classifying every recovered block by the permissions of
the PT_LOAD it lands in:

digest	arch	blocks recovered	in an r-x segment	in a non-executable segment
99701519c3f9eda9	x86	9,212	9,212	0
4ca4a0e10dad513a	arm	382	382	0
77039c204547bfbf	arm	376	376	0

Before the change those same objects contributed 1,620, 393 and 393 findings, and
160 of 160 sampled addresses sat in a PT_LOAD the core marks read-write. After
it, none of the recovered blocks lands in a non-executable mapping at all.

Worth noting the x86 case recovers more blocks than the finding count would
suggest — 9,212 of them — and every one is inside r-x. The change does not cost
recovery; it stops recovery happening in the process's data pages.

Measured with the pair applied, because they are companions: this supplies the
permissions and #6864 reads the executable map from segments so they take effect.
Attributing the result to either alone would overstate it.

[angr-bot]

Corpus decompilation diffs can be found at angr/dec-snapshots@master...angr/cle_758