Regions: Stop the address lookups from assuming an order the list may not have

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Regions keeps its members sorted by start address and then bisects their end
addresses, which is a total order only while the regions are disjoint. They are
not always, so a lookup can walk past a region that does cover the address and
report there is none: on binaries/tests/armel/btrfs.ko,
find_section_containing(0x400024) returns None although .text spans
0x400000-0x4b9b58.

.tbss is the common cause in ordinary linked ELFs, and it is not a region of
the image at all: its address is where a thread's own copy of the template
begins, and the linker places the section after it over the top. Keeping it out
of the sorted list removes that overlap and stops lookups answering with a
section that holds none of the bytes it claims. Overlap cannot be ruled out in
general, so the lookups now carry a running maximum of the end addresses, which
is sorted whether or not the regions are; while they are disjoint both reduce to
exactly the bisection they were.

Fixes #742.

Validation: #760 (comment)

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head b759dea9561d01614b16a1068febf7191e494b34 against baseline 45c6509c753d07f740099035cd41f7f473dc6f31.

• Regression: pytest tests/test_regions.py::TestOverlappingRegions — both cases fail on the baseline; passes on head
• Focused: pytest tests/test_regions.py tests/test_overlap.py tests/test_blob.py tests/test_namedregion.py — 7 passed
• Full suite: cle 229 passed, 9 skipped; angr 2479 passed, 46 skipped, 2 xfailed, 260 subtests; angr Rust 35 passed
• Lint/type: run-ci-diff-checks.py --repository cle — pylint and pyright unchanged or better on every changed file
• Workspace gate: cle and angr adopted in one feature instance; archinfo, pypcode, pyvex, claripy and angr-management skipped as unadopted and untouched — pass

Reproducer on a public fixture: cle.Loader("binaries/tests/armel/btrfs.ko").main_object.find_section_containing(0x400024) returns None on the baseline although .text spans 0x400000-0x4b9b58; on head it returns .text.

Both lookups were evaluated against the same already-loaded object, so the comparison contains nothing but the lookup. Over the 821 loadable files in angr/binaries (817 scored, 108,301 probed addresses):

	baseline	head
Objects whose sorted region list overlaps	106	51
find_region_containing answers differing	—	404 over 71 objects
find_region_next_to answers differing	—	404 over 71 objects
max_addr differing	—	49 objects

• Deterministic deltas, find_region_containing: 308 addresses where the baseline answered .tbss, which holds none of the bytes there, and now answer the section that does; 86 where the baseline answered None although a region covers the address; 10 inside .tbss and nothing else, which now answer None because no section holds those bytes.
• Deterministic deltas, find_region_next_to: 318 .tbss answers replaced, 71 None answers replaced, and 15 on btrfs.ko, armhf/libc.so.6 and s390x/libstdc++.so.6 where the baseline bisection skipped past the first region whose end is beyond the address.
• max_addr is higher on all 49, 48 of them relocatable objects; nothing in cle, angr or angr-management reads Regions.max_addr.
• Breadth: a further 453 corpus objects sampled 30 per backend, 253 scored across Universal2, PE, XBE, UEFI, TE, COFF, CGC, ELF and Mach-O. Only two ELF objects differ at all: one .tbss overlap, and one where the baseline lost four addresses inside .text.
• Timing: 200,000 random lookups on binaries/tests/x86_64/libc.so.6 take 0.171 s on the baseline and 0.033 s on head; on binaries/tests/armel/btrfs.ko, 0.139 s and 0.131 s.

Caveats: corpus objects are referred to by architecture, container and digest because the dataset is not public. The remaining 51 overlapping objects are relocatable ELFs placing several sections at one address, which #739 addresses separately.

[angr-bot]

Corpus decompilation diffs can be found at angr/dec-snapshots@master...angr/cle_760