Skip to content

Add an AVR static archive for object placement in a 16-bit address space - #228

Open
zardus wants to merge 1 commit into
masterfrom
feature/avr-granularity
Open

zardus wants to merge 1 commit into
masterfrom
feature/avr-granularity

Conversation

@zardus

@zardus zardus commented Sep 8, 2026 •

Copy link
Copy Markdown
Member

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Problem

cle tracks no static archive built for a target whose address space is narrower
than 32 bits, so its suite has no real compiler output to test member placement
in a very small space against. Reading ar magic over every tracked file, then
the ELF header of the first object member:

$ python shapescan.py repos/binaries
  tests/aarch64/bsd_symdef_archive.a: first member is not ELF
  tests/mips64/sym64_archive.a: EM_MIPS, ELF64
  tests_src/.../TOOLCHAIN_GCC_ARM/libmbed.a: EM_ARM (32-bit), ELF32
scanned 1851 tracked files, 3 are ar archives

That gap hides a real defect: cle.Loader gives up on the AVR gcc runtime with
CLEOperationError: Ran out of room in address space after placing 27 of 28
members, with 0xf90 of 0x10000 bytes in use, and cle's suite cannot see it.

Root cause

Of the three archives already tracked, one is 64-bit MIPS, one is 32-bit ARM,
and one opens with a BSD __.SYMDEF rather than an ELF member. A 16-bit target
is the case where a 4 KiB page stops being small relative to the whole address
space, and nothing tracked reaches it. Synthesised inputs do exist in cle's own
suite, but no archive a real toolchain emitted.

Fix

tests/avr/libgcov_avr31.a is usr/lib/gcc/avr/14.2.0/avr31/libgcov.a from
Debian Ports' gcc-avr 1:14.2.0-2, copied out byte for byte: 31,660 bytes,
28 ELF32 little-endian AVR relocatables running 620 to 11,096 bytes on disk, for
a target whose entire address space is 0x10000. tests/avr/README.md beside it
records the source, both digests, the member breakdown and the licence, since
LICENSE.md otherwise defaults the tree to MIT and this is GCC runtime code.

https://snapshot.debian.org/archive/debian-ports/20241102T074343Z/pool-hurd-i386/main/g/gcc-avr/gcc-avr_14.2.0-2_hurd-i386.deb
  deb    sha256 d1a5db88c0fc04b518ec0c70e50d85644e357a957b33205e9b18e75f34796342
  member sha256 63ffbb28b90916b17ceb446eea8fef3c5523c4f719c5642324fbc72762f27e0f

Take it from Debian Ports and not from Debian main. Main ships the same version
built for amd64, whose copy of this archive is also 31,660 bytes but hashes to
e9fd418d99dda7c0c34da108dc4fc143add544c5ba5d72128c93ec0597a8347b; the two
differ in exactly 230 bytes, every one of them in an ar header -- the buildd's
uid and gid and the index mtime -- and in no member body byte. gcc-avr is a
cross compiler, so the package architecture is hurd-i386 while everything
inside the archive is AVR. Nothing was rebuilt, patched or truncated.

Testing

The same scan run against this branch reports 4 archives among 1853 tracked
files, the new one reading EM_AVR (16-bit), ELF32. The archive is the
reproducer for angr/cle's placement fix: loading it with cle at master raises
CLEOperationError: Ran out of room in address space, and with that change
applied all 28 members are placed into 0x23f0 bytes. Both runs are in the
comment below.

git hash-object of the file is 8e77517696cd7f0936bb66339b2a10c7fb4e32d8,
which matches no blob this repository already tracks, so it is not a second copy
of something filed under another name.

Validation: #228 (comment)

session: sharpen

tests/avr/libgcov_avr31.a is usr/lib/gcc/avr/14.2.0/avr31/libgcov.a from
Debian Ports' gcc-avr 1:14.2.0-2, hurd-i386 build:

  https://snapshot.debian.org/archive/debian-ports/20241102T074343Z/pool-hurd-i386/main/g/gcc-avr/gcc-avr_14.2.0-2_hurd-i386.deb

sha256 63ffbb28b90916b17ceb446eea8fef3c5523c4f719c5642324fbc72762f27e0f,
31660 bytes. The .deb it came out of is sha256
d1a5db88c0fc04b518ec0c70e50d85644e357a957b33205e9b18e75f34796342. gcc-avr
is a cross compiler, so the package architecture is hurd-i386 while the
archive contents are AVR.

The archive holds a symbol index, a long-name string table and 28 ELF32
little-endian AVR relocatables (EM_AVR, ET_REL, e_flags 0x9f, so EF_AVR_MACH
31 for the avr31 multilib with EF_AVR_LINKRELAX_PREPARED set), all built by
GCC 14.2.0. On disk the members run from 620 bytes (0x26c) to 11,096 bytes
(0x2b58): 21 of 620 bytes, four of 848, one of 844, one of 852, and the
11,096-byte libgcov driver object.

It is here to give cle a real test of where archive members are placed in a
16-bit address space. AVR code addresses are 16 bits wide, and an archive
with this many members does not fit if each one is given a generously
rounded slot, so member placement is observable here on a real toolchain's
output rather than on a synthesised archive.

Take this file from Debian Ports, not Debian main. Main's amd64 build of the
same version ships a copy that is also 31660 bytes but hashes to
e9fd418d99dda7c0c34da108dc4fc143add544c5ba5d72128c93ec0597a8347b; the two
differ in exactly 230 bytes, all of them in ar headers (buildd uid and gid,
index mtime), and in no member body byte.

libgcov is part of the GCC runtime. Upstream its sources are
GPL-3.0-or-later WITH GCC-exception-3.1; the Debian package's own
usr/share/doc/gcc-avr/copyright records the coarser GPL-3+. Redistribution
is allowed either way. tests/avr/README.md records this, since LICENSE.md
defaults the tree to MIT.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@zardus

zardus commented Sep 8, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

What the tree holds before and after, and what the fixture is for. Absolute store paths in the traceback are trimmed to cle/loader.py; nothing else is edited.

The gap -- no tracked archive targets an address space narrower than 32 bits:

angr/binaries master fc07821
  tests/aarch64/bsd_symdef_archive.a: first member is not ELF
  tests/mips64/sym64_archive.a: EM_MIPS, ELF64
  tests_src/i2c_master_read-nucleol152re/mbed/TARGET_NUCLEO_L152RE/TOOLCHAIN_GCC_ARM/libmbed.a: EM_ARM (32-bit), ELF32
scanned 1851 tracked files, 3 are ar archives

With this change -- one 16-bit archive, from a real toolchain:

this branch
  tests/aarch64/bsd_symdef_archive.a: first member is not ELF
  tests/avr/libgcov_avr31.a: EM_AVR (16-bit), ELF32
  tests/mips64/sym64_archive.a: EM_MIPS, ELF64
  tests_src/i2c_master_read-nucleol152re/mbed/TARGET_NUCLEO_L152RE/TOOLCHAIN_GCC_ARM/libmbed.a: EM_ARM (32-bit), ELF32
scanned 1853 tracked files, 4 are ar archives

What it reproduces -- angr/cle master cannot load it:

angr/cle master 0e77ade3
$ python -c 'import cle; cle.Loader("binaries/tests/avr/libgcov_avr31.a")'
Unknown reloc 24 on avr8:LE:16:default
Unknown reloc 25 on avr8:LE:16:default
Unknown reloc 18 on avr8:LE:16:default
Unknown reloc 2 on avr8:LE:16:default
Unknown reloc 3 on avr8:LE:16:default
Unknown reloc 4 on avr8:LE:16:default
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "cle/loader.py", line 187, in __init__
    self.initial_load_objects = self._internal_load(
                                ^^^^^^^^^^^^^^^^^^^^
  File "cle/loader.py", line 944, in _internal_load
    self._map_object(obj)
  File "cle/loader.py", line 1036, in _map_object
    base_addr = self._find_safe_rebase_addr(obj_size)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "cle/loader.py", line 1095, in _find_safe_rebase_addr
    raise CLEOperationError("Ran out of room in address space")
cle.errors.CLEOperationError: Ran out of room in address space

With the sibling cle change -- all 28 members are placed into 0x23f0 of 0x10000 bytes:

the cle placement fix applied
$ python -c '
import cle
ld = cle.Loader("binaries/tests/avr/libgcov_avr31.a")
members = [o for o in ld.all_objects if o.parent_object is ld.main_object]
for o in sorted(members, key=lambda o: o.min_addr):
    print(f"{o.binary_basename:24} 0x{o.min_addr:05x}-0x{o.max_addr:05x}  0x{o.max_addr - o.min_addr + 1:x}")
print(f"{len(members)} members placed in a 0x{2 ** ld.main_object.arch.bits:x}-byte address space")
'
Unknown reloc 24 on avr8:LE:16:default
Unknown reloc 25 on avr8:LE:16:default
Unknown reloc 18 on avr8:LE:16:default
Unknown reloc 2 on avr8:LE:16:default
Unknown reloc 3 on avr8:LE:16:default
Unknown reloc 4 on avr8:LE:16:default
_gcov_merge_add.o        0x00000-0x0010f  0x110
_gcov_merge_topn.o       0x00110-0x0021f  0x110
_gcov_merge_ior.o        0x00220-0x0032f  0x110
_gcov_merge_time_profile.o 0x00330-0x0039f  0x70
_gcov_interval_profiler.o 0x003a0-0x0040f  0x70
_gcov_interval_profiler_atomic.o 0x00410-0x0047f  0x70
_gcov_pow2_profiler.o    0x00480-0x004ef  0x70
_gcov_pow2_profiler_atomic.o 0x004f0-0x0055f  0x70
_gcov_topn_values_profiler.o 0x00560-0x005cf  0x70
_gcov_topn_values_profiler_atomic.o 0x005d0-0x0063f  0x70
_gcov_average_profiler.o 0x00640-0x006af  0x70
_gcov_average_profiler_atomic.o 0x006b0-0x0071f  0x70
_gcov_ior_profiler.o     0x00720-0x0078f  0x70
_gcov_ior_profiler_atomic.o 0x00790-0x007ff  0x70
_gcov_indirect_call_profiler_v4.o 0x00800-0x0086f  0x70
_gcov_time_profiler.o    0x00870-0x008df  0x70
_gcov_dump.o             0x008e0-0x009ef  0x110
_gcov_fork.o             0x009f0-0x00a5f  0x70
_gcov_execl.o            0x00a60-0x00acf  0x70
_gcov_execlp.o           0x00ad0-0x00b3f  0x70
_gcov_execle.o           0x00b40-0x00baf  0x70
_gcov_execv.o            0x00bb0-0x00c1f  0x70
_gcov_execvp.o           0x00c20-0x00c8f  0x70
_gcov_execve.o           0x00c90-0x00cff  0x70
_gcov_reset.o            0x00d00-0x00e0f  0x110
_gcov_lock_unlock.o      0x00e10-0x00e7f  0x70
_gcov.o                  0x00e80-0x00f8f  0x110
_gcov_info_to_gcda.o     0x00f90-0x023ef  0x1460
28 members placed in a 0x10000-byte address space

@zardus

zardus commented Sep 8, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head 500506ea75c337e359c8ebae396b52a02169e207 against baseline fc07821c89535534979b02760e7e1bfc35faf690.

  • Shape scan: every tracked file read for ar magic, then the ELF header of its first object member -- 1851 tracked files and 3 archives at the baseline, none of them 16-bit; 1853 files and 4 archives at the head, the new one reading EM_AVR (16-bit), ELF32
  • Regression it unblocks: pytest tests/test_rebase.py in angr/cle -- 1 failed on cle master (CLEOperationError: Ran out of room in address space at cle/loader.py:1095), 4 passed with the cle change applied
  • Workspace gate: ./feature.sh test avr-granularity, 2026-09-08. Every suite that exercises this change passed: cle 262 passed 9 skipped; angr 2,917 passed, 47 skipped, 2 xfailed, 286 subtests passed; angr's Rust suite; the mono pipeline, 139 tests; pysoot; test-inputs; test-packages; and every configured pre-commit hook on both checkouts. The gate's overall return code is 1 for two failures in the workspace repository's own test_no_venv_survives checks -- AGENTS.md and angr-maintain-mono/scripts/rollup.sh still mention .venv-native, and one allowlist entry matches no line. Neither touches cle, both are in a repository this change does not modify, and both reproduce on other candidates.
  • Gate coverage, quoted from the gate itself: SUITES SKIPPED (feature has not adopted, did NOT run): archinfo pypcode pyvex angr-management. A gate that skipped a suite is green over less than it appears to be; this feature adopted angr, cle and binaries, which are the repositories the change and its fixture touch.
  • File digest: tests/avr/libgcov_avr31.a sha256 63ffbb28b90916b17ceb446eea8fef3c5523c4f719c5642324fbc72762f27e0f, 31,660 bytes, blob 8e77517696cd7f0936bb66339b2a10c7fb4e32d8
  • Second file: tests/avr/README.md, recording the source, both digests, the member breakdown and the licence

Provenance, re-derived here rather than quoted from an earlier record:

  • Source: https://snapshot.debian.org/archive/debian-ports/20241102T074343Z/pool-hurd-i386/main/g/gcc-avr/gcc-avr_14.2.0-2_hurd-i386.deb, sha256 d1a5db88c0fc04b518ec0c70e50d85644e357a957b33205e9b18e75f34796342, 54,886,296 bytes. The ar container and its data.tar.xz were unpacked and usr/lib/gcc/avr/14.2.0/avr31/libgcov.a taken out of it; the committed file is that member, byte for byte.
  • Debian Ports, not Debian main. Main ships the same source version built for amd64 (https://deb.debian.org/debian/pool/main/g/gcc-avr/gcc-avr_14.2.0-2_amd64.deb), and its copy of this archive is also 31,660 bytes but hashes to e9fd418d99dda7c0c34da108dc4fc143add544c5ba5d72128c93ec0597a8347b. The two differ in exactly 230 bytes, every one of them inside an ar header -- the buildd's uid and gid, and the index mtime -- and in no member body byte. A maintainer checking the digest against Debian main will get a mismatch and it will not mean the file is wrong.
  • gcc-avr is a cross compiler, so the package architecture is hurd-i386 while everything inside the archive is AVR. Nothing was rebuilt, patched, truncated or reassembled.

Contents, parsed from the ar format directly:

  • A symbol index, a long-name string table, and 28 ELF32 little-endian AVR relocatables (EM_AVR, ET_REL, e_flags 0x9f, so EF_AVR_MACH 31 for the avr31 multilib with EF_AVR_LINKRELAX_PREPARED set), all built by GCC 14.2.0.
  • On disk the members run 620 bytes (0x26c) to 11,096 bytes (0x2b58): 21 of 620, one of 844, four of 848, one of 852, and the 11,096-byte libgcov driver object.
  • Mapped, which is the number that matters to placement, cle asks for 0x70 (21 members), 0x110 (6) and 0x1460 (1). These are not the on-disk sizes and the two should not be confused.

Duplication: git hash-object of the file is 8e77517696cd7f0936bb66339b2a10c7fb4e32d8, which appears in no tree this repository tracks. The positive control, hashing the already-tracked tests/avr/isqrt_atmega128.o the same way, does match a tracked blob, so the check is live.

Licence: libgcov is part of the GCC runtime. Upstream the sources it is built from (libgcc/libgcov.h, libgcc/libgcov-driver.c, libgcc/libgcov-interface.c at releases/gcc-14.2.0) are GPL-3.0-or-later WITH GCC-exception-3.1; the Debian package's own usr/share/doc/gcc-avr/copyright records the coarser GPL-3+ and does not mention the exception. Redistribution is allowed under either reading. tests/avr/README.md states both, because LICENSE.md otherwise defaults the tree to MIT and angr authorship.

Caveat: the AVR archives were first noticed in a private corpus, but this fixture is a public Debian package and every provenance figure above is reproducible from it.

session: sharpen

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant