Skip to content

feat: NetBSD amd64/arm64, platform matrix CI, purego coexistence job - #88

Open
unxed wants to merge 3 commits into
go-webgpu:mainfrom
unxed:feature/platform-matrix
Open

unxed wants to merge 3 commits into
go-webgpu:mainfrom
unxed:feature/platform-matrix

Conversation

@unxed

@unxed unxed commented Sep 25, 2026

Copy link
Copy Markdown
Contributor

Step 5 of the split agreed in #76. Depends on #83. This branch contains #83's commit, so review only the last commit. Once #83 is merged I'll rebase and the diff will drop that commit.

NetBSD amd64/arm64, full tier. These reuse the System V and AAPCS64 backends unchanged. The NetBSD-specific parts are dlopen from libc.so and errno via __errno (internal/dl/dl_netbsd.go, internal/syscall/errno_netbsd.go), and netbsd is added to the Unix build constraints. Like FreeBSD, NetBSD needs -gcflags=github.com/go-webgpu/goffi/internal/fakecgo=-std.

One real fix is included. internal/fakecgo/netbsd.go defined environ, __progname and __ps_strings with //go:linkname only. That gives local symbols, but NetBSD's libc.so expects rtld to resolve them from the main program, so the process would die before main. They are now also exported with //go:cgo_export_dynamic, as freebsd.go already does.

scripts/check-platforms.sh replaces the cross-compile job. It covers every target purego supports under CGO_ENABLED=0. For each target it:

  • builds ./... and links a consumer binary, and checks the tier the target claims (full, nocb, load or pending);
  • on targets that use goffi's own assembly callback trampolines, requires ffi.callbackTrampoline to be a text symbol. This is fix: build callback_amd64.s on FreeBSD #83's check-callback-trampolines.sh merged into the new script, now also run on netbsd;
  • expects pending targets (linux 386/arm/loong64/ppc64le/riscv64/s390x) to keep failing. If one starts building while the table still says pending, the script fails.

It also builds the examples.

purego-coexistence job. It builds a CGO_ENABLED=0 -tags nofakecgo program that imports both purego and goffi, calls getpid through each, and compares the results with os.Getpid().

Docs: docs/PLATFORMS.md covers the tiers, the build flags and the purego parity diff. It also records that windows/386 is loader-only (CallFunction returns ErrUnsupportedArchitecture); that target already built, but the docs never said so. Also updated: the README platform table, a ROADMAP section on the remaining Linux architectures, and CHANGELOG.

NetBSD support is checked by cross-compiling and linking only. It has not run on a NetBSD host.

The Android arm64 jobs fail in "Set up Android SDK" before any goffi code runs. That failure is unrelated to this PR and is fixed by #85.

🤖 Generated with Claude Code

Both Android arm64 jobs now fail in "Set up Android SDK", before any goffi
code runs:

    Warning: Failed to find package 'tools'
    Error: The process '/usr/local/lib/android/sdk/cmdline-tools/16.0/bin/sdkmanager' failed with exit code 1

android-actions/setup-android@v3 defaults its `packages` input to
`tools platform-tools` and runs `sdkmanager <pkg>` for each entry. Google's
SDK repository index (repository2-3.xml) no longer contains a `tools`
package, while `platform-tools` and `cmdline-tools` are still listed, so
the `tools` call exits 1. The same breakage is tracked upstream in
android-actions/setup-android#537. The last green run of these jobs on main
was for c8f74c6 on 2026-09-10; the workflow has not changed since, and the
failure is identical on a docs-only PR.

Nothing in this workflow or in scripts/check-android-arm64.sh uses the
`tools` package. Pass `packages: platform-tools`, which keeps everything
the step installed before except the package that no longer exists. The
action still accepts licenses, exports ANDROID_HOME and puts sdkmanager on
PATH, which the following NDK step relies on.
@unxed
unxed requested a review from kolkov as a code owner September 25, 2026 14:16
@codecov

codecov Bot commented Sep 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

unxed and others added 2 commits September 25, 2026 16:31
ffi/callback.go is built for (linux || darwin || freebsd) && amd64, but
ffi/callback_amd64.s, which holds the 2000-entry trampoline table and
callbackDispatcher, was built only for (linux || darwin) && amd64.

The Go side takes the table's address through a placeholder:

    //go:linkname _callbackTrampoline github.com/go-webgpu/goffi/ffi.callbackTrampoline
    var _callbackTrampoline byte

With the assembly excluded, that one-byte variable becomes the definition
of ffi.callbackTrampoline. Nothing fails to compile or link. On a linked
freebsd/amd64 consumer, `go tool nm` reports

    D github.com/go-webgpu/goffi/ffi.callbackTrampoline

and no callbackDispatcher or callbackWrap at all, where linux/amd64 and
freebsd/arm64 report T for all three. NewCallback therefore returns
addresses computed from a data-segment byte, not code.

The dispatcher is plain System V AMD64 (the same ABI FreeBSD uses) and
enters Go through crosscall2, which internal/fakecgo already provides on
FreeBSD, so the fix is adding freebsd to the constraint.

The existing cross-compile job could not catch this: `go build ./...`
links nothing, and the broken build compiles. scripts/check-callback-
trampolines.sh links a small NewCallback consumer for every target whose
callbacks come from goffi's own assembly (linux, darwin, freebsd x amd64,
arm64) and fails unless the symbol is T. Run against the tree before this
change it reports freebsd/amd64 as 'D'; after it, all six targets pass.
The Cross-Compile job now runs it.

Not verified on a FreeBSD host; the evidence is the symbol table of the
cross-linked binaries.

Refs: go-webgpu#76 (review)
NetBSD amd64/arm64 get the full FFI tier, reusing the System V and
AAPCS64 backends unchanged. NetBSD keeps the dlopen family in libc.so and
its errno accessor is __errno (not glibc's __errno_location, not
FreeBSD/macOS __error). netbsd joins the Unix-family build constraints.

NetBSD fakecgo did not export environ/__progname/__ps_strings into the
dynamic symbol table. //go:linkname alone gives a local definition, and
NetBSD's libc.so has undefined references to all three that rtld
resolves against the main object, so the process would have died before
main. Added //go:cgo_export_dynamic, as freebsd.go does.

scripts/check-platforms.sh becomes the source of truth for the platform
table: it builds and links every target purego supports under
CGO_ENABLED=0, asserts the tier each one claims, checks that
ffi.callbackTrampoline links as code on every asm-trampoline target
(absorbing scripts/check-callback-trampolines.sh), and fails if a target
listed as pending starts building. It replaces the cross-compile job.

A purego-coexistence job builds a CGO_ENABLED=0 -tags nofakecgo program
that calls getpid through both purego and goffi and checks the results.

docs/PLATFORMS.md documents the tiers, windows/386 as loader-only (it
already built, it was never stated), and the purego parity diff;
ROADMAP.md gets the plan for the six remaining Linux architectures.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@kolkov kolkov left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. A single script that owns the platform table and fails when a pending target starts building is a good idea, and the purego-coexistence job covers a combination users keep asking about (#22, #72). I reviewed only the last commit, as you suggested.

Verified locally (Go 1.27.1): scripts/check-platforms.sh passes, with all 12 supported targets building and linking and the six pending ones failing as expected. go vet and go test -count=1 ./... pass too.

Blocking

1. NetBSD is declared full tier on cross-compilation alone. #83 is the cautionary tale here: FreeBSD amd64 was "cross-compile verified" for months while every callback pointed into a data page. purego tests FreeBSD and NetBSD in a real VM on every PR (vmactions/freebsd-vm / vmactions/netbsd-vm, see their .github/workflows/test.yml bsd job and .github/scripts/bsd_tests.sh). Could you add the same for goffi, at least go test ./ffi with CGO_ENABLED=0 and the fakecgo -std flag on both BSDs? Without a runtime run, NetBSD should be listed as something like full (untested), not alongside the runtime-tested targets.

2. The environ/__progname/__ps_strings export: the claim is unverified and contradicted by purego. The CHANGELOG and docs/PLATFORMS.md say that without //go:cgo_export_dynamic "the process would have died before main", and PLATFORMS.md goes on to say goffi is "ahead of purego" because of it. purego's current internal/fakecgo/netbsd.go has only the //go:linkname lines, the same as Go's own runtime/cgo/netbsd.go, and purego's NetBSD VM job runs the whole test suite with CGO_ENABLED=0 and passes (for example the 2026-10-15 run on main: NetBSD on Go 1.25.8 and 1.26.1 both green). So either rtld resolves these differently than described, or something else covers it. The export may well be harmless, but please:

  • drop the "ahead of purego … can actually start under rtld" sentence. We shouldn't publish a claim that a sibling project's tested platform is broken without evidence;
  • reword the CHANGELOG entry so it describes the change, not a crash nobody has observed. Or back it with a NetBSD run with and without the export, which item 1 would provide.

3. RTLD_LOCAL is wrong for NetBSD. NetBSD's <dlfcn.h> has #define RTLD_LOCAL 0x200 (FreeBSD and Linux use 0). goffi itself doesn't pass RTLD_LOCAL on NetBSD today, but the constant and the comment ("match the FreeBSD/Linux numbering") are wrong. purego carries the same mistake, so that's probably where it came from.

4. Rebase onto current main before NetBSD goes in. Our internal/fakecgo defined SIG_BLOCK/SIG_UNBLOCK/SIG_SETMASK once with the Linux values 0/1/2. NetBSD (like Darwin and FreeBSD) uses 1/2/3 (sys/sys/signal.h:242-244), so _cgo_sys_thread_start would unblock every signal across pthread_create instead of blocking them. #90 (ported from purego#553, now merged) moved the constants into the per-OS files, including libcgo_netbsd.go, so NetBSD gets the right values once this branch is rebased.

#92 (also merged) went further on the fakecgo side of NetBSD, and part of this PR is now on main:

  • trampolines_{amd64,arm64}.s build on NetBSD, the same tag change as here.
  • Three layouts that cross-compiling can't catch are fixed. stack_t had the Linux field order (purego#433). pthread_mutex_t and pthread_cond_t were 8-byte integers, but on NetBSD they are structs with a magic field that libpthread checks, so going by NetBSD's libpthread source, locking the runtime-init mutex returns EINVAL without locking.
  • The layouts are asserted at compile time, and CI vets internal/fakecgo for netbsd.

This branch doesn't have the layout fixes. That is item 1 again: nothing short of a run on NetBSD would have shown them. I haven't run NetBSD either, which is why main still doesn't list it as supported. After the rebase, the only fakecgo change left here is the netbsd.go export from item 2.

5. Required status check rename. cross-compile / "Cross-Compile" is a required check in branch protection, and this PR removes the job, so even after the rebase it would stay blocked waiting for a check that never reports. That's on our side. Just flagging that we'll swap it for "Platform Matrix" (and decide about "purego coexistence") when we merge, so nobody is surprised by the state. One rebase detail: on main, the cross-compile job that this PR replaces now also runs the FreeBSD test build with the libthr.so.3 DT_NEEDED check and the NetBSD fakecgo vet. Those steps need to move into platform-matrix (or stay in a job of their own). lint-targets (added in #92) would need netbsd rows once NetBSD becomes a supported target.

Non-blocking

  1. Pin purego in purego-coexistence. go get github.com/ebitengine/purego takes whatever was released last, so a purego release can turn our CI red without any change here. Pin a version (and bump it deliberately).
  2. The script checks compile + link + trampoline symbol, not tiers. Both the script header ("the script then proves the tier") and PLATFORMS.md ("a row here that disagrees with reality breaks the build") promise more. nocb and load are only built, and NewCallback failing or CallFunction returning ErrUnsupportedArchitecture is never asserted. Either tone the wording down or add small assertions.
  3. README vs PLATFORMS.md disagree. Windows arm64 is "Tested (Snapdragon X)" in README but "Cross-compile verified" in PLATFORMS.md. macOS arm64 is "Tested (M3 Pro)" vs "Runtime tested (M-series)", although macos-latest is arm64 and it runs in CI. One table should be the source and the other should match it.
  4. internal/arch/amd64/call_unix.s is an intentionally empty file; widening its constraint is harmless, it's just noise in the diff.

The ROADMAP "Architecture Expansion" section is good, and the ground rule there ("no architecture is promoted out of pending on cross-compilation alone") is the right one. Items 1–2 are really just that rule applied to NetBSD.

Once #83 is merged, a rebase will make this diff a lot easier to read.

@kolkov kolkov added status: review In code review priority: medium Normal priority labels Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: ci CI workflows, check scripts, branch protection platform: netbsd NetBSD-specific priority: medium Normal priority status: review In code review type: feature New capability

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants