Catch translator and host changes that break games nobody on the PR owns #272

This issue is fundamentally about how this project manages the deluge of changes that are landing in such a way so as fixes for one game don't break others. Please read the below @seedds! Cheers.

What happened

c3a0880 (from #178, "Guard Metal image atomics") was validated thoroughly on Pokémon Legends: Z-A: all 28,603 shaders, GPU probes and a 240 s run. Even so, it broke at least two games outside that set:

• SHIN MEGAMI TENSEI V (SHIN MEGAMI TENSEI V (010063B012DC6000, 1.0.0): game profile, version 8 Unreal paks, UE 4.23's FMalloc layout and engine region, 60 fps and an iOS Screen setting #173): the host aborts with FATAL [RED] image atomics: unsupported or overflowing IMG extent at the first atomic on a buffer image. Image atomics on buffer images: declared as buffers, and an unbound one gets a buffer placeholder and an atomic scratch of its own (general, from SHIN MEGAMI TENSEI V) #263 fixes the reflection side.
• Yoshi's Crafted World (Yoshi's Crafted World: profile, sparse texture layout, GPU copies into texture storage #218): the same abort about 6 s in, on compute shader a94ca5c952ce0d72. In addition, the translator now rejects 3 of its shaders (572c1b843ca32afb, 8dbd4ee9b3da30d4, f0b33a6b4f3cc64f) because they use the value an image atomic returns, and it changes a94ca5c952ce0d72 to read 0. Builds made with shaders from before c3a0880 run fine.

Neither failure is a mistake in the testing of #178. The project has one shader chain (DMSH_CHAIN) for every game, so each translator or host change affects all of the roughly 70 profiles. But it can only be checked on the games its author owns, and no contributor owns them all. CI can't help either, since it can't hold game files or run Metal 4 (#112).

What the existing tests miss

• dm/tools/tests/test_image_atomic_bounds.py already builds synthetic Maxwell shaders from InstTable.cs and runs them through the translator, spirv-val and spirv-cross, with no game inputs. In CI, though, the translator cases are skipped: DMSH_TEST_XLATE is unset, and spirv-cross and spirv-val aren't installed.
• The same test asserts that an atomic whose result is used must be rejected (require IGN with a discarded result). That is the pattern Yoshi uses, so the regression was locked in as intended behaviour. Nothing told the author that shipping games depend on it.
• When shaders fail to translate, dmsh_build.py exits 0 and the NSP script packs the app anyway with "N of M shaders ready; K failed". The first sign is a crash at run time, often long after the change, once a rebuild picks up the new chain.

Proposal

Every item works for any contributor and in CI. None needs game files, keys, Metal or a particular contributor owning a particular game.

1. Run the translator fixture tests in CI. Point DMSH_TEST_XLATE at the committed dm/gfx/tools/dmsh_xlate_bin, and install the .NET runtime and spirv-cross/spirv-val in tool-tests. The Metal compile step (DMSH_TEST_METAL) stays local. This is small: about 15 lines of workflow.
2. A synthetic fixture for each pattern a game turns up. When a game hits a translator gap, the fix comes with a hand-encoded shader reproducing the instruction pattern, as test_image_atomic_bounds.py already does. For this case, that means a SUATOM/SURED with a used result, plus a buffer-image binding, replacing the assertion that requires rejection.
3. Report shader regressions against the contributor's own previous build. dmsh_build.py compares the new _b_build.json with the previous one in the same build folder. It prints the shaders that used to translate and now fail, with reasons, and the NSP script stops unless told to continue. Nothing is committed: every contributor gets the check for the games they own.
4. A per-profile instruction census (for discussion). shader_split.py already decodes every shader to make the profile's tables. It could also write per-shader counts of the instruction forms the translator treats specially (for example SUATOM with a used result, by clamp mode). Like shader_facts.tsv, it would hold only hashes and counts, no game code. The translator would declare the forms it rejects, and a CI test would list every profile whose census contains one. A PR that rejects a form used by a shipping game would then name those games, even when its author doesn't own them. Profiles without a census report "unknown" rather than pass, so it can be filled in gradually by people who own each game.

Ruled out: golden MSL outputs and committed per-game translation baselines. Both need the game to verify, and golden MSL is code derived from the game.

A smaller question: should the image-atomic admission skip the one dispatch with a [RED] line instead of aborting the app? I understand the abort is deliberate, since an unchecked atomic must never reach a dummy buffer. Skipping writes nothing, so it keeps that guarantee while a game keeps running.

Offer

I'm happy to open PRs for items 1 to 3 now, as separate small PRs (2 together with the Yoshi translator fix). Item 4 changes what a profile holds, so I'd like your view first. Does the census sound acceptable, and would you prefer it in shader_facts.tsv or a separate table?

Related: #178, #263, #173, #218, #112.