Repair locale-dependent reads in test_research_current_status (no new kappa)

[navinor]

Scope

No kappa, no certificate change. One reproducible environment failure repaired: on a native Windows cp1252 host (the Git-for-Windows default), the documented check

python3 -B -m unittest discover -s tests -p 'test_research_*.py'

fails with three errors when PYTHONUTF8 is not set, all from one root cause: tests/test_research_current_status.py reads docs/research/current-status.md with the locale codec, and that document contains byte 0x9d (in U+201D), which is undefined in cp1252.

• Card: verifier-engineering / portable-replay (scope key 831cc1a3c8f9ff01b197, chooser OS nonce c7669648496a57422bfa478f1d356cad, priority revision 3), scope refined to this one failure. Base 0b1cccf86e2f5132ed286a26ef0a681a495740ea.
• Claim: Handoffs and requests for independent checks #1166 (comment)
• Files changed: tests/test_research_current_status.py (explicit encoding="utf-8" on its two reads, plus two regression tests) and the new package research/test-suite-locale-reads/ (README with the task card, TASK.json, SOURCES.json, VERIFY.json, verify.py). Nothing else; upstream/, frozen pins, certificates and the selected record are untouched.

Evidence

Windows 11, cp1252, Git 2.55.0.windows.5 (core.autocrlf=true), CPython 3.13.15 and 3.12.

Before (base 0b1cccf8, no PYTHONUTF8):

ERROR: test_current_section_matches_selected_record ... setUp
  File "tests/test_research_current_status.py", line 26, in setUp
    self.text = STATUS.read_text()
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 1014
Ran 42 tests in 13.787s — FAILED (errors=3, skipped=2)

The same command with PYTHONUTF8=1, and everything under Python 3.12, passed at base.

After (this branch):

python -B -m unittest discover -s tests -p 'test_research_*.py'   # no PYTHONUTF8, 3.13.15
Ran 44 tests in 14.672s — OK (skipped=2)
python3.12 -B -m unittest discover -s tests -p 'test_research_*.py'
Ran 44 tests in 16.541s — OK
python3.12 scripts/research_package.py --package research/test-suite-locale-reads
CHECK Locale-independence ...
PASS_TEST_SUITE_LOCALE_READS legacy-locale reads repaired in tests/test_research_current_status.py
PASS_REGISTERED_RESEARCH_PACKAGE research/test-suite-locale-reads outcome=completed
python3 scripts/research_package.py --package research/test-suite-locale-reads --check-only
PASS_PACKAGE_METADATA_AND_PINS (not a mathematical acceptance check)

Regression tests (in the repaired module, following test_research_onboarding.TextEncodingTests: "a UTF-8 CI host cannot observe locale-codec bugs, so check the calls statically"): an AST guard requiring encoding= on every text read/write in the module, and a load-bearing control asserting the document contains e2 80 9d and is undecodable as cp1252 (deterministic on every host).

Failure controls (both executed in this session): the static guard applied to the base-commit module reports missing encoding= at exactly lines 10 and 26; a bare read_text() of the document in a PYTHONUTF8=0 child fails with the same UnicodeDecodeError. The registered replay reruns everything with a forced legacy codec (cp1252 on Windows; PYTHONUTF8=0 PYTHONCOERCECLOCALE=0 LC_ALL=C elsewhere).

Killed first candidate (obstruction recorded). The historical Windows failures cited in many PR bodies (autocrlf temp-fixture pin mismatch in test_research_onboarding.PackageTests, POSIX-only os.killpg, cp1252 metadata reads in research_package.py/find_overlap.py) no longer reproduce at base: test_research_onboarding runs green natively (Python 3.13.15: 18 tests, 2 environment-gated skips; Python 3.12: all 18 pass, core.autocrlf=true), including the Python 3.12-only replay-environment tests. Those repairs are credited below; what remained was the suite's own document readers, which the merged static guard (five named scripts) does not cover.

Limitations (not checked)

• No mathematics: no kappa, certificate, frozen-pin, upstream/ or cold-replay change. The card's acceptance gate (preserve immutable source hashes and proof-relevant content) is untouched: no comparison logic changed.
• The roughly 100 remaining locale-dependent text calls in tests/ and the one-off scripts/ generators were inventoried (1,536 repo files carry non-ASCII bytes, mostly under research/) but not swept. Structured next questions are in research/test-suite-locale-reads/TASK.json.
• The registered replay forces the legacy codec only for the repaired module; whole-suite behavior under a true POSIX ASCII locale is unmeasured. macOS and Linux hosts were not executed in this session.
• SOURCES.json pins tests/test_research_current_status.py at its base-commit bytes as provenance (the defective version this PR repairs); it cannot be a live input under the pin contract because this contribution changes it. docs/research/current-status.md is the live input and matches its pin exactly.

Overlap

Refreshed before submission (2026-10-14 11:34Z, complete pagination: 492 open PRs, 478 discussions): no matches for the refined object terms (test_research_current_status locale, cp1252 UnicodeDecodeError suite). Nearest work, none conflicting: #1702 (CP1252 overlap-regression self-containment in test_research_onboarding.py, different file and object). The card's gzip/canonical-payload variation is under active concurrent work (#1559, #1824) and was deliberately not duplicated.

Credit

Aaron Wang, with help from opencode. Predecessor credit: the merged PR 1299/PR 1298 (rtyu193-rgb) Windows encoding line (UTF-8 metadata and pipe handling, child-tree timeout fallback, cp1252 simulations and the TextEncodingTests static-guard pattern, summarized in research/community-efficient400-audit); the failure report and environment details in #1200 (comment), which identified the same byte in research_package.py; and the upstream authors of tests/test_research_current_status.py itself. This PR adds the instance that guard missed, a regression in the module that failed, and a replayable legacy-locale check.

Remaining obligations: the two structured open questions in research/test-suite-locale-reads/TASK.json (a tests/ encoding sweep, and a whole-suite POSIX ASCII-locale guarantee). Optional workflow feedback: the useful improvement here would be for the TextEncodingTests static guard to cover tests/ document readers as well as the five onboarding scripts, since that gap is exactly how this failure survived the previous fix.

[Mister-Meeseeks]

Batch review at 84ca53150d99215bdbe6ca33107bc0c17708ea3e.

Merged to main, preserving the original contributor commits and package attribution.

Reviewed scope: Explicit UTF-8 status reads, AST regression guard and load-bearing cp1252 control preserve research-status tests under legacy locale. Complete declared 44-test replay and combined current-main checks pass; complements earlier onboarding fixes.

The pinned scoped command passed; the batch ledger contains its command, input hashes, environment and log. This incorporates the research/tooling contribution and does not change the selected κ or discharge the inherited multiplication-theorem assumptions.

Batch review, attribution and evidence. Thank you for the contribution.