Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/signal_contracts.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ written.** Corollaries every audited contract has needed so far:

## Signals

5 stated, 63 draft. A **draft** row is the schema comment transcribed as-is; a **stated** row has been audited across the corpus languages and has a contract doc. `planted` = the keyword-rosetta corpus plants a known count of it (so the cross-language gate can hold it equal); unplanted signals that feed a risk formula are the ones the roadmap's Phase 3 must plant or declare absent.
6 stated, 62 draft. A **draft** row is the schema comment transcribed as-is; a **stated** row has been audited across the corpus languages and has a contract doc. `planted` = the keyword-rosetta corpus plants a known count of it (so the cross-language gate can hold it equal); unplanted signals that feed a risk formula are the ones the roadmap's Phase 3 must plant or declare absent.

| signal | phase | kind | status | planted | contract | doc |
|---|---|---|---|---|---|---|
Expand All @@ -77,7 +77,7 @@ written.** Corollaries every audited contract has needed so far:
| `safety` | safety | `site` | draft | yes | Defensive programming constructs that prevent crashes at runtime | |
| `safety_bypasses` | safety | `site` | draft | yes | Syntax that actively bypasses type safety, swallows errors, or relies on unpredictable state | |
| `state_mutation` | safety | `site` | stated | yes | A statement that writes a new value into state that already exists | [state_mutation_rule_contract.md](../docs/state_mutation_rule_contract.md) #2765 |
| `test` | safety | `site` | draft | yes | Assertions and unit testing framework keywords | |
| `test` | safety | `site` | stated | yes | A site that engages a testing framework: a test-case or fixture declaration, a framework assertion or expectation, or the framework named as such | [test_rule_contract.md](../docs/test_rule_contract.md) #2852 |
| `closures` | architecture | `declaration` | draft | | Anonymous functions, lambdas, inline callbacks | |
| `comprehensions` | architecture | `site` | draft | | Collection iterators or inline looping | |
| `concurrency` | architecture | `site` | draft | | Asynchronous logic and parallel execution | |
Expand Down
184 changes: 184 additions & 0 deletions docs/test_rule_contract.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,184 @@
# The `test` rule contract (#2852)

> **One hit is a site that engages a testing framework — a test-case or
> fixture declaration, a framework assertion or expectation call, or the
> framework itself named — in a form an ordinary identifier cannot match;
> the language's own runtime guard is safety's hit, never test's.**

Stated 2026-09-07 by the #2852 audit (roadmap Phase 3, epic #2812).
Precedents: `docs/api_rule_contract.md` (#2730), `docs/args_rule_contract.md`
(#2773), `docs/state_mutation_rule_contract.md` (#2765),
`docs/branch_rule_contract.md` (#2822), `docs/io_rule_contract.md` (#2841).
The machine-readable row is `gitgalaxy/standards/signal_contracts.py`; the
cross-language pins are `tests/extraction/languages/test_test_contract_2852.py`.

`test` is a **site**-kind signal. Its consumers: the `risk_verification`
formula (`signal_processor.py` reads the hit count as `verification`, weighted
by the fidelity table), the `FIDELITY_SIGNALS` tuple, the "Testing Exposure"
display row, the `statistical_auditor` extraction-density group, and
`network_risk_sensor`'s per-function `test_hits`. (`detector.py`'s
name-verb heuristic tags a function whose *name* contains test/assert/mock —
that is a name heuristic, not this rule.) Before this contract it carried 3 of
the corpus's 34 open-defect cells (c, embedded_python, perl — 7% of its
comparable cells); after it, every corpus language reads **2**.

## Corollaries

**C1 · The runtime guard is safety's.** The language's built-in production
assertion guards the running program — it fires in deployment, not under a
harness — so safety's contract claims it, and in every affected language the
safety rule already owned the token (mass conserved): C's `assert(` (assert.h),
python's and embedded_python's `assert` statement, Lua's builtin `assert(`.
This is #2626's python resolution stated as a rule for language 47:
embedded_python still carried the bare `assert` token #2626 removed from
python (the twin was missed), and c's rule ended `|\bassert\s*\(`. A
*framework's* assertion form is test's: `CU_ASSERT`, `ASSERT_*`/`EXPECT_*`
(the uppercase macro is the framework form — CPython's own
`ASSERT_DICT_LOCKED()` runtime macros are indistinguishable by surface and
accepted), `assertEquals`, `XCTAssert…`, `assert_eq!`, and luassert's matcher
chain `assert.<chain>` (`assert.are.equal`, `assert.True`), which is exactly
the shape that separates busted's assertion from Lua's guard: the dot, not
the paren. Lua's crucible count fell 3471 → 134 on this line alone, and the
134 that remain are all `assert.are_*`/`assert.error_*` chains.

**C2 · One statement is one hit; a module qualifier is not a separate hit
from the call it qualifies.** perl's `Test::More::ok($kit)` fired twice —
once for the framework name `Test::More`, once for the call form `ok(` — so
the corpus read 3 against a planted 2. The module names now carry `(?!::)`:
a qualified call counts once through its call form, a bare mention
(`use Test::More;`) still counts once as the framework named (C4).

**C3 · The ambiguity anchor** (io C1). An everyday word fires only anchored
to its framework form; the crucible measured each of these misfiring:

- **dart** — bare `test` matched a predicate parameter (`if (test(element))`)
and bare `group` a loop variable (`for (final group in
siblingMergeGroups)`); a test-case declaration is now the call with its
description string (`test('adds', …)`, `group('kit', …)`), the corpus probe
re-planted to the honest form. Crucible 38 → 2.
- **php** — bare `test` matched Twig's ordinary `->test(` method and bare
`mock` the `$mock` variable; everyday words now anchor to their call form
(`test('adds', function`, `mock(User::class)`, `$this->expects(`), with
`(?<!->)` on `test`/`it` because Pest's forms are bare calls and a method
named `test` is not a test case. Crucible 53 → 7, the remainder Mockery
calls.
- **cobol** — `-` is a regex word boundary, so `TEST-CASE` matched *inside*
`UT-TEST-CASE-COUNT` — cblunit's own counter variables, 51 hits of an
identifier, the exact #2622 hyphen shape. The rule now carries
`(?<!-)…(?!-)` guards. Crucible 51 → 0 (the framework's counters were the
only hits; a real `ASSERT X = Y` statement still fires).

An unambiguous framework name (`pytest`, `unittest`, `PHPUnit`, `busted`,
`luassert`, `Test::More`, `testWidgets`, `XCTest`) may fire bare — the
crucible shows no identifier collisions on them.

**C4 · Naming the framework counts, once.** A testing-framework name is test
evidence wherever it appears — the planted position in python and
embedded_python is a bare mention (`suite = unittest`), and it stands. Where
the mention sits inside an inclusion statement (`import unittest`,
`use Test::More`), the include keyword is import's hit and the framework name
test's — different tokens on one line, a deliberate overlapping taxonomy
(the fortran COMMON shape), not a dual to retire.

**C5 · Stated absence.** A language records `test: None` when it has no
per-case idiom a framework executes: yacc (ledger
`yacc-test-no-native-testing-concept` — `%expect N` is a one-shot declarative
pragma already claimed for file routing), jcl ("test has no JCL concept",
`jcl-2610-rebaseline-residual-morphology`), markdown (lit plane,
`markdown-lit-plane-morphology`). The absence answer and the rule answer
cannot coexist in one language; absence needs crucible evidence, not merely
an unplanted corpus.

## Deliberate duals and deferred residue

- **groovy Spock labels** (`when:`/`then:`/`expect:` line-anchored) are the
framework form of a block label — 948 crucible hits, all Spock specs; kept.
- **assembly's `testcase` macro table** — each `testcase {…}` data row
declares one case for the harness that expands it; 275 hits, kept.
- **powershell Pester** (`Should`, `BeforeAll`) — framework vocabulary; kept.
- **dart `when (`** — dart 3's switch-case guard clause collides with
mockito's `when(` (1 crucible hit); accepted, the guard form is rare and
mockito's is planted nowhere.
- **php `public static function spy()`** — Mockery declaring its own `spy`
method fires the call-anchored form once; accepted (it is framework
source).
- **Bare-word menus the crucible cannot measure** (ruby's
`before`/`after`/`let`/`subject`, javascript's bare `describe`/`expect`/
`assert`, java's and groovy's `assert…(` runtime-statement exposure,
assembly's `(?i)` prose vocabulary): deferred with #2853 — no crucible
files, no corpus cell, nothing moves today. The audit records these as
conforming-by-absence-of-evidence, not validated.

## The 46-language audit

`rule_probe.py test all --samples 8` before → after. crucible = hits across
the control corpus; rosetta = hits across the 4 keyword-rosetta shell files
(median plant: 2). Languages not listed: rule `None` by stated absence (yacc)
or no test morphology declared (batch, blp, csv, glsl, hlo, json, mlir, nix,
pbtxt, plaintext, proto, td, xml); jcl and markdown are n/a by intended
morphology.

| language | crucible | rosetta | verdict |
|---|---|---|---|
| abap | 0 | 2 | conforms (`FOR TESTING`/`CL_ABAP_UNIT_ASSERT` anchored) |
| ada | — | 2 | conforms (`AUnit`, `Assert(` — Ada has no runtime assert keyword) |
| agc_assembly | 13 | 2 | conforms (SELFCHECK/ROPECHK are its self-test ops) |
| apex | 116 | 2 | conforms (@isTest/System.assert anchored forms) |
| assembly | 275 | 2 | conforms; `testcase` macro rows are case declarations; `(?i)` prose surface → #2853 |
| c | 1052 → 97 | 3 → 2 | **C1**: `assert(` → safety's; uppercase `ASSERT_*` macros stay (framework form) |
| cobol | 51 → 0 | 2 | **C3**: hyphen guards; `UT-TEST-CASE-COUNT` was an identifier (#2622 shape) |
| cpp | 0 | 2 | conforms (framework macros only, no bare assert() alternative) |
| csharp | 0 | 2 | conforms (attribute-anchored `[Test]`/`[Fact]`) |
| css | 0 | 2 | conforms (`data-testid` attribute form) |
| dart | 38 → 2 | 2 | **C3**: `test`/`group` anchored to description-string call; probe re-planted |
| dockerfile | 0 | 2 | conforms (command forms `npm test`/`pytest`) |
| embedded_python | 0 | 3 → 2 | **C1**: bare `assert` removed (#2626 applied to the twin) |
| fortran | 0 | 2 | conforms (pFUnit `@test`/`@assertEqual` directive-anchored) |
| go | 2 | 2 | conforms (`TestX`/`t.Run`/`assert.X(` anchored) |
| groovy | 948 | 2 | conforms; Spock labels line-anchored; `assert…(` exposure → #2853 |
| haskell | 0 | 2 | conforms (hspec/QuickCheck names, `prop_` prefix) |
| html | 0 | 2 | conforms (`data-testid=` attribute form) |
| java | 0 | 2 | conforms; `assert…(` runtime-statement exposure → #2853 |
| javascript | 5 | 2 | conforms; anchored `it`/`test` halves already contract-shaped; bare `describe`/`expect`/`assert` → #2853 |
| kotlin | 1 | 2 | conforms (annotation + call-anchored forms) |
| livecode | 0 | 2 | conforms (`command test*`/`pass test` statement forms) |
| lua | 3471 → 134 | 2 | **C1**: bare `assert` → safety's; luassert chain `assert.<chain>` added (the 134 are all chains) |
| m4 | 0 | 2 | conforms (autotest `AT_*` macros) |
| makefile | 0 | 2 | conforms (command forms) |
| matlab | 0 | 2 | conforms (`matlab.unittest`/`verifyEqual` names) |
| objective-c | 0 | 2 | conforms (`XCTAssert*` family) |
| perl | 0 | 3 → 2 | **C2**: `(?!::)` on module names; `Test::More::ok(` counts once |
| php | 53 → 7 | 2 | **C3**: everyday words call-anchored, `(?<!->)` on `test`/`it`; remainder Mockery calls |
| powershell | 203 | 2 | conforms (Pester `Should`/`Describe`/`It` forms) |
| python | 1221 | 2 | conforms (#2626 already removed `assert`; framework names + `def test_`) |
| ruby | 0 | 2 | conforms-by-absence-of-evidence; bare-word menu → #2853 |
| rust | 464 | 2 | conforms (`#[test]`/`assert!` macros — the macro *is* the framework form) |
| scala | 0 | 2 | conforms (framework names + `it should`) |
| scheme | 0 | 2 | conforms (paren-anchored srfi-64 forms) |
| shell | 59 | 2 | conforms (bats `@test`/shunit2 `assert_eq` forms) |
| solidity | 0 | 2 | conforms (Foundry `test*`/`assertEq` prefix forms) |
| sqlite | 75 | 2 | conforms (`.testcase`/`PRAGMA integrity_check` statement forms) |
| swift | 0 | 2 | conforms (`XCTest` family + `@Test`/`#expect`) |
| tcl | 68 | 2 | conforms (`do_test`/`tcltest::` command forms) |
| typescript | 310 | 2 | conforms (call-anchored, `(?<!\.)` guards already present) |
| yaml | 0 | 2 | conforms (workflow command forms) |
| zig | 105 | 2 | conforms (`test "…"` block declaration) |

## Ledger dispositions this contract settles

- `assert-overlaps-safety-and-test` — **resolved** for its two remaining
languages (c, perl); python was resolved by #2626. c's dual is retired by
C1, perl's double-fire by C2. The entry closes.
- `batch4-dual-keyword-overlaps` — loses its test half for embedded_python
(the bare `assert` token); the other itemized duals are untouched.
- `yacc-test-no-native-testing-concept` — unchanged, now cited as the C5
precedent.
- `jcl-2610-rebaseline-residual-morphology` / `markdown-lit-plane-morphology`
— unchanged (n/a cells, not defects).

## Bless scope

Golden-master movement is confined to the `test` signal and the formulas that
read it (`risk_verification`, the fidelity-weighted verification term);
newly parsed/excluded: none. The large crucible deltas (c −955, lua −3337,
php −46, cobol −51, dart −36) are the C1/C3 misfires leaving the count.
2 changes: 1 addition & 1 deletion gitgalaxy/standards/how_to_add_a_language.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ Generate a valid Python dictionary matching this exact structure.
"dead_code": re.compile(r""),
# doc: Structured documentation meant to be parsed by IDEs or generators. Includes: JSDoc, Docstrings.
"doc": re.compile(r""),
# test: Assertions and unit testing framework keywords. Includes: describe, it, assert, expect.
# test: A SITE THAT ENGAGES A TESTING FRAMEWORK: A TEST-CASE OR FIXTURE DECLARATION, A FRAMEWORK ASSERTION OR EXPECTATION, OR THE FRAMEWORK NAMED AS SUCH. The language's own runtime guard (C `assert(`, python's `assert` statement, Lua `assert(`) is safety's hit, never test's; a framework's assertion form (CU_ASSERT, ASSERT_*/EXPECT_*, assertEquals, luassert's `assert.<chain>`) is test's. One statement is one hit -- a module qualifier is not a separate hit from the call it qualifies (`Test::More::ok(` = 1). An everyday word (test, it, ok, group, mock...) fires only anchored to its framework form (call parens with a description string, declaration position, annotation `@Test`/`#[Test]`/`[Test]`, block label) with hyphen guards where `-` is a word boundary; an unambiguous framework name (pytest, unittest, PHPUnit, Test::More, busted) may fire bare and counts wherever it appears. A language with no per-case idiom a framework executes records the stated absence instead (`None` + ledger entry, yacc). Full contract, corollaries and the 46-language audit: docs/test_rule_contract.md (#2852).
"test": re.compile(r""),

# --- PHASE 3: ARCHITECTURE & DOMAIN SENSORS ---
Expand Down
5 changes: 2 additions & 3 deletions gitgalaxy/standards/language_standards/languages/c.py
Original file line number Diff line number Diff line change
Expand Up @@ -237,9 +237,8 @@
r"/\*\*[\s\S]{0,15000}?\*/|///[^\n]*|@param|@return|@brief|@details|\\param|\\return|\\brief|\\details"
),
# 14. test (Testing & Assertions)
"test": re.compile(
r"\b(?:TEST|TEST_F|TEST_CASE|CU_ASSERT|RUN_TEST|EXPECT_[A-Z_]+|ASSERT_[A-Z_]+)\b|\bassert\s*\("
),
# #2852 contract C1: bare assert( is the C runtime guard -- safety's hit (python precedent #2626)
"test": re.compile(r"\b(?:TEST|TEST_F|TEST_CASE|CU_ASSERT|RUN_TEST|EXPECT_[A-Z_]+|ASSERT_[A-Z_]+)\b"),
# --- PHASE 3: ARCHITECTURE & DOMAIN SENSORS ---
# 15. concurrency (Asynchronous Execution)
"concurrency": re.compile(
Expand Down
3 changes: 2 additions & 1 deletion gitgalaxy/standards/language_standards/languages/cobol.py
Original file line number Diff line number Diff line change
Expand Up @@ -367,7 +367,8 @@
re.I | re.M,
),
# 14. test: Testing & Assertions. Unit testing framework markers (ZUnit).
"test": re.compile(r"\b(ZUNIT|CBLUNIT|ASSERT|TEST-CASE|READY\s+TRACE)\b", re.I),
# #2852 contract C3: hyphen guards -- UT-TEST-CASE-COUNT is an identifier, not a test case (the #2622 hyphen shape)
"test": re.compile(r"(?<!-)\b(ZUNIT|CBLUNIT|ASSERT|TEST-CASE|READY\s+TRACE)\b(?!-)", re.I),
# --- PHASE 3: ARCHITECTURE & DOMAIN SENSORS ---
# 15. concurrency: Temporal Static. CICS Task and resource coordination.
"concurrency": re.compile(r"\bEXEC\s+CICS\s+(?:ENQ|DEQ|WAIT|START|DELAY)\b", re.I),
Expand Down
4 changes: 3 additions & 1 deletion gitgalaxy/standards/language_standards/languages/dart.py
Original file line number Diff line number Diff line change
Expand Up @@ -300,7 +300,9 @@
"doc": re.compile(r"/\*\*[\s\S]{0,15000}?\*/|///[^\n]*|@param|@return"),
# 14. test: Testing & Assertions. Flutter test frameworks and standard expect/verify markers.
"test": re.compile(
r"\b(?:test|testWidgets|group|setUp|tearDown|pumpWidget|pumpAndSettle|find\.(?:byType|text|byKey))\b|\b(?:expect|verify|when)\s*\("
# #2852 contract C3: bare test/group matched ordinary identifiers (a predicate parameter, a loop
# variable); a test-case declaration is the call with its description string
r"\b(?:testWidgets|setUp|tearDown|pumpWidget|pumpAndSettle|find\.(?:byType|text|byKey))\b|\b(?:test|group)\s*\(\s*['\"]|\b(?:expect|verify|when)\s*\("
),
# --- PHASE 3: ARCHITECTURE & DOMAIN SENSORS ---
# 15. concurrency: Temporal Static. Event Loop primitives (Future, Stream, Isolate).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,8 @@
# as `def[ \t]+test_` instead (matches python's own fix for the
# identical trap), dropping the trailing `\b` so it fires on the
# realistic `def test_<name>` shape.
"test": re.compile(r"\b(unittest|pytest|assert|setUp|tearDown|Mock)\b|def[ \t]+test_"),
# #2852 contract C1: the assert statement is the runtime guard -- safety's hit (#2626 applied to the twin)
"test": re.compile(r"\b(unittest|pytest|setUp|tearDown|Mock)\b|def[ \t]+test_"),
# --- PHASE 3: ARCHITECTURE & DOMAIN SENSORS ---
# 15. concurrency (Asynchronous Execution)
"concurrency": re.compile(
Expand Down
3 changes: 2 additions & 1 deletion gitgalaxy/standards/language_standards/languages/lua.py
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,8 @@
),
# 14. test: Testing & Assertions. Busted, LuaUnit, and custom verification markers.
"test": re.compile(
r'\b(?:setup|teardown|busted|luassert|assert|mock|stub|spy|luaunit|Test[A-Z]\w*)\b|\b(?:describe|it)\s*[\'"(]'
# #2852 contract C1: bare assert( is Lua's runtime guard -- safety's hit; the framework form is the matcher chain assert.<chain>
r'\b(?:setup|teardown|busted|luassert|mock|stub|spy|luaunit|Test[A-Z]\w*)\b|\bassert\s*\.|\b(?:describe|it)\s*[\'"(]'
),
# --- PHASE 3: ARCHITECTURE & DOMAIN SENSORS ---
# 15. concurrency: Temporal Static. Lua coroutines and task schedulers.
Expand Down
3 changes: 2 additions & 1 deletion gitgalaxy/standards/language_standards/languages/perl.py
Original file line number Diff line number Diff line change
Expand Up @@ -247,7 +247,8 @@
),
# 14. test: Testing & Assertions. Assertions and Test frameworks.
"test": re.compile(
r"\b(?:Test2::V0|Test::More|cmp_ok|is_deeply|subtest|done_testing|BAIL_OUT)\b|\b(?:ok|is|isnt|like|unlike|plan|diag|note)\s*\("
# #2852 contract C2: a module qualifier is not a separate hit from the call it qualifies (Test::More::ok( = 1)
r"\b(?:(?:Test2::V0|Test::More)(?!::)|cmp_ok|is_deeply|subtest|done_testing|BAIL_OUT)\b|\b(?:ok|is|isnt|like|unlike|plan|diag|note)\s*\("
),
# --- PHASE 3: ARCHITECTURE & DOMAIN SENSORS ---
# 15. concurrency: Temporal Static. Async, forks, and threads.
Expand Down
Loading
Loading