Skip to content

test has no stated contract: c and embedded_python count safety's planted assert, and perl's Test::More::ok( fires the rule twice #2852

Closed

Description

@squid-protocol

Roadmap Phase 3 family (docs/contract_roadmap.md, epic #2812), next by the mechanical audit: test carries 3 of the report's 34 open-defect cells (c, embedded_python, perl — all extraction), and the sheet row is draft:

_c("test", "safety", "site", "Assertions and unit testing framework keywords", planted=True)

Nothing states what one test hit is, and the three red cells are two diseases (all three verified by direct regex fire against the corpus plants):

  • c — the rule ends |\bassert\s*\(, so probe_safety's planted assert(value) counts test +1 (3 vs planted 2). This is the exact shape fix(python): stop io/test rules double-counting globals/safety keywords (#2593) #2626 already resolved for python: the ledger's assert-overlaps-safety-and-test entry records it as RESOLVED for python (assert removed from the test rule, still matched by safety) and still reproducing for c.
  • embedded_python — the rule is \b(unittest|pytest|assert|setUp|tearDown|Mock)\b|def[ \t]+test_: it still carries the bare assert token that fix(python): stop io/test rules double-counting globals/safety keywords (#2593) #2626 removed from python's rule, so probe_safety's assert isinstance(value, int) counts test +1. The twin was simply missed.
  • perl — probe_test's Test::More::ok($kit) fires the rule twice: the alternation matches both the framework name Test::More and the call form ok( on one statement. One assertion site, two hits — the module qualifier is being counted separately from the call it qualifies.

The contract has to decide the shapes every audit so far has hit:

  1. Who owns assert — the language's own runtime guard (C's assert.h macro, python's assert statement) is a safety construct, not a test; a framework's assertion form (CU_ASSERT, ASSERT_EQ, assertEquals, XCTAssert…, assert_eq!) is test's. fix(python): stop io/test rules double-counting globals/safety keywords (#2593) #2626 is the precedent; the contract must state it so c/embedded_python follow and so the other languages carrying bare assert forms (assembly, cobol ASSERT, groovy, java, javascript, lua) can be audited against it rather than case-by-cased.
  2. One statement is one hit — a framework/module qualifier is not a separate hit from the call it qualifies (perl's Test::More::ok(). Same corollary the io contract (io has no stated contract: javascript counts an identifier named path, sqlite counts every DML keyword, and tcl's source/close are claimed by two rules each #2841 C4) and branch contract (branch has no stated contract: handler keywords, the syntax words of one conditional, and goto count as decisions in some languages and not in others #2822 C2) stated for their layers.
  3. Bare-token menus vs anchored constructs (io C1) — several rules count ordinary words with no anchor: php's \b(…|test|it|mock|spy|expects|toBe)\b, ruby's \b(…|before|after|let|subject|context)\b, lua's bare assert|mock|stub|spy, javascript's bare describe|expect|assert. None carries a red cell today (the corpus doesn't stress them), but the rule-probe crucible measurement decides which are real-world misfires and which anchoring the contract requires.
  4. Framework mention vs verification site — python/embedded_python plant bare module references (suite = unittest); the contract must say whether naming a testing framework is itself evidence of test surface (the current planted position) or whether only a declaration/assertion site counts, and the plants must match the answer.
  5. Contract-level absence — yacc deliberately carries test: None (ledger yacc-test-no-native-testing-concept: %expect is a declarative pragma already claimed for file routing, not a per-case test idiom), and jcl/markdown are n/a (jcl-2610-rebaseline-residual-morphology: "test has no JCL concept"; markdown-lit-plane-morphology). The contract must state the absence rule so language 47 knows when None is correct.

Not in scope: the jcl/markdown morphology entries above (those cells are n/a, not defects, and must not move).

Done in the #2765/#2822/#2841 shape: one sentence + corollaries in docs/test_rule_contract.md, rules fixed to the sentence with the matching keyword-rosetta plants, sheet row → stated, test open defects 3 → 0 (or filed), the assert-overlaps-safety-and-test ledger entry resolved for its two remaining languages and batch4's test half retired or recorded as deliberate.

Metadata

Metadata

Assignees

No one assigned

    Labels

    core-engineModifications to the central physics and parsing enginemetricsHeuristics, risk exposures, and topological math updates

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions