Skip to content

t27c test does not execute assertions: counts declarations only (false-green) #11730

Open

Description

@gHashTag

Anomaly

t27c test <spec> prints declaration counts (Tests/Invariants/Benchmarks) and exits 0 WITHOUT executing any assertion.

Proof

A spec containing assert always_one() == 999 (always false) passes with EXIT=0:

module faketest;
pub fn always_one() i32 { return 1; }
test "lying test" {
    assert always_one() == 999;
}

Result: Tests: 1, Total: 1, EXIT=0.

Impact

Any CI gate relying on t27c test is a false-green for assertion correctness. L4 TESTABILITY is only checked for presence of test blocks, not for their truth.

Expected

t27c test should lower the test bodies (they already gen to Zig via t27c gen) and run them, or a new t27c test --exec should do so, failing on a false assertion.

Context

Found while bringing tri git (compiler/cli/git.t27) to release coverage. Discovered because coverage measures call presence, not assertion truth.

Boundary

  • bootstrap/src/main.rs
  • tests/t27c_cli.rs

User Scenarios

Given a spec whose test block contains a deliberately false assertion (assert 1 == 2;), When the agent runs t27c test <spec>, Then the command must exit non-zero instead of reporting declaration counts and exit 0.

Requirements

  • FR-001: t27c test MUST evaluate every assertion in test/invariant blocks and exit non-zero when any assertion is false.
  • FR-002: t27c test MUST distinguish "declared" from "evaluated" in its report.

Success Criteria

With a false assertion planted, t27c test specs/pipeline/loop_guard.t27 exits non-zero (exit 1); with the plant removed the same command exits 0.

Activity

  1. added
    shape-refusedClosed by the issue-shape gate (specs/policy/issue_shape.t27); a passing edit reopens it
    on Oct 23, 2026
  2. github-actions commented on Oct 23, 2026

    @github-actions
    Contributor

    Closed by the issue-shape gate (specs/policy/issue_shape.t27, #7498): an issue must have the template's sections. Edit the body; an edit that passes reopens it.

    • missing: ## Boundary -- list every file this task may touch, one per line (- specs/x.t27); without it the Queen reserves nothing and no bee can take the issue
    • missing: ## User Scenarios -- at least one Given/When/Then scenario
    • missing: ## Requirements -- numbered obligations, FR-001: ... MUST ...
    • missing: ## Success Criteria -- an outcome a machine can settle: a number with a unit, a backticked file, exit 0, or a backticked command
  3. removed
    shape-refusedClosed by the issue-shape gate (specs/policy/issue_shape.t27); a passing edit reopens it
    on Oct 23, 2026
  4. github-actions commented on Oct 23, 2026

    @github-actions
    Contributor

    The shape check passes now (specs/policy/issue_shape.t27); reopened.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions