Skip to content

Corrección: fix(qa): report failing field and reason on evidence validation (Fixes #375) (#376) #377

Open

Description

@barbatdev

Source PR: #376
Branch: fix/375-qa-evidence-schema-discovery
Head reviewed: e006021

Push the fixes to the branch above; do not open a new pull request. Comment on the PR and on this issue when done.

Blocking findings

  • file_not_found diagnostic is unreachable; a missing evidence file reports file_unavailable (internal/cli/doctor.go:345): readQAFile collapses every os.Stat failure into fmt.Errorf("request is not a bounded regular file") (internal/cli/doctor.go:476). That error is a plain errors.New value with no Unwrap and no *PathError, so both os.IsNotExist(err) and errors.Is(err, os.ErrNotExist) evaluated at line 345 are always false: the branch is dead code. Control falls through the two statErr == nil checks (which also fail for a missing path) and reaches the final else, printing evidence_field=path evidence_reason=file_unavailable. The PR body's Runtime Harness item 6 states as verified that a nonexistent file emits evidence_field=path evidence_reason=file_not_found, and the new CLI test TestQARunEvidenceDiagnosticAttribution (internal/cli/qa_test.go:396) covers none of the four read-phase reasons, so nothing catches the mismatch. The whole point of qa run --evidence record schema is undiscoverable: bare invalid_request and no schema docs #375 is accurate attribution of the failing input; a typo in a path is the most common way to hit this path and it is the one case that reports the wrong reason.
    • Failure scenario: cortex qa run --role requirements-analyst --request request.txt --catalog ./catalog --evidence ./missing.json (path does not exist) -> stderr ends with evidence_field=path evidence_reason=file_unavailable. The claimed and documented file_not_found reason is never emitted for any input, because readQAFile's stat error is unwrapped and both not-exist predicates are false.

Acceptance criteria

  • Fixed: file_not_found diagnostic is unreachable; a missing evidence file reports file_unavailable (internal/cli/doctor.go:345). The scenario below no longer reproduces: cortex qa run --role requirements-analyst --request request.txt --catalog ./catalog --evidence ./missing.json (path does not exist) -> stderr ends with evidence_field=path evidence_reason=file_unavailable. The claimed and documented file_not_found reason is never emitted for any input, because readQAFile's stat error is unwrapped and both not-exist predicates are false.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions