Repository navigation
Description
Problem and impact
SAFEDEBUG in EzBidPro is implemented, independently reviewed, tested, and merged, but phase-loop-closeout-audit --phase SAFEDEBUG cannot classify preserved workflow review/evidence artifacts under .dev-skills/. They are legitimate workflow outputs, but there is no supported provenance path for them. The executor correctly keeps the phase blocked with dirty_worktree_conflict.
Please add a bounded, provenance-checked way to retain these artifacts without broadly trusting .dev-skills, deleting evidence, moving it into .phase-loop to borrow runner trust, or rerunning historical model reviews merely to manufacture new outputs.
Confirmed implementation gap
Observed installed release: 2026.10.16-grok.1-f9c18ab78c0b. Also checked the relevant files directly from GitHub main at 0fcc004bf105e023e9376c4c2ad5e31052cab4f2 on 2026-10-19; the relevant behavior is still present.
closeout_classifier.py::_grade_by_provenanceenumerates unknown ignored directories, then tries path classification,is_harness_handoff, andverify_declared_output. There is no review/evidence attribution route.generated_outputs.py::_RESERVED_ROOTSincludes.dev-skills;_safe_patternrejects producer declarations covering it. Keep that broad prohibition intact.generated_outputs.py::is_harness_handoff(line 786 at that revision) accepts validated handoff contracts, not review bundles, model outputs, or operational receipts.load_declaration(line 214) reads declarations fromHEAD; adding an uncommitted declaration is not a workaround, and even a committed declaration cannot cover the reserved root.- Tests already enforce important negative behavior in
phase-loop-runtime/tests/test_closeout_generated_outputs.py:test_other_files_under_dev_skills_block,test_unbounded_or_reserved_globs_are_rejected,test_a_generated_file_edited_after_the_producer_ran_blocks, and handoff repo/commit identity tests. Extend these protections rather than relaxing them.
This is not merely a collapsed-parent ignore problem: the actual audit enumerates all 95 leaf files and still cannot attribute them.
Measured reproduction
Read-only commands used on 2026-10-17:
phase-loop-closeout-audit --repo "$UMBRELLA_ROOT" --phase SAFEDEBUG
phase-loop-closeout-audit --repo "$EXTENSION_ROOT" --phase SAFEDEBUG| Tree | Exit | Results |
|---|---|---|
| EzBidPro/EZBidPro umbrella | 1 | 95 unknown_ignored, plus recognized runner output and tool cache |
| EzBidPro/navblue-extension component | 0 | 15 declared_output, 0 unknown |
Unknown umbrella files are exactly:
- 3 JSON files under
.dev-skills/evidence/SAFEDEBUG/:interactive-receipt.json,safedebug-ablation-matrix.json,safedebug-prod-behavior.json. - 92 historical artifacts under
.dev-skills/reviews/roadmap-v7-20261014T063025Z/, including review input bundles, result JSON/logs, retries, failed/empty outputs, and explicitly labelled stand-in reviews.
The independent Sol audit hashed all 95 files. Thirteen explicit internal digest bindings matched; zero mismatched. Seventeen files had full-digest references in inspected metadata; 78 did not. No symlinks or unexpected output families were found. Subsequent verification confirmed the original 95 file hashes remained unchanged.
These facts prove consistency, not authenticated historical authorship. There is no immutable original baseline for every historical file. A solution must not pretend that hashing old files today proves who produced them.
Local maintainer reproduction paths (metadata only, no capture data):
- Umbrella:
/mnt/workspace/worktrees/viperjuice/EZBidPro-debug-release-roadmap-v7 - Component:
/mnt/workspace/worktrees/viperjuice/navblue-extension-safedebug-v7 - Per-file inventory: umbrella
.phase-loop/runs/safedebug-sol-audit-20261017/readonly-audit.json - Latest repair report: umbrella
.phase-loop/runs/safedebug-ec5-repair-20261017205131Z/repair-report.json
These local artifacts are not available from a fresh clone. Tests should reproduce the failure using synthetic files, not copy private captures or local review transcripts into fixtures.
Minimal synthetic regression shape
- Create a temporary Git repo with
.gitignorecontaining/.dev-skills/and.phase-loop/; commit normal fixture inputs. - Write a valid harness handoff and synthetic review/evidence artifacts through the supported producer path being added.
- Include a known failed review artifact and an empty output: retention must not imply successful review.
- Run the actual audit, including Git's collapsed-directory reporting and leaf enumeration.
- Verified artifacts should be attributable; an arbitrary sibling
.dev-skills/notes.txtmust still block.
Today steps 2–4 have no supported review/evidence producer record to use, and review/evidence leaves remain unknown even when referenced by a plan/evidence document.
Suggested implementation direction (design proposal, not a mandated schema)
Introduce a narrowly scoped artifact-provenance record for harness review/evidence producers and an explicit attribution branch in the classifier. Reuse existing sealing/path/identity helpers where appropriate. Bind exact paths and SHA-256 values to a repository identity, producing run/task and artifact kind, and the relevant phase or explicitly consumed upstream review. Record producer identity at creation, rather than accepting a hand-written claim merely because it contains matching hashes.
Important design distinctions:
- Retention provenance versus acceptance: a failed, degraded, empty, or stand-in review can be valid retained history while providing no successful review vote. Classification must never upgrade its verdict.
- Historical adoption versus original authorship: define an explicit, auditable adoption route for existing artifacts lacking producer receipts, or continue rejecting those files with an actionable reason. Any authorized adoption records who accepted custody, when, exact digests, and known limitations; it must not backdate or invent producer identity. A self-authored digest manifest alone must not confer trust.
- Cross-phase evidence: roadmap reviews predate SAFEDEBUG. Allow an explicit verified dependency relation if appropriate; do not require rewriting history to label every artifact as produced by the consuming phase.
- Commit changes: reviewed artifacts can legitimately outlive a code commit. Bind them to their actual reviewed inputs and consumption record. Do not weaken the existing HEAD/phase binding of deterministic build-output records.
- Privacy: audit metadata and diagnostics should identify paths/digests/reasons without echoing evidence contents or secrets. No blanket recursive content import.
Likely touch points: closeout_classifier.py, a dedicated or shared provenance helper near generated_outputs.py, actual review/evidence producer write paths, corresponding runtime tests, and operator/skill documentation for existing artifacts. Locate the real producer paths before selecting final ownership; do not patch only the consumer and leave producers unable to emit accepted records.
Acceptance and negative tests
- Actual CLI audit accepts correctly recorded synthetic review/evidence artifacts under parent-directory ignore rules, with a specific explanatory attribution reason.
- Modified bytes, missing receipt/artifact, malformed/unsupported record, wrong repo, wrong producing task/run, unapproved historical adoption, or absent phase/dependency linkage fail closed with useful diagnostics.
- Reject path traversal, absolute/outside-repo paths, symlink escape, and inappropriate cross-repo receipt reuse. Cover relevant case-insensitive reserved-path behavior.
- Unregistered siblings under
.dev-skillsstill block;.dev-skills/**remains forbidden as a generic producer declaration. - Failed/empty review artifacts may be retained without becoming successful reviews or acceptance evidence.
- Existing handoff, deterministic build-output, stale-record, and unknown-output tests stay green.
- Artifact capture/recording failures are surfaced; no partial registration silently blesses unmatched files.
- A read-only audit does not move, rewrite, delete, rerun, or relabel historical evidence.
- Provide a documented migration/adoption procedure and clearly report any historical files that still cannot be attributed. Passing all 95 without adequate evidence is NOT an acceptance requirement.
- After a reviewed release is installed, rerun the actual EzBidPro audits; close SAFEDEBUG only if its remaining requirements genuinely pass. Runtime classification is not a substitute for phase acceptance.
Evidence and work already completed
- Product fixes: https://github.com/EzBidPro/navblue-extension/pull/24 (merged).
- Roadmap, regression tests, and truthful incomplete evidence: https://github.com/EzBidPro/EZBidPro/pull/26 (merged).
- Documentation-review operational recording repair: https://github.com/EzBidPro/EZBidPro/pull/27 (merged).
- Latest sealed verification: run
safedebug-ec5-repair-20261017205131Z; artifact SHA-2567b9927412ffeeb4501ed0341486eee98635e3d8b6132b43b6169985b9110310d; loga52c5985d91f5a491b806a0af4e15ce09cba8c4a2fb7b28e4098e0ea8596ce74. - Verification passed: focused extension 120/120, selected umbrella 15/15, full extension 533/533, typecheck, production/development builds. A separate full umbrella run passed 46/46.
- EC5 is already operationally recorded: original completed T3 review identities, verdicts, and reviewed hashes were embedded through public
verification_evidence.run_verification, with original artifacts preserved. This issue does not require a newpresident.ruling.v1authorization or another model review merely to record an existing one.
Related issues / scope boundaries
- Closeout audit blocks on harness handoffs and declared Node/BAML outputs #1139 and Attribute Node/BAML and resolver handoff outputs in closeout audit #1226: closed work for handoffs and deterministic Node/BAML outputs. This issue covers retained review/operational evidence not handled by those mechanisms.
- Collapsed parent ignore rule still blocks required skill handoffs after agent-harness#1084 #1209: collapsed-parent handoff ignore handling; related but not this leaf-level attribution gap.
- closeout audit follow-ups from agent-harness#1189 landing: scope on non-closing routes, route tests, one phase resolver #1231: closeout audit follow-ups; coordinate to avoid divergent provenance decisions.
- No EZBidPro product-source changes, evidence deletion, bulk artifact relocation, broad ignored-path allowlisting, or acceptance-gate waiver is requested.
Activity