Skip to content

Document scaffolded-to-embedded file mapping to prevent drift detection misses #116

Closed

Description

@fullsend-ai-retro

What happened

PR #109 updated .opencode/skills/always-on-guidance/SKILL.md via the v0.17.0 scaffold deployment but did not update the byte-identical embedded copy at internal/agentkit/content/skills/always-on-guidance/SKILL.md. This broke TestSkillFiles_DriftDetection (byte-compares the two copies) and TestAlwaysOnGuidance_StructuralHardening.

The fullsend review agent (run 34944276471) found 14 medium/low findings but missed this CRITICAL issue entirely. The human reviewer em-redhat caught it 2 days later using the repo's local /uf.review-pr command, which had access to CI status showing Build and Test: FAIL. The author fixed it by reverting the scaffolded copy to match the embedded one.

The relationship between scaffolded files (.opencode/) and their embedded copies (internal/agentkit/content/) is enforced by drift detection tests but is not documented in AGENTS.md or CLAUDE.md. Neither the review agent nor the PR author was prompted to check both locations.

What could go better

If the scaffolded-to-embedded file mapping were documented in AGENTS.md (which all agents read), reviewers would be prompted to check both locations when either changes. The Go convention pack already mentions drift detection as a pattern (TC-007: "Implement drift detection tests to ensure embedded assets match their canonical source files"), but without documenting which specific files are paired, reviewers cannot apply this rule.

Confidence: High that documentation would help. The pattern is well-established in this repo (multiple test files enforce it), but the mapping is implicit — you have to read the test code to discover it. Making it explicit in developer documentation is a low-cost, high-signal improvement.

Uncertainty: Adding documentation does not guarantee the review agent will act on it. The review agent may still miss the connection if it does not cross-reference changed files against the documented mapping. However, it raises the floor — any reviewer who reads AGENTS.md will know to check.

Proposed change

Add a "Mirrored Files" or "Drift Detection Mapping" section to AGENTS.md documenting the paired file relationships. Example:

## Drift Detection (Mirrored Files)

The following scaffolded files have byte-identical embedded copies
under `internal/agentkit/content/`. Changes to either location
MUST be synchronized. `TestSkillFiles_DriftDetection` enforces
parity at CI time.

| Scaffolded Path | Embedded Path |
|----------------|---------------|
| `.opencode/skills/always-on-guidance/SKILL.md` | `internal/agentkit/content/skills/always-on-guidance/SKILL.md` |
| `.opencode/agents/*.md` | `internal/agentkit/content/agents/*.md` |
| `.opencode/commands/*.md` | `internal/agentkit/content/commands/*.md` |

When reviewing or implementing changes to files in either column,
always verify the corresponding copy is updated. Run `make check`
to confirm drift detection tests pass.

The exact file pairs should be verified against the current TestSkillFiles_DriftDetection test implementation.

Validation criteria

  1. The next scaffold deployment PR to this repo should either (a) receive a review finding about embedded copy sync from the review agent, or (b) the PR author should proactively sync both copies, citing the documented mapping.
  2. Run grep -r 'Drift Detection\|Mirrored Files' AGENTS.md returns the new section.
  3. The documented mapping matches the actual file pairs checked by TestSkillFiles_DriftDetection.

Generated by retro agent from #109

Activity

  1. fullsend-ai-triage commented on Sep 28, 2026

    @fullsend-ai-triage

    🤖 Finished Triage · ✅ Success · Started 9:39 AM UTC · Completed 9:43 AM UTC

    Commit: c110dc1 · View workflow run →

    Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $0.76

  2. fullsend-ai-triage commented on Sep 28, 2026

    @fullsend-ai-triage

    Related issue #70 tracks drift detection test hygiene (dead branches, misleading test names, fragmented coverage across only 3 of 7 embedded skills). While #70 is not a prerequisite for this documentation change, the implementer should cross-reference both the current test implementation and #70's findings to ensure the documented mapping is complete — particularly since #70 notes that 4 of 7 embedded skills currently lack drift detection tests.

    The proposed example in the issue body uses wildcard patterns (.opencode/agents/*.md → internal/agentkit/content/agents/*.md), but only a subset of scaffolded files have embedded copies. The implementation should list exact pairs derived from the test file.

    ## Validation
    - grep -r 'Drift Detection\|Mirrored Files' AGENTS.md returns the new section
    - Each documented pair matches TestSkillFiles_DriftDetection assertions
    - Section includes MUST synchronization rule and `make check` instruction
    

    Labels: Documentation-only change to AGENTS.md; good first issue reflects the straightforward nature of adding a documentation section.


    Next steps:

    • /fs-code — agent creates a PR to implement this issue
    • /fs-code <your instruction> — agent implements with your specific guidance
  3. fullsend-ai-coder commented on Sep 28, 2026

    @fullsend-ai-coder
    Contributor

    🤖 Finished Code · ✅ Success · Started 9:44 AM UTC · Completed 9:50 AM UTC

    Commit: c110dc1 · View workflow run →

    Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $1.85

  4. added a commit that references this issue on Sep 29, 2026
  5. added 2 commits that reference this issue on Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

documentationImprovements or additions to documentationgood first issueGood for newcomersready-for-triageTriggers triage agent dispatchready-to-codeTriggers code agent dispatch

Type

No type

Projects

Relationships

None yet

Development

No branches or pull requests

Issue actions