fix(knowledge): demote youtube-digest variation-matrix-backlog out of evals/fixtures

[kyle-sexton]

Summary

plugins/knowledge/skills/youtube-digest/evals/fixtures/variation-matrix-backlog.json sat under a skill's evals/fixtures/ but no eval case referenced it (no files[] entry in the sibling evals.json) and no test asserted on it. It was grandfathered in scripts/orphaned-fixtures-baseline.txt — the last un-owned entry there (the autonomy otlp-demo fixtures are tracked by #662).

Fix

Decision: demote (not grade). The file is a manual smoke-test tracking backlog — candidate videos across footage variations (code screencast / slide talk / talking-head / mixed) carrying status fields (smoke-pass), acquisition notes, and blocked-caption records. It is reference/tracking data, not an input→expected-output graded fixture (the genuine graded fixture in this skill is driver-video-goldens.json, wired via files[] in evals 2 and 4). SKILL.md itself labeled it "backlog only".

• Moved evals/fixtures/variation-matrix-backlog.json → reference/variation-matrix-backlog.json (out of the graded-fixture scope, into the skill's reference/ per the issue's steer).
• Repointed the two prose references at the new path: SKILL.md (now describes it as a tracking backlog, not an eval fixture) and vendor TUNING.md.
• Removed its grandfather line (and its comment block) from scripts/orphaned-fixtures-baseline.txt.
• Bumped plugins/knowledge 0.8.0 → 0.8.1 (patch: relocation + docs, no behavior change) with a top-inserted CHANGELOG.md entry.

Verification

Ran on the rebased branch tip:

$ scripts/check-orphaned-fixtures.sh --check
No orphaned eval fixtures (every file under **/evals/fixtures/ is consumed by a grader or grandfathered).
# exit 0

$ scripts/check-orphaned-fixtures.sh discover | grep variation-matrix
# (no output — file no longer under evals/fixtures/)

$ scripts/check-changelog-parity.sh --check
Every versioned plugin has a CHANGELOG.md (or a stale-guarded baseline entry).
# exit 0

$ scripts/check-changelog-parity.sh --check-bump origin/main
Every plugin whose version changed vs origin/main has a '## [<version>]' CHANGELOG.md entry.
# exit 0

$ node scripts/validate-plugin-contracts.mjs
Plugin contracts validated: 33 setup skills and 1816 plugin files checked.
# exit 0

$ jq empty reference/variation-matrix-backlog.json   # relocated file still valid JSON
# exit 0

Closes #688

Related

• knowledge youtube-digest: variation-matrix-backlog.json eval fixture is orphaned — grade or demote #688 — this issue (grade-or-demote the orphaned fixture)
• ci: orphaned-fixture gate + CHANGELOG-parity gate (every fixtures/evals file consumed by a grader; every plugin.json bump requires a CHANGELOG entry) #663 — orphaned-fixture CI gate that surfaced it
• feat(ci): orphaned-fixture + CHANGELOG-parity gates (#663) #681 — PR that introduced the gate + baseline

[claude]

Claude finished @kyle-sexton's task in 2m 26s —— View job

---

PR Review — fix(knowledge): demote youtube-digest variation-matrix-backlog out of evals/fixtures

• Read PR description and understand the change
• Review diff and changed files
• Check alignment with CLAUDE.md guidelines
• Verify CHANGELOG and version bump correctness
• Post review findings

---

Verdict: LGTM — the demote decision is well-justified and the mechanical changes are clean. One minor nit on path anchoring worth noting.

---

What changed

File	Change
evals/fixtures/variation-matrix-backlog.json → reference/variation-matrix-backlog.json	Pure rename, 100% similarity, no content changed
SKILL.md	Removed the fixture table row; replaced with inline prose note pointing to new path
vendor/video-digestion/TUNING.md	Updated path reference
scripts/orphaned-fixtures-baseline.txt	Removed 5-line comment block + 1 fixture path (7 lines total including trailing blank)
.claude-plugin/plugin.json	0.8.0 → 0.8.1 (patch bump)
CHANGELOG.md	Top-inserted [0.8.1] entry

---

Correctness

Demote is the right call. Confirmed via evals/evals.json: no eval case has a files[] entry for variation-matrix-backlog.json. The graded fixture driver-video-goldens.json stays wired in evals 2 and 4 and is undisturbed. The moved file is demonstrably tracking data (status fields, smoke notes, blocked-caption records, alternate_blocked entries), not an input→expected-output pair.

reference/ is the correct destination. The pattern is well-established: 30+ reference/ directories exist across plugins, including plugins/knowledge/skills/course-digest/reference/. No new convention is being introduced.

Baseline cleanup is correct. The 7 removed lines are exactly the comment block + fixture path for the now-relocated file. The remaining autonomy entries (tracked by #662) are untouched.

Version bump is appropriate. Relocation + docs with no behavior change → patch-level. Consistent with how 0.7.x bumps are handled in this changelog.

---

Nit — TUNING.md path anchor is inconsistent with SKILL.md

vendor/video-digestion/TUNING.md line 5:

-Variation smokes in `evals/fixtures/variation-matrix-backlog.json`.
+Variation smokes in `skills/youtube-digest/reference/variation-matrix-backlog.json`.

SKILL.md line 398 uses a skill-root-relative path:

reference/variation-matrix-backlog.json

TUNING.md now uses a plugin-root-relative path (plugins/knowledge/ as the anchor). The old TUNING.md path was also skill-root-relative, and TUNING.md lives two levels away in vendor/video-digestion/ — so both forms require a mental context switch anyway. The new path is navigable and unambiguous from the plugin root. This is a minor inconsistency inherited from the pre-existing style, not introduced by this PR. Not worth blocking on.

---

Summary

Change is mechanically complete and correctly scoped. All cross-references updated, baseline entry cleaned, version and changelog correct. No behavioral change, no security surface, no orphan created. Ready to merge.

[kyle-sexton]

Post-green review pass complete (rule 8). Bot review: LGTM, demote decision confirmed correct (verified against evals.json myself: no files[] reference to the fixture), reference/ destination matches 30+ existing precedents in this repo. One nit flagged (TUNING.md path-anchor style inconsistency, pre-existing pattern not introduced by this PR) — non-blocking, not fixing.

No threads to resolve, CI fully green (17/17).

Flipping to ready for review; handing off to the babysit lane for merge.

[claude]

Claude finished @kyle-sexton's task in 2m 22s —— View job

---

PR Review — fix(knowledge): demote youtube-digest variation-matrix-backlog out of evals/fixtures

• Read PR description and understand the change
• Review diff and changed files
• Verify demote decision against evals.json
• Check CHANGELOG and version bump correctness
• Check cross-reference updates (SKILL.md, TUNING.md)
• Verify baseline cleanup

---

Verdict: LGTM — demote decision is correct, all six files are mechanically clean, and no regressions introduced. One pre-existing nit on the baseline comment and one inherited path-anchor inconsistency (both carried from the first review pass), neither blocking.

---

Demote decision — confirmed correct

Verified against evals/evals.json: the only files[] entries across all nine eval cases reference evals/fixtures/driver-video-goldens.json (evals 2, 4, and 5). No case references variation-matrix-backlog.json. The file is demonstrably tracking data — status fields, smoke_pass notes, alternate_blocked records — not an input→expected-output pair.

The reference/ destination is correct: the relocated file is the sole entry in reference/ for this skill, consistent with the pattern used across 30+ reference/ directories in the repo.

---

Changed files

File	Assessment
plugin.json 0.8.0 → 0.8.1	Correct patch-level bump for relocation + docs, no behavior change
CHANGELOG.md	Clean top-insert of [0.8.1] entry; accurately describes the demote
SKILL.md	Table row removed; prose note added with correct new path — remaining table entries (evals.json, driver-video-goldens.json) are both eval-wired and untouched
reference/variation-matrix-backlog.json	Pure rename (0 content changes), valid JSON confirmed
vendor/video-digestion/TUNING.md	Path updated; see nit below
scripts/orphaned-fixtures-baseline.txt	Exactly 7 lines removed: 1 blank + 5-line comment block + 1 fixture path; autonomy entries untouched

---

Nit 1 — TUNING.md path anchor (carried from first review, non-blocking)

vendor/video-digestion/TUNING.md L5 now uses a plugin-root-relative path (skills/youtube-digest/reference/…), while SKILL.md L398 uses a skill-root-relative path (reference/…). The old TUNING.md path was also skill-root-relative — the style inconsistency is pre-existing and wasn't introduced here. Both forms are navigable and unambiguous. Not worth blocking on.

Nit 2 — baseline comment starts mid-sentence (pre-existing, non-blocking)

scripts/orphaned-fixtures-baseline.txt L15 opens with # eval files[] entry and graded by no test) plus the otlp-demo sample corpus, — a sentence fragment, suggesting the preceding line(s) of this comment were removed in an earlier edit. The PR's diff only touches lines 25–31 (after the otlp-demo path entries) so this was already present before this change. Minor maintenance debt, not introduced here.

---

Summary

All cross-references updated, baseline entry cleanly removed, version and changelog correct. No behavioral change, no new orphan created, no security surface introduced. The orphaned-fixtures gate now passes clean with no grandfather entry remaining for the knowledge plugin. Ready to merge.