Repository navigation
[Phase 2] Validate the feature-ideation backfill on the real .github-private Ideas backlog (dry-run then live) #937
Copy link
Copy link
Closed
Labels
dev-lead:hands-offExclude from dev-lead agent automationExclude from dev-lead agent automationinitiativeEpic / initiative tracking issueEpic / initiative tracking issueinitiative:holdInitiative driver must not release this issue (human hold)Initiative driver must not release this issue (human hold)
Description
Story
As a maintainer making the backfill go/no-go decision,
I want exercise the feature-ideation backfill mode against .github-private's real open, un-enhanced Ideas backlog — dry-run first, then live — and confirm one structured comment per candidate, idempotent re-runs, and marker continuity,
so that we prove the restored backfill + dry-run capability works on real Discussions (no synthetic fixtures) before idea-enhancer is deprecated (#876), so the cutover is safe.
Acceptance Criteria
- A dry-run dispatch of .github/workflows/feature-ideation.yml with
enhance_backlog=true dry_run=trueover the real .github-private Ideas backlog writes one intended-enhancement entry per open, human-authored, un-enhanced Idea to the JSONL artifact and posts NO comments. - The dry-run artifact's entry count equals an independently enumerated count of candidates (open + human-authored + carrying neither
<!-- idea-enhancer:enhanced -->nor<!-- feature-ideation:enhanced -->). - A subsequent live dispatch (
enhance_backlog=true dry_run=false) posts exactly ONE structured enhancement comment (sharpened problem / context / impact-effort / acceptance criteria) per candidate and writes one canonical idempotency marker on each. - A second live backfill dispatch is a no-op: already-enhanced ideas (including any carrying the legacy
<!-- idea-enhancer:enhanced -->marker) are skipped, and bot-authored Ideas are skipped. - A go/no-go validation note (dry-run artifact link, run links, candidate count) is recorded on this issue; the run is FLAGGED as FAIL (not worked around) if any idea is double-enhanced, or if the marker the live run WRITES is not one the candidate/skip check treats as already-enhanced (i.e. the written marker and the idempotency check are not self-consistent across a re-run).
Tasks / Subtasks
- Dispatch feature-ideation.yml with enhance_backlog=true dry_run=true and capture the intended-enhancement artifact over the live backlog. (AC: test issue from agent #1, Go-live improvements for PR review agent #2)
- Independently enumerate the candidate Discussions and confirm the artifact entry count matches. (AC: Go-live improvements for PR review agent #2)
- Dispatch live (enhance_backlog=true dry_run=false) and verify exactly one structured comment + one canonical marker per candidate. (AC: Add @claude delegation, auto-merge, and rebase handling #3)
- Re-run live and confirm a no-op; confirm legacy-marker and bot-authored ideas are skipped. (AC: Optimize review: small-PR and incremental fast paths #4)
- Record the go/no-go note (artifact + run links + candidate count); flag any double-enhancement or marker mismatch as FAIL. (AC: feat: add Copilot engine support via REVIEW_ENGINE toggle #5)
Dev Notes
- HANDS-OFF: this is a human operational validation/go-no-go gate against live Discussions, not a code change — hence hands_off. It must run AFTER Story 1 (the stub exposes
enhance_backlog) AND after the central reusable's E2/E3 are live on@feature-ideation/next(untracked prerequisites). - Use REAL Discussions only — no synthetic fixtures (same standard as [Phase 2] Validate the folded feature-ideation enhancement mode on real .github-private Discussions (next ring) #875). Enumerate candidates the same way scripts/idea-enhancer/gather-candidates.sh does: open + category Ideas + author is a non-Bot human + no comment containing either marker.
- Marker continuity is the highest-risk check: if the live backfill posts but writes a marker different from the canonical string, or if an idea already carrying
<!-- idea-enhancer:enhanced -->gets re-enhanced, that is a FAIL — flag it and stop; do not proceed to deprecate idea-enhancer ([Phase 3] Deprecate idea-enhancer in .github-private: remove workflow + scripts + test, update the pipeline doc #876). - This complements [Phase 2] Validate the folded feature-ideation enhancement mode on real .github-private Discussions (next ring) #875 (single-idea folded-enhancement validation): [Phase 2] Validate the folded feature-ideation enhancement mode on real .github-private Discussions (next ring) #875 proves the per-discussion path; this story proves the backlog SWEEP + dry-run path. Sequenced after [Phase 2] Validate the folded feature-ideation enhancement mode on real .github-private Discussions (next ring) #875 since both drive the same live next-ring reusable.
- Trigger surface: feature-ideation.yml's
enhance_backlog+dry_runworkflow_dispatch inputs; the reusable uploads the intended mutations to a JSONL artifact on dry-run (same pattern as the legacy idea-enhancer dry-run log).
Project Structure Notes
No repo files change; this is an operational validation of the workflow against live Discussions, recorded as a go/no-go note on the issue.
References
- .github/workflows/feature-ideation.yml#workflow_dispatch-inputs
- scripts/idea-enhancer/gather-candidates.sh (candidate-filtering logic to mirror when enumerating)
- scripts/idea-enhancer/post-enhancement.sh (marker + dry-run-log contract)
- issue [Phase 2] Validate the folded feature-ideation enhancement mode on real .github-private Discussions (next ring) #875 (single-idea validation precedent)
- discussion feature-ideation enhancement backfill + dry-run + marker continuity (prereq to deprecating idea-enhancer) #933 (E2/E3 acceptance criteria)
Likely target surface
.github/workflows/feature-ideation.yml (executed, not edited)
Story prepared by the BMAD Scrum Master (Bob) for epic #934. Status: ready-for-dev.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
dev-lead:hands-offExclude from dev-lead agent automationExclude from dev-lead agent automationinitiativeEpic / initiative tracking issueEpic / initiative tracking issueinitiative:holdInitiative driver must not release this issue (human hold)Initiative driver must not release this issue (human hold)
Activity
enhance_backlog#935don-petry commentedon Jun 27, 2026
GO — backlog backfill PASSED live validation.
Runs:
workflow_dispatch -f enhance_backlog=true, conclusion success.Candidate set (live, 3): open
ideasDiscussions authored by a real human (__typename == User) with no pre-existing marker — #958, #951, #933. (The 2026-06-26 dry-run 28270549407 reported 5; the live set differs as expected: #958 was created later for the #875 test, and the live state is the source of truth.)Results — one canonical comment per candidate:
<!-- feature-ideation:enhanced -->✓<!-- initiative-planner -->note (not a marker), so feature-ideation enhancement backfill + dry-run + marker continuity (prereq to deprecating idea-enhancer) #933 was correctly a candidate; the new comment carries the canonical marker ✓ (its body also mentions the legacy marker string as subject matter, but it is a single enhancement comment, not two markers).Marker continuity / skip-check: all human ideas already carrying the legacy
<!-- idea-enhancer:enhanced -->marker (#851, #866, #877, #906, #578, #631, #177, #427, #332, #743, #572, #593, #607, #283, #650, #666, #656, #735) were SKIPPED — none received a canonical comment. Confirms the legacy marker is recognized by the skip-check. All bot-authored (github-actionsBot) ideas were untouched.Idempotency: second identical run posted ZERO new comments. After both runs, exactly 3 canonical-marked comments exist across all open ideas (= candidate count). No discussion has more than one canonical comment → no double-enhancement.
PASS criteria met: one comment per candidate, canonical marker, second run no-op, no double-enhancement. Closing as completed.