Repository navigation
feat: implement issue #1078 — Register agents.solution-architect in canary-rings.json (cross-repo prerequisite for .github-private#1631) #1079
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -1078,6 +1078,71 @@ | |
| } | ||
| } | ||
| } | ||
| }, | ||
| "solution-architect": { | ||
| "_note": "Onboarded per persona-standards.md section 6 step 2 (petry-projects/.github#1078). This persona has NO dedicated reusable: it rides the SHARED, convention-driven persona runtime -- persona-runner-reusable.yml on petry-projects/.github-private, dispatched by the persona-mention router -- which runs any persona that supplies a manifest plus prompts/<role>/advisory.md. The reusable registered here is therefore that shared runner (the reusable that actually executes this persona), and rings[]/gate mirror agents.persona-mention so ring membership and soak knobs stay consistent with the runtime that serves it. CONSEQUENCE 1 (autocut): autocut will NOT fire on this persona's own prompt-only changes -- agent_ref_paths is overridden below to [\"scripts/\",\"personas/\"] (explicitly excluding prompts/) so the global fleet default (which includes prompts/) does not apply here; a prompt-only change therefore never moves either the runner's blob OR the agent-level watched paths (petry-projects/.github-private#1592 -- Script-only changes never reach the fleet). A ring label is therefore NOT evidence of promotion; scripts/persona_reach_check.sh in .github-private is the compensating control that separates routes-to from promoted and fails on the skew state. CONSEQUENCE 2 (fleet precedent): eight other personas are drafted behind this same shared runtime; if each registers against this reusable, several agents end up watching one blob and one change cuts several versions -- decided deliberately here, not by accumulation. NOTE (gate sampling): run_workflow='Persona Runtime' is shared across all personas on this runner; the gate therefore samples all persona runs against the shared runner blob -- this is intentional (the gate tracks runner health, not per-persona prompt health) and is documented here so the deliberate design is not mistaken for an oversight.", | ||
| "host": "petry-projects/.github-private", | ||
| "reusable": ".github/workflows/persona-runner-reusable.yml", | ||
| "run_workflow": "Persona Runtime", | ||
|
Comment on lines
+1085
to
+1086
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Suggestion: Assessment: 🟠 Prompt for AI Agent 🤖This is a comment left during a code review.
**Path:** standards/canary-rings.json
**Line:** 1085:1086
**Comment:**
*Api Mismatch: `Persona Runtime` is shared by multiple personas, but the gate filters only by repository and workflow name, so unrelated persona runs contaminate solution-architect sampling and failure counts.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Acknowledged and documented explicitly. The shared There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ✅ Customized review instruction saved! Instruction:
Applied to:
💡 To manage or update this instruction, visit: CodeAnt AI Settings |
||
| "agent_ref_paths": ["scripts/", "personas/"], | ||
| "rings": [ | ||
| { | ||
| "channel": "next", | ||
| "order": 0, | ||
| "members": [ | ||
| "$host" | ||
| ] | ||
| }, | ||
| { | ||
| "channel": "ring0", | ||
| "order": 1, | ||
| "members": [ | ||
| "$org_infra" | ||
| ] | ||
| }, | ||
|
don-petry marked this conversation as resolved.
|
||
| { | ||
| "channel": "ring1", | ||
| "order": 2, | ||
| "members": [ | ||
| "petry-projects/TalkTerm", | ||
| "petry-projects/bmad-bgreat-suite" | ||
| ] | ||
| }, | ||
| { | ||
| "channel": "stable", | ||
| "order": 3, | ||
| "members": [ | ||
| "*" | ||
| ] | ||
| } | ||
| ], | ||
| "gate": { | ||
| "_standard": "petry-projects/.github#548 — graduated dwell/sample gate over a per-candidate cumulative window. These are registry-configurable per-transition knobs; the values below are the #548 defaults.", | ||
| "baseline_window_days": 14, | ||
| "baseline_spike_cap_multiple": 3, | ||
| "benign_failure_classes": [], | ||
| "control": { | ||
| "allow_pre_existing": false | ||
| }, | ||
| "_benign_note": "benign_failure_classes (#1025 P2): per-reusable allowlist matched against a failed run's workflow name (`workflow` ERE) + failed-step signature (`step` ERE, matched via `gh run view`). Matches are excluded from cum_fail — but ONLY when the candidate's reusable is byte-identical to the prior channel (differs=0), so the allowlist can never mask a candidate-introduced regression. Exception (#668): a class marked `version_independent: true` fails before/independent of the candidate's own code (e.g. a Dependabot-context secrets failure at startup) and can NEVER be candidate-caused, so it is excluded even when differs=1 — reserve the flag for classes where that is provable from the failure mechanism, not merely likely. `control.allow_pre_existing`, or `promote --allow-pre-existing`, advances a BLOCKED+PRE_EXISTING frontier (never REGRESSION).", | ||
| "transitions": { | ||
| "next->ring0": { | ||
| "dwell_hours": 4, | ||
| "sample_fraction_permille": 250, | ||
| "sample_clamp_min": 3, | ||
| "sample_clamp_max": 15, | ||
| "waive_sample_if_no_caller": true | ||
| }, | ||
| "ring0->ring1": { | ||
| "dwell_hours": 8, | ||
| "waive_sample": true | ||
| }, | ||
| "ring1->stable": { | ||
| "dwell_hours": 12, | ||
| "sample_min": 1 | ||
| } | ||
| } | ||
| } | ||
| } | ||
| }, | ||
| "unmanaged": {} | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -862,6 +862,52 @@ GHEOF | |
| [ "$status" -eq 0 ] | ||
| } | ||
|
|
||
| @test "canary-rings.json: solution-architect onboarded on the shared persona runtime (no dedicated reusable, #1078)" { | ||
| # Registered against the SHARED, convention-driven persona runtime — persona-runner-reusable.yml | ||
| # in .github-private (the reusable that actually executes this persona), NOT a per-persona blob. | ||
| run jq -e '.agents["solution-architect"].host == "petry-projects/.github-private"' "$RINGS" | ||
| [ "$status" -eq 0 ] | ||
| run jq -e '.agents["solution-architect"].reusable == ".github/workflows/persona-runner-reusable.yml"' "$RINGS" | ||
| [ "$status" -eq 0 ] | ||
| # run_workflow = the persona-runner.yml caller display name the gate samples on the ring tiers. | ||
| run jq -e '.agents["solution-architect"].run_workflow == "Persona Runtime"' "$RINGS" | ||
|
Comment on lines
+872
to
+873
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Suggestion: The test approves a shared workflow name without checking persona filtering, so unrelated Persona Runtime runs can contaminate solution-architect health decisions while all assertions pass. [api mismatch] Assessment: 🟠 Prompt for AI Agent 🤖This is a comment left during a code review.
**Path:** tests/canary_rollout.bats
**Line:** 872:873
**Comment:**
*Api Mismatch: The test approves a shared workflow name without checking persona filtering, so unrelated Persona Runtime runs can contaminate solution-architect health decisions while all assertions pass.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The test correctly asserts There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ✅ Customized review instruction saved! Instruction:
Applied to:
💡 To manage or update this instruction, visit: CodeAnt AI Settings |
||
| [ "$status" -eq 0 ] | ||
| # standard ring model | ||
| run bash -c "jq -r '.agents[\"solution-architect\"].rings | sort_by(.order) | map(.channel) | join(\",\")' '$RINGS'" | ||
| [ "$output" = "next,ring0,ring1,stable" ] | ||
| # AC #3: next = the dogfood ring ($host = .github-private), matching agents.persona-mention | ||
| run jq -e '.agents["solution-architect"].rings[] | select(.channel=="next") | .members | index("$host")' "$RINGS" | ||
| [ "$status" -eq 0 ] | ||
| # ring0 = org_infra repos ($org_infra), consistent with dev-lead and ci-failure-analyst | ||
| run jq -e '.agents["solution-architect"].rings[] | select(.channel=="ring0") | .members | index("$org_infra")' "$RINGS" | ||
| [ "$status" -eq 0 ] | ||
| # ring1 = the real consumers that gate ring1->stable (consistent with persona-mention) | ||
| run jq -e '.agents["solution-architect"].rings[] | select(.channel=="ring1") | (.members|index("petry-projects/TalkTerm")) and (.members|index("petry-projects/bmad-bgreat-suite"))' "$RINGS" | ||
| [ "$status" -eq 0 ] | ||
| # gate mirrors agents.persona-mention: #548 defaults, standard transition knobs, no opt-in blocks | ||
| run jq -e '.agents["solution-architect"].gate.baseline_window_days == 14 and .agents["solution-architect"].gate.baseline_spike_cap_multiple == 3' "$RINGS" | ||
| [ "$status" -eq 0 ] | ||
| run jq -e '.agents["solution-architect"].gate.transitions["next->ring0"].waive_sample_if_no_caller == true and .agents["solution-architect"].gate.transitions["ring0->ring1"].waive_sample == true and .agents["solution-architect"].gate.transitions["ring1->stable"].sample_min == 1' "$RINGS" | ||
| [ "$status" -eq 0 ] | ||
| run jq -e '.agents["solution-architect"].gate.transitions["ring1->stable"].dwell_hours == 12' "$RINGS" | ||
| [ "$status" -eq 0 ] | ||
| # no opt-in gate machinery (like persona-mention): no correctness, no suspect classes, empty benign list, | ||
| # no synthetic-canary fields | ||
| run jq -e '.agents["solution-architect"].gate | (has("correctness")|not) and (has("suspect_failure_classes")|not) and ((.benign_failure_classes // []) | length == 0)' "$RINGS" | ||
| [ "$status" -eq 0 ] | ||
| run jq -e '.agents["solution-architect"] | (has("next_tier_health_signal")|not) and (has("soak_start_ring")|not)' "$RINGS" | ||
| [ "$status" -eq 0 ] | ||
| # AC #2: inline _note documents (a) no dedicated reusable / shared runtime, and (b) autocut does | ||
| # not fire on prompt-only changes — naming .github-private#1592 and the persona_reach_check.sh control. | ||
| run jq -e '.agents["solution-architect"]._note | test("1592") and test("persona_reach_check.sh") and test("shared"; "i")' "$RINGS" | ||
|
Comment on lines
+900
to
+902
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Suggestion: The test only checks that the note claims prompt changes do not autocut; it never exercises autocut, so it passes while Assessment: 🟠 Prompt for AI Agent 🤖This is a comment left during a code review.
**Path:** tests/canary_rollout.bats
**Line:** 897:899
**Comment:**
*Possible Bug: The test only checks that the note claims prompt changes do not autocut; it never exercises autocut, so it passes while `prompts/` changes still create candidates.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed in tests/canary_rollout.bats: added a behavioral assertion (after line 902) that verifies |
||
| [ "$status" -eq 0 ] | ||
| # AC #2 (behavioral): agent_ref_paths overrides the global fleet default to exclude prompts/ so | ||
| # autocut's change detection never fires on a prompt-only change. scripts/ and personas/ are | ||
| # retained; prompts/ must be absent from the agent-level list. | ||
| run jq -e '.agents["solution-architect"].agent_ref_paths | (index("scripts/") != null) and (index("personas/") != null) and (index("prompts/") == null)' "$RINGS" | ||
| [ "$status" -eq 0 ] | ||
| } | ||
|
|
||
| @test "canary-rings.json: valid JSON + dev-lead host + ordered rings" { | ||
| run jq -e '.agents["dev-lead"].host == "petry-projects/.github-private"' "$RINGS" | ||
| [ "$status" -eq 0 ] | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.