Skip to content

feat(fleet-monitor): raise issues in the repo they pertain to #331

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
don-petry merged 4 commits into from
May 21, 2026
Merged

feat(fleet-monitor): raise issues in the repo they pertain to #331

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6d8c6cc
feat(fleet-monitor): raise issues in the repo they pertain to
don-petry May 21, 2026
1b060a8
fix(fleet-monitor): address Copilot review comments
don-petry May 21, 2026
754caeb
fix(ci): resolve yamllint errors in issue-triage-runner.yml
don-petry May 21, 2026
8c7bf0b
fix(fleet-monitor): address CodeRabbit and ChatGPT Codex review comments
don-petry May 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
219 changes: 148 additions & 71 deletions .github/workflows/actions-fleet-monitor.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,11 @@
# • This is the canonical fleet-monitor implementation; there is no external
# reusable. The scanning logic lives in scripts/fleet_monitor.sh and
# scripts/fleet_report.sh.
# • The "Track high-failure workflows as Issues" step uses github.token (not
# GH_PAT_WORKFLOWS) because github.token has explicit issues:write on this
# repo. Dev-Lead Agent is explicitly woken via repository_dispatch (which
# requires contents:write) using GH_PAT_WORKFLOWS in the step below.
# • The "Migrate misaligned fleet-tracker issues" and "Track high-failure
# workflows as Issues" steps both use GH_PAT_WORKFLOWS (not github.token)
# because they need cross-repo issues:write on target repos.
# github.token only has issues:write on .github-private itself.
# Dev-Lead Agent is woken via repository_dispatch using GH_PAT_WORKFLOWS.
# • Do NOT change checkout ref from 'main' — the github.ref_name variant is
# for feature-branch testing only and must never reach production.
# ─────────────────────────────────────────────────────────────────────────────
Expand Down Expand Up @@ -99,14 +100,70 @@ jobs:
core.notice(`Issue created: ${issue.data.html_url}`);


- name: Migrate misaligned fleet-tracker issues
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
github-token: ${{ secrets.GH_PAT_WORKFLOWS }}
script: |
// Find open fleet-tracker issues in .github-private that belong to other repos
const allIssues = await github.paginate(github.rest.issues.listForRepo, {
owner: context.repo.owner,
repo: context.repo.repo,
labels: 'fleet-tracker',
state: 'open',
per_page: 100,
});

let migrated = 0;
for (const issue of allIssues) {
if (!issue.title.startsWith('[Fleet Monitor] ')) continue;

// Parse: "[Fleet Monitor] owner/repo — workflow.yml"
const match = issue.title.match(/^\[Fleet Monitor\] ([^/]+\/\S+) — (.+)$/);
if (!match) continue;

const [, targetFullRepo] = match;
const [targetOwner, targetRepoName] = targetFullRepo.split('/');

// Already in the right repo — skip
if (targetRepoName === context.repo.repo && targetOwner === context.repo.owner) continue;

core.info(`Migrating #${issue.number} "${issue.title}" → ${targetOwner}/${targetRepoName}`);
try {
const { repository } = await github.graphql(`
query($owner: String!, $name: String!) {
repository(owner: $owner, name: $name) { id }
}
`, { owner: targetOwner, name: targetRepoName });

const result = await github.graphql(`
mutation($issueId: ID!, $repoId: ID!) {
transferIssue(input: {
issueId: $issueId,
repositoryId: $repoId,
createLabelsIfMissing: true
}) {
issue { url number }
}
}
`, { issueId: issue.node_id, repoId: repository.id });

core.notice(`Transferred #${issue.number} → ${result.transferIssue.issue.url}`);
migrated++;
} catch (err) {
core.warning(`Failed to migrate #${issue.number}: ${err.message}`);
}
}
core.info(`Migration complete: ${migrated} issue(s) transferred.`);

- name: Track high-failure workflows as Issues
if: hashFiles('fleet_high_failure.json') != ''
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
env:
ORG: ${{ env.ORG }}
with:
# Use GITHUB_TOKEN (has explicit issues:write) for label creation and assignment
github-token: ${{ github.token }}
# Use GH_PAT_WORKFLOWS for cross-repo issues:write on target repos
github-token: ${{ secrets.GH_PAT_WORKFLOWS }}
script: |
const fs = require('fs');

Expand All @@ -117,77 +174,97 @@ jobs:
return;
}

// Ensure required labels exist (422 = already exists — safe to ignore)
for (const [name, color, desc] of [
['dev-lead', 'd93f0b', 'Requires dev-lead attention'],
['fleet-tracker', '0075ca', 'Tracked by the Actions Fleet Monitor'],
]) {
try {
await github.rest.issues.createLabel({
owner: context.repo.owner, repo: context.repo.repo,
name, color, description: desc,
});
} catch (e) { if (e.status !== 422) throw e; }
}

const labelEmoji = { CRITICAL: '🔴', DEGRADED: '🟠', WARNING: '🟡', ERROR: '⚫' };
const today = new Date().toISOString().split('T')[0];
const runUrl = `${process.env.GITHUB_SERVER_URL}/${process.env.GITHUB_REPOSITORY}/actions/runs/${process.env.GITHUB_RUN_ID}`;

// Track repos where we have already ensured labels exist
const labelledRepos = new Set();

for (const item of items) {
const emoji = labelEmoji[item.label] ?? '🔴';
const title = `[Fleet Monitor] ${item.repo} — ${item.workflow}`;
const body = [
`## ${emoji} ${item.label}: \`${item.repo}\` / \`${item.workflow}\``,
'',
`| Metric | Value |`,
`|---|---|`,
`| **Failure Rate** | ${item.rate} (${item.failed} / ${item.total} runs) |`,
`| **Status** | ${item.label} |`,
`| **p50 Duration** | ${item.p50}s |`,
`| **p95 Duration** | ${item.p95}s |`,
`| **Successful Runs** | ${item.success} |`,
`| **Cancelled Runs** | ${item.cancelled} |`,
'',
`**Workflow:** [\`${item.workflow}\`](https://github.com/${item.repo}/actions)`,
'',
`> _Last updated by [Fleet Monitor run](${runUrl}) on ${today}._`,
`> _Threshold: failure rate > 10% over the monitored window._`,
].join('\n');

// Search for an existing open issue with this exact title
const { data: results } = await github.rest.search.issuesAndPullRequests({
q: [
`repo:${context.repo.owner}/${context.repo.repo}`,
'is:issue',
'is:open',
`in:title`,
`"${title}"`,
].join(' '),
});
const existing = results.items.find(i => i.title === title);

if (existing) {
await github.rest.issues.update({
owner: context.repo.owner, repo: context.repo.repo,
issue_number: existing.number,
body,
});
// Ensure dev-lead label is present (addLabels is idempotent)
await github.rest.issues.addLabels({
owner: context.repo.owner, repo: context.repo.repo,
issue_number: existing.number,
labels: ['dev-lead', 'fleet-tracker', 'health-check'],
});
core.notice(`Updated issue #${existing.number}: ${existing.html_url}`);
} else {
const created = await github.rest.issues.create({
owner: context.repo.owner, repo: context.repo.repo,
title,
body,
labels: ['dev-lead', 'fleet-tracker', 'health-check'],
// Issues are created in the repo they pertain to, not in .github-private
const [targetOwner, targetRepo] = item.repo.split('/');
try {
// Ensure required labels exist in the target repo (once per repo)
if (!labelledRepos.has(item.repo)) {
for (const [name, color, desc] of [
['dev-lead', 'd93f0b', 'Requires dev-lead attention'],
['fleet-tracker', '0075ca', 'Tracked by the Actions Fleet Monitor'],
['health-check', '0e8a16', 'Automated health check result'],
]) {
try {
await github.rest.issues.createLabel({
owner: targetOwner, repo: targetRepo,
name, color, description: desc,
});
} catch (e) {
if (e.status === 422) { /* already exists — ok */ }
else if (e.status === 403 || e.status === 410) {
// 403: no write access; 410: issues disabled on this repo
core.warning(`Cannot create label '${name}' in ${targetOwner}/${targetRepo} (${e.status}) — skipping`);
} else {
throw e; // propagate to outer catch for unexpected errors
}
}
}
labelledRepos.add(item.repo);
}

const emoji = labelEmoji[item.label] ?? '🔴';
const title = `[Fleet Monitor] ${item.repo} — ${item.workflow}`;
const body = [
`## ${emoji} ${item.label}: \`${item.repo}\` / \`${item.workflow}\``,
'',
`| Metric | Value |`,
`|---|---|`,
`| **Failure Rate** | ${item.rate} (${item.failed} / ${item.total} runs) |`,
`| **Status** | ${item.label} |`,
`| **p50 Duration** | ${item.p50}s |`,
`| **p95 Duration** | ${item.p95}s |`,
`| **Successful Runs** | ${item.success} |`,
`| **Cancelled Runs** | ${item.cancelled} |`,
'',
`**Workflow:** [\`${item.workflow}\`](https://github.com/${item.repo}/actions)`,
'',
`> _Last updated by [Fleet Monitor run](${runUrl}) on ${today}._`,
`> _Threshold: failure rate > 10% over the monitored window._`,
].join('\n');

// Search for an existing open issue in the target repo
const { data: results } = await github.rest.search.issuesAndPullRequests({
q: [
`repo:${targetOwner}/${targetRepo}`,
'is:issue',
'is:open',
`in:title`,
`"${title}"`,
].join(' '),
});
core.notice(`Created issue #${created.data.number}: ${created.data.html_url}`);
const existing = results.items.find(i => i.title === title);

if (existing) {
await github.rest.issues.update({
owner: targetOwner, repo: targetRepo,
issue_number: existing.number,
body,
});
await github.rest.issues.addLabels({
owner: targetOwner, repo: targetRepo,
issue_number: existing.number,
labels: ['dev-lead', 'fleet-tracker', 'health-check'],
});
core.notice(`Updated issue #${existing.number}: ${existing.html_url}`);
} else {
const created = await github.rest.issues.create({
owner: targetOwner, repo: targetRepo,
title,
body,
labels: ['dev-lead', 'fleet-tracker', 'health-check'],
});
core.notice(`Created issue #${created.data.number}: ${created.data.html_url}`);
}
} catch (err) {
core.warning(`Failed to track ${item.repo}/${item.workflow}: ${err.message}`);
}
}

Expand Down
29 changes: 12 additions & 17 deletions .github/workflows/issue-triage-runner.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,27 +27,22 @@ jobs:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}

- name: Install Python dependencies
run: pip install "pyyaml==6.0.3" --only-binary :all: --quiet
run: pip install "pyyaml==6.0.3" --only-binary=":all:" --quiet

- name: Run issue-triage (staged)
id: run
env:
ISSUE_NUMBER: ${{ github.event.issue.number }}
ISSUE_TITLE: ${{ github.event.issue.title }}
ISSUE_BODY: ${{ github.event.issue.body }}
ISSUE_LABELS: ${{ join(github.event.issue.labels.*.name, ',') }}
ISSUE_URL: ${{ github.event.issue.html_url }}
ISSUE_ACTION: ${{ github.event.action }}
ISSUE_NUMBER: ${{ github.event.issue.number }}
ISSUE_TITLE: ${{ github.event.issue.title }}
ISSUE_BODY: ${{ github.event.issue.body }}
ISSUE_LABELS: ${{ join(github.event.issue.labels.*.name, ',') }}
ISSUE_URL: ${{ github.event.issue.html_url }}
ISSUE_ACTION: ${{ github.event.action }}
GITHUB_REPOSITORY: ${{ github.repository }}
run: |
set -euo pipefail
result=$(bash scripts/aw.sh run issue-triage --staged)
skip=$(AW_RESULT="$result" python3 - <<'PYEOF'
import json, os
d = json.loads(os.environ['AW_RESULT'])
print('true' if d.get('skip') else 'false')
PYEOF
)
skip=$(AW_RESULT="$result" python3 -c "import json,os; d=json.loads(os.environ['AW_RESULT']); print('true' if d.get('skip') else 'false')")
# Store encoded to survive multi-line JSON in GITHUB_OUTPUT
encoded=$(printf '%s\n' "$result" | base64 -w0)
echo "result=$encoded" >> "$GITHUB_OUTPUT"
Expand All @@ -67,14 +62,14 @@ PYEOF
persist-credentials: false

- name: Install Python dependencies
run: pip install "pyyaml==6.0.3" --only-binary :all: --quiet
run: pip install "pyyaml==6.0.3" --only-binary=":all:" --quiet

- name: Apply triage output (safe-output)
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_REPOSITORY: ${{ github.repository }}
ISSUE_NUMBER: ${{ github.event.issue.number }}
TRIAGE_RESULT: ${{ needs.triage.outputs.result }}
ISSUE_NUMBER: ${{ github.event.issue.number }}
TRIAGE_RESULT: ${{ needs.triage.outputs.result }}
run: |
result_file=$(mktemp)
printf '%s\n' "$TRIAGE_RESULT" | base64 -d > "$result_file"
Expand Down
22 changes: 14 additions & 8 deletions scripts/fleet_monitor.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -177,16 +177,22 @@ done

# ---------------------------------------------------------------------------
# 2b. Build issue lookup for open fleet-tracker issues (linked inline in report)
# Issues now live in their target repo, so search org-wide.
# Note: the Search API caps at 1,000 results total; if the org has >1,000 open
# fleet-tracker issues the lookup may be incomplete, but that is not expected in
# practice. On search failure (auth error, rate-limit, etc.) inline links will
# simply be absent from the report — non-fatal for the monitoring workflow.
# ---------------------------------------------------------------------------
issues_lookup_file=$(mktemp)
if [ -n "${GITHUB_REPOSITORY:-}" ]; then
gh api "repos/${GITHUB_REPOSITORY}/issues?labels=fleet-tracker&state=open&per_page=100" \
--paginate \
--jq '.[] | select(.title | startswith("[Fleet Monitor] ")) |
(.title | ltrimstr("[Fleet Monitor] ") | split(" — ")) as $p |
select(($p | length) == 2) |
[$p[0], $p[1], .html_url, (.number | tostring)] | @tsv' \
> "$issues_lookup_file" 2>/dev/null || true
if ! gh api "search/issues?q=org:${ORG}+label:fleet-tracker+is:open+is:issue&per_page=100" \
--paginate \
--jq '.items[] | select(.title | startswith("[Fleet Monitor] ")) |
(.title | ltrimstr("[Fleet Monitor] ") | split(" — ")) as $p |
select(($p | length) == 2) |
[$p[0], $p[1], .html_url, (.number | tostring)] | @tsv' \
> "$issues_lookup_file" 2>&1; then
echo "::warning::Could not fetch fleet-tracker issues org-wide — inline issue links may be incomplete. Check token auth/rate-limit."
: > "$issues_lookup_file" # ensure file exists but is empty on failure
fi

# ---------------------------------------------------------------------------
Expand Down
Loading