Skip to content

feat: implement issue #1649 — [qa-lead S6] Validate skills[] and fix the entries that are wrong - #1690

Merged
don-petry merged 2 commits into
mainfrom
dev-lead/issue-1649-20260907-1507
Sep 7, 2026
Merged

don-petry merged 2 commits into
mainfrom
dev-lead/issue-1649-20260907-1507

Conversation

@don-petry

@don-petry don-petry commented Sep 7, 2026 •

Copy link
Copy Markdown
Collaborator

User description

Closes #1649

Implemented by dev-lead agent. Please review.


CodeAnt-AI Description

Validate persona skill inventories and correct the QA lead skill index

What Changed

  • Persona validation now rejects skill entries whose paths do not exist.
  • Skill entries pointing to SKILL.md must use the matching parent directory as their name, with clear validation errors instead of tracebacks.
  • Added coverage for valid skills, missing paths, empty skill lists, and name/path mismatches.
  • Updated the QA lead inventory to reference the intended eight vendored test-architecture workflows and added three previously missing skills.

Impact

✅ Fewer invalid persona manifests
✅ Clearer skill inventory validation errors
✅ Accurate QA lead workflow coverage

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@codeant-ai

codeant-ai Bot commented Sep 7, 2026 •

Copy link
Copy Markdown

🤖 CodeAnt AI — Review Status

Status Commit Started (UTC) Finished (UTC)
✅ Reviewed your PR acf9e6f Sep 07, 2026 · 15:15 15:17

@qodo-code-review

Copy link
Copy Markdown

ⓘ Qodo reviews are paused because your trial has ended. Ask your workspace admin to add credits to resume reviews. Manage billing

@codeant-ai

codeant-ai Bot commented Sep 7, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@coderabbitai

coderabbitai Bot commented Sep 7, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 22 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Team

Run ID: 3d45ccb7-fa76-405d-9395-cd898537dbe1

📥 Commits

Reviewing files that changed from the base of the PR and between a674a6c and 6bb0ea8.

📒 Files selected for processing (3)
  • personas/qa-lead/persona.yml
  • personas/validate-personas.py
  • tests/test_validate_personas.bats

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codeant-ai codeant-ai Bot added the size:L This PR changes 100-499 lines, ignoring generated files label Sep 7, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces declarative skill inventory validation for personas. It updates the qa-lead persona's skills list, adds a check_skills validation function in validate-personas.py to verify that skill paths exist and that their names match their directories, and adds corresponding BATS tests. The review feedback suggests improving the robustness of the YAML validation in validate-personas.py by defensively checking that skills is a list and that its elements contain string values for name and path to prevent unhandled TypeError tracebacks.

Comment thread personas/validate-personas.py Outdated
Comment thread personas/validate-personas.py Outdated
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — review-changes (applied)

Changes committed and pushed.

@don-petry
don-petry enabled auto-merge (squash) September 7, 2026 15:20
@sonarqubecloud

sonarqubecloud Bot commented Sep 7, 2026

Copy link
Copy Markdown

@donpetry-bot

Copy link
Copy Markdown
Contributor

CI checks on this PR are still running. Once they complete, re-mention @donpetry-bot to trigger a fresh review.

Posted by the donpetry-bot PR-review cascade.

@donpetry-bot

Copy link
Copy Markdown
Contributor

Advisory bots were rate-limited; auto-approval is withheld until they recover. pr-review-sweep will re-review this PR after 2026-09-07T16:27:17Z.

@don-petry
don-petry disabled auto-merge September 7, 2026 15:27
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — fix-bot-comment (no-changes)

Agent reasoning
**Review States:**
- All 3 reviews: `state: "COMMENTED"` (not `CHANGES_REQUESTED`)
- gemini-code-assist[bot]'s comment describes the PR but doesn't block it
## Conclusion
There are **zero Tier 1 blockers**:
- No failing CI checks
- No `CHANGES_REQUESTED` reviews
- No specific, actionable defects reported
Per the task instructions: *"A neutral overview is not an actionable finding...there is nothing to fix — do not revert or undo the PR's own changes to 'address' it."*
**No changes needed.** The PR is in a passing state with a clean quality gate. The SonarCloud comment is merely reporting that the PR meets all quality standards.

@don-petry
don-petry enabled auto-merge (squash) September 7, 2026 15:28

@donpetry-bot donpetry-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review — APPROVED ✓

Risk: MEDIUM
Reviewed commit: 6bb0ea8d90ab2aa497338832cc93cb39d0203136
Review mode: triage-approved (single reviewer)

Summary

Implements issue #1649: adds skills[] validation to personas/validate-personas.py (path existence, repo-root containment, name↔SKILL.md-directory agreement, defensive shape checks), corrects the mislabelled bmad-testarch-test-design entry in personas/qa-lead/persona.yml, adds three previously missing skills (atdd, automate, framework), and documents skills[] as a declarative inventory not consumed at runtime. Eight new hermetic bats tests cover the valid, missing-path, empty-list, name-mismatch, absolute-path, ..-traversal, non-list, and non-string-field cases.

Linked issue analysis

All seven acceptance criteria of #1649 are substantively addressed:

  1. ✅ skills[].path existence check with a diagnostic naming manifest, index, and path (mirrors the definition.layers[].path message shape).
  2. ✅ Bats coverage: valid entry passes, missing path fails, absent/empty skills[] passes.
  3. ✅ Name↔path agreement enforced generally (<name>/SKILL.md directory must equal name) — no BMAD layout hardcoded.
  4. ✅ bmad-testarch-test-design now points at src/workflows/testarch/bmad-testarch-test-design/SKILL.md.
  5. ✅ Declared skills reconciled with declared expertise: 8 of 9 vendored testarch workflows declared (verified against the vendored tree at this SHA); bmad-teach-me-testing deliberately excluded with in-manifest justification (learner tutorial, not advisory expertise).
  6. ✅ The check runs for every persona via check_invariants; the validate-personas CI check is green at this SHA, so all nine manifests pass.
  7. ✅ Decision documented in the manifest: skills[] is a declarative index-of-record, not a runtime input, with rationale for not wiring it into the advisory prompt (context-size cost, duplication).

Findings

  • No blocking findings.
  • Security posture is improved, not degraded: the new containment check rejects absolute and ..-escaping skill paths (with try/except (ValueError, OSError)), preventing unrelated filesystem objects from satisfying validation. Covered by tests.
  • No traceback risk from the defensive shape checks: fail() is NoReturn and exits immediately, so malformed entries produce clean diagnostics.
  • Both prior bot review threads (gemini-code-assist type-safety, codeant-ai path-escape) were fixed in commit 6bb0ea8 and are resolved.
  • Vendored frameworks/ tree untouched, validator remains hermetic (filesystem-only, no network) — per the issue's dev notes.
  • Secret scan: run_secret_scanning MCP tool unavailable in this environment; gitleaks CI check is green and the diff contains no credential-like content.

CI status

All required checks green at 6bb0ea8d90ab2aa497338832cc93cb39d0203136: unit-tests, bats, validate-personas, Lint, ShellCheck/shellcheck, actionlint, CodeQL (actions + python), SonarCloud quality gate, Secret scan (gitleaks), agent-shield, holdout-guard, template-drift, caller-stub-freeze, and all persona/contract validators. Skipped checks are ecosystem-not-present dependency audits and dependabot automerge — expected.


Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.

@don-petry
don-petry merged commit e64bae7 into main Sep 7, 2026
45 checks passed
@don-petry
don-petry deleted the dev-lead/issue-1649-20260907-1507 branch September 7, 2026 15:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L This PR changes 100-499 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[qa-lead S6] Validate skills[] and fix the entries that are wrong

2 participants