Skip to content

feat(skills): add dev-desk-check and qa-review browser QA foundation skills - #31

Draft
vitor-rangel-vinta wants to merge 14 commits into
vintasoftware:mainfrom
vitor-rangel-vinta:feat/dev-desk-check-qa-review-skills
Draft

vitor-rangel-vinta wants to merge 14 commits into
vintasoftware:mainfrom
vitor-rangel-vinta:feat/dev-desk-check-qa-review-skills

Conversation

@vitor-rangel-vinta

@vitor-rangel-vinta vitor-rangel-vinta commented Sep 10, 2026 •

Copy link
Copy Markdown

What this adds

Two new opt-in, template-rendered foundation skills for browser QA, plus the config contract they read.

dev-desk-check — the engineer path

Drives a real browser through the running app the way a user would, judges what actually renders, and leaves evidence someone who wasn't in the session can trust. The manual counterpart to add-e2e-test's automation, not a substitute.

  • Plan source is the branch diff (against the PR's real base, looked up per code host — "no PR" and "the CLI failed" are different answers, and only the first falls back to the default branch), ids from the project's QA use-cases doc, or a ticket.
  • Preflights services / frontend / backend per environment before opening the browser.
  • Enforces the project's per-environment write policy and data-sensitivity rules as a gate, not a suggestion.
  • Keeps an incremental write ledger with a reversal per entry, and writes an evidence-backed report plus a PR Q.A. section.
  • Deliberately not wired into implement-plan's phase gate — a browser pass per phase is too slow, the same reasoning that keeps run_options.run_e2e off.

qa-review — the portable path

The same browser walk and the same report shape for product managers and designers, with git, diffs, dev servers, service probes, PR edits and database access stripped out. Rendered self-contained: zero relative links, zero repo-dependent commands, no engineer jargon — so an engineer can hand over the single SKILL.md and it works for someone with no checkout, no terminal and no repo.

Enforced at derive time by a portability check in vinta-derive-skills, whose jargon term set is deliberately wider than the template-only check, because jargon can arrive through a rendered block even when the template itself is clean.

One shared config block

Both skills render from skills.dev-desk-check and are gated by independent foundation_skills entries — a project can take the engineer path alone, the review path alone, or both.

Guard rails are data, not prose: a project declares its policy per environment and the rendered skill enforces exactly what it was given.

Guard rails baked into the contract

  • No credentials in config, ever. auth.accounts[].where is a pointer to where a secret lives, never the secret.
  • auth.login is a single-valued enum (manual) on purpose — the schema gives a project no way to express an automated login. The agent never types a password, in any environment.
  • PHI never gets transcribed. Under data_sensitivity: phi the report and the write ledger carry opaque ids only, and phi blocks the shareable artifact deliverable outright.
  • Setup writes stay propose-only in every environment, including writes: free.
  • Anything whose effects reach beyond the one thing being tested is proposed and left for an engineer, everywhere.

Files

File Change
schemas/vinta-ai-workflows-config.v1.schema.json foundation_skills.dev-desk-check + .qa-review, run_options.dev-desk-check, skills.dev-desk-check
skills/vinta-derive-skills/resources/dev-desk-check-template.md new — engineer path
skills/vinta-derive-skills/resources/qa-review-template.md new — portable path
skills/vinta-derive-skills/SKILL.md buckets, 16 placeholder rows, render outputs, derive-time portability check
skills/vinta-bootstrap-ai-tools/SKILL.md interview questions 8 and 9, shared follow-ups, emitted YAML
skills/vinta-analyze-codebase/SKILL.md foundation-shape classifier list
skills/vinta-update-project-skills/SKILL.md qa-review re-renders on a config change, not just a template change
README.md, CHANGELOG.md cheat-sheet rows, gating note, 0.7.0 section

package.json is untouched — dev-skills/release owns the version bump. The CHANGELOG heading carries a literal YYYY-MM-DD placeholder for the same reason.

Notes for review

  • The skill was named qa-frontend through most of its development; the last commit renames it to dev-desk-check across every identifier (enum entry, config block, template file, frontmatter, H1, report line). The {{QA_*}} placeholder names are intentionally unchanged — they are shared with qa-review and name the domain, not the skill.
  • Both skills need a browser skill or MCP in the harness and refuse to run without one. They ship no browser tooling of their own.

Review status

A whole-branch review returned three findings, all fixed in the last commit and confirmed resolved by a scoped re-review:

  • Dangling links (blocker). The intro linked add-e2e-test and create-qa-use-cases unconditionally. Both are independent opt-ins, so in a project that enabled neither, the rendered skill shipped two links to directories that do not exist — and the placeholder contract in this same package already makes exactly those two links conditional. The intro now names both skills without linking them; the conditional links stay where they belong, inside {{QA_E2E_IMPACT_BLOCK}} and {{QA_USE_CASES_BLOCK}}. The template now ships zero markdown links, which puts it under the same grep that guards qa-review.
  • An unsatisfiable verification (must-fix). ## Verification required the report to state the viewport used, but the report skeleton had no field for it. Added **Viewport** to the skeleton header, matching the phase instruction that already asks for it.
  • A plan-only reference (should-fix). The derive-time portability check cited "Task 3 Step 10", numbering that exists only in an untracked planning document. It describes the check instead.

Verified clean by the review: all placeholders match 1:1 between both templates and the placeholder table in both directions; the bootstrap interview's example YAML validates against the schema field-by-field, including additionalProperties: false; no write-gate rule is shipped twice (once as prose, once via a rendered block); and none of the four guard rails above has a bypass reachable from a schema-valid config.

Known-minor, deliberately deferred

  • The --writes=allow flag wording is self-cancelling. Traced: it never actually relaxes anything — confirmation is unconditional under writes: confirm and the flag is refused under forbidden — so this is a dead flag, not a guard-rail hole.
  • One verification step hardcodes "an environment named local" instead of deriving from the declared environments. Schema and template agree word-for-word, so it is a documented limitation rather than a contradiction.
  • The bootstrap frontend_start follow-up states the mechanism but not the cost of leaving it blank.

Dogfooding pass — rendered into a real project

Rendered dev-desk-check into altlegal/com (TypeScript/React frontend that already carries a hand-written qa-frontend skill) from a schema-valid config, with services and design_source omitted so both conditional-strip paths were exercised. Three more defects surfaced, all confirmed in the rendered output and fixed here:

  • Phase 3 advertised a stripped step. With no services declared the ### Services pre-check section vanished correctly, but the fixed prose above it still announced services → frontend → backend → data preconditions and justified the ordering with a services-only rationale. The frontmatter description: over-promised the same way. Services ordering and its rationale now live in {{QA_SERVICES_BLOCK}}, which vanishes with them — same class as the dangling-link blocker above, one region further up.
  • De-linking the intro fixed the symptom, not the cause. Dropping the two hyperlinks removed the broken paths but left the assertions: a project with both skills disabled still shipped an intro claiming create-qa-use-cases "writes flows in user language". The framing now describes the situation instead of naming sibling skills; the conditional blocks still name them where that is accurate by construction.
  • {{QA_PREFLIGHT_BLOCK}}'s contract contradicted itself. It told the renderer both "No curl preflight — the browser is the preflight" (when edge_proxy is set) and "Probe <base_url> and stop" (when frontend_start is absent) — and those are the same environment for every remote environment a normal project declares. The frontend probe was also never specified, only the backend one, which is where the conflict hid. edge_proxy now explicitly shapes the whole entry and overrides every probe instruction below it; frontend_start governs starting, not probing.

Left open, worth a decision rather than a quiet fix:

  • The <!-- rendering note: --> comments have no documented disposition. Nothing says to strip them, and an unstripped one would put skills.dev-desk-check.feature_flags into a rendered qa-review — which the portability check greps for.
  • environments.<name>.base_url is required with no way to say "supplied per run". com's Azure pre-production slot has no fixed URL, and Step 0 tells the agent never to open a URL that is not in the table.
  • skills.dev-desk-check has no free-form escape hatch. skills.write-unit-test.additional_conventions[] already solves this shape; without it, repo-specific knowledge (a changed-path → reachable-surface map, a feature-flag discovery grep) cannot reach the render, which is most of what the hand-written skill has and this one does not.

Still draft

Left as a draft for your call on the deferred items above and on the 0.7.0 release date, which is a literal YYYY-MM-DD placeholder until dev-skills/release runs.

…rails

- write-gate rule 6: restore the spec's second sentence, so the write ledger
  itself is told to carry opaque IDs rather than field values under phi
- escalation section: distinguish the agent flipping a flag itself from
  proposing the command and waiting, so it agrees with rule 7's propose-and-wait
- dev-server pitfall: starting anything is local-only and asks first
- screenshot resolution: captures land under the report dir, required (not
  conventional) under client or phi
…on and share-scope qualifiers to qa-review template
Renames the engineer-facing browser QA skill and every identifier that
names it: the foundation_skills enum entry, run_options.qa-frontend, the
shared skills.qa-frontend config block that qa-review also renders from,
and the template file itself. The {{QA_*}} placeholder names are
unchanged - they are shared with qa-review and describe the domain, not
the skill.

Adds "desk check this branch" to the template's trigger phrases so the
new name is discoverable.
… plan-only reference

Final review findings:

- The dev-desk-check intro linked ../add-e2e-test/SKILL.md and
  ../create-qa-use-cases/SKILL.md unconditionally. Both are independent
  opt-ins, so the links dangle in any project that did not enable them.
  The conditional links already live where they belong, inside
  {{QA_E2E_IMPACT_BLOCK}} and {{QA_USE_CASES_BLOCK}}; the intro now names
  the skills without linking them. The template ships zero links.
- Verification demanded the report state the viewport, but the Phase 8
  report skeleton had no field for it. Added Viewport to the header line,
  matching the Phase 5 instruction to say which viewport the run used.
- The derive-time portability check cited "Task 3 Step 10", a numbering
  that exists only in the planning document and has no referent in the
  shipped package. Describes the check instead.
…preflight conflict

Rendering the skill into a real project surfaced three defects the branch
review missed:

- Phase 3's fixed 'Fixed order' line announced a services step and gave a
  services-only rationale for the ordering, so a project declaring no
  services shipped a phase advertising a step it would never run. The
  frontmatter description over-promised the same way. Services ordering and
  its rationale move into QA_SERVICES_BLOCK, which vanishes with them.

- De-linking the intro's two conditional-skill pointers removed the broken
  paths but left the broken references. The framing now describes the
  situation rather than naming add-e2e-test and create-qa-use-cases; the
  conditional blocks still name them where that is accurate.

- QA_PREFLIGHT_BLOCK's contract told the renderer both 'no curl preflight'
  and 'probe base_url and stop' for any environment with edge_proxy set and
  frontend_start absent — every remote environment. edge_proxy now
  explicitly shapes the whole entry, the frontend probe is specified for the
  non-proxied case, and frontend_start governs starting rather than probing.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant