Skip to content

feat(review): interactive generate-tests checkbox in ORB's PR comment, maintainer-only #4589

Description

@JSONbored

Context

Follow-up to #4583 (which added a text-only @gittensory generate-tests mention inline in the review
comment). The maintainer wants a real, clickable checkbox instead — mirroring both CodeRabbit's own
"Generate unit tests" walkthrough checkbox AND gittensory's own pre-existing "Re-run Gittensory review"
checkbox (PR_PANEL_RETRIGGER_MARKER, src/signals/engine.ts + maybeProcessPrPanelRetrigger in
src/queue/processors.ts), which already solves the exact same problem for a different action.

Hard requirement: only maintainers may trigger it — never contributors, never confirmed miners, never
"random people." A markdown task-list checkbox in a GitHub comment can be toggled by anyone who can
comment on the PR (GitHub's own permission model, not gittensory's) — so the checkbox itself cannot be
the enforcement point. The existing retrigger checkbox already solves this correctly: payload.sender
(who actually clicked) is re-verified server-side through the SAME authorization pipeline as the
@gittensory commands on every issue_comment.edited webhook, and an unauthorized click is a silent
no-op
— no comment fetch, no patch, no revert, no explanation. This issue replicates that exact pattern
for generate-tests, hardened to ["maintainer"] only (stricter than review-now's
["maintainer", "collaborator"] retrigger floor, matching generate-tests's existing narrower default).

Technical constraint (confirmed from the existing code)

src/review/unified-comment.ts's own comment on the retrigger checkbox: "GitHub disables interactive
task-list checkboxes inside a blockquote" — the whole comment body (including every existing collapsible
like "Signal definitions") renders inside the alert blockquote (asAlert), and the re-run checkbox is
deliberately appended AFTER it, outside the blockquote, for exactly this reason. A new checkbox must
follow the same placement, appended alongside the existing re-run checkbox — it cannot live inside a
<details> collapsible without becoming inert.

Deliverables

  • A new collapsible section ("Test coverage" or similar), appended after "Signal definitions" in
    buildPublicSafeCollapsibles (src/signals/engine.ts) — informational only (what's missing / why the
    finding fired), reusing the already-computed manifest_missing_tests finding rather than new detection
    logic. Empty body (thus invisible, matching the existing empty-collapsible convention) when there's no
    coverage gap or the feature isn't enabled for the repo.
  • A second checkbox line, alongside "Re-run Gittensory review" but with its own dedicated marker
    (PR_PANEL_GENERATE_TESTS_MARKER), rendered only when e2eTests is enabled for the repo AND the PR
    actually has a manifest_missing_tests gap.
  • A new issue_comment.edited handler (maybeProcessPrPanelGenerateTests, mirroring
    maybeProcessPrPanelRetrigger structurally): detects the marker checked in the bot's own comment,
    resolves payload.sender as the real actor, authorizes through the SAME authorizePrActionActor pipeline
    used by every other @gittensory action command, hardcoded to ["maintainer"] only regardless of what a
    repo's own .gittensory.yml might configure for the text-command version. Authorized → dispatches through
    the EXISTING shared runE2eTestGenerationAndDeliver core (the same one @gittensory generate-tests and
    the manifest_missing_tests auto-trigger already share) with a new trigger: "checkbox" value (behaves
    like "command" for delivery-mode purposes — NOT forced comment-only — since a checkbox click carries a
    real, re-verified maintainer invoker just like the text command does). Unauthorized → silent no-op,
    audit-logged only, exactly matching the retrigger's own denial behavior.
  • Tests mirroring the retrigger's own suite: authorized happy path (checkbox → generation dispatched,
    checkbox resets unchecked on the bot's next re-render), unauthorized actor (silent no-op, zero GitHub
    writes beyond the permission check), bot-sender guard, feature-disabled repo, no coverage gap (checkbox
    never rendered in the first place).

Expected outcome

A maintainer sees ORB's own review comment with a real, clickable "Generate an AI Playwright test for
this PR" checkbox whenever the PR is missing test coverage and the repo has opted into e2eTests — one
click, no typing a slash command required, matching CodeRabbit's own UX exactly — while remaining
provably inert for anyone who isn't a maintainer, using the identical, already-proven-in-production
enforcement mechanism the re-run checkbox already relies on.

Related: #4189 (epic), #4583 (the text-only CTA this supersedes/complements), #4195/#4201 (the
maintainer-only authorization + scoring-integrity precedent this must not weaken).

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.

Projects

Status
Done

Relationships

None yet

Development

No branches or pull requests

Issue actions