Skip to content

feat(skills): ship orca-cli as a first-generation hybrid stub - #9238

Merged
brennanb2025 merged 4 commits into
mainfrom
brennanb2025/orca-cli-hybrid-stub
Jul 20, 2026
Merged

brennanb2025 merged 4 commits into
mainfrom
brennanb2025/orca-cli-hybrid-stub

Conversation

@brennanb2025

@brennanb2025 brennanb2025 commented Jul 17, 2026 •

Copy link
Copy Markdown
Contributor

What & why

Converts the orca-cli skill's installable SKILL.md from a full "fat" guide into a first-generation hybrid discovery stub, per notes/skill-freshness-design.md §B/§E.4.

The version-sensitive command reference now lives only in the Orca binary (the embedded guide table, served by orca skills get orca-cli). The distributed skill file becomes a thin, safe pointer, so it can no longer drift from the binary that will actually run the commands — the root cause the thin-stub plan targets.

Only orca-cli converts here; the other 7 skills stay fat.

What changed

  • skill-stubs/orca-cli.md (new) — body-only hybrid stub: discovery-stub notice, the safe CLI resolver (ORCA_CLI_COMMAND → orca-dev → orca-ide → orca, never bare orca on Linux/GNOME screen reader), the ORCA skills get orca-cli pointer, a "don't guess subcommands" warning, and a bounded read-only fallback for pre-guide binaries (so old Orca doesn't dead-end or invite guessing).
  • config/scripts/generate-bundled-skill-guides.mjs — the generator now projects STUB_TOPICS from skill-stubs/<name>.md, composing projection = <guide frontmatter block> + <stub body>. The guide's frontmatter (name + description) is reused verbatim, so the routing/description surface is byte-identical. The embedded full guide (bundled-skill-guides.ts) is untouched.
  • skills/orca-cli/SKILL.md — regenerated: 331 → 78 lines.
  • resources/skills/current-manifest.json + snapshot-registry.json — regenerated: orca-cli releaseRevision 32 → 33 as an append-only snapshot; new digest/size. release-mapping.json is byte-identical. Existing fat installs now classify outdated and get the targeted npx skills update nudge — no in-app writes.
  • .gitattributes — LF pin for /skill-stubs/*.md.
  • Tests — orca-cli-skill-guidance.test.mjs: command-guidance assertions repointed to the guide source (their home now the projection is thin) + new stub safety coverage (resolver survives, no bare orca, old-binary fallback, frontmatter unchanged). generate-bundled-skill-guides.test.mjs: stub-projection coverage; fixtures copy/CRLF skill-stubs.

Verification

  • ✅ verify:bundled-skill-guides + verify:skill-bundle-manifest gates pass.
  • ✅ Unit tests green: bundled-guides generator, manifest generator (incl. git-tree identity), orca-cli-skill-guidance, src/cli/skills.test.ts.
  • ✅ oxlint clean.
  • ✅ Live CLI: orca skills get orca-cli still prints the full guide from the embedded table; skills list shows orca-cli with its unchanged description; the on-disk file is the 73-line hybrid stub.

Not in scope / gates still ahead (do not merge past these)

Per skill-freshness-design.md:

  • §E.3 pointer-compliance spike — install this stub in a throwaway home and run real agents (Claude Code, Codex) against the released guide-serving binary; measure how reliably they resolve the correct CLI + fetch the guide before their first Orca command, task success vs. fat, old-binary behavior, and token cost. This is the intended pre-merge gate.
  • §E.5 RC window — only thin further / convert remaining skills after an RC validation window passes; otherwise retain the hybrid.
  • allowed-tools frontmatter — intentionally not added (frontmatter kept byte-identical to the guide). The plan sketches it; validating its exact form is better done in the spike than guessed here.

Manual validation (pointer-compliance spike, demo-project)

Summary / result

Verdict: pass — hybrid stub is effective for real agents on this spike.

Dimension Finding
Overall Both Claude Code and Codex completed the Orca CLI task successfully with the thin stub + production orca skills get
Skill invoke Effective: Claude used Skill tool (Skill(orca-cli)); Codex loaded installed SKILL.md when asked
Stub recognition Both correctly treated the file as a discovery stub, not a full command guide
CLI resolution Both picked plain orca on macOS with no env overrides (no wrong binary / no fallthrough)
Guide-before-use Both ran orca skills get orca-cli before status / worktree / terminal
Task success Read-only ops all succeeded (status, worktree ps, terminal list; Claude also worktree current)
Guessing risk No invented subcommands from memory; no pre-guide fallback path needed
Friction Only note: large worktree ps JSON (display truncation in TUI); not a stub failure
Environment demo-project worktrees only; stub temporarily installed then restored; validation worktrees removed

Not measured in this run: old-binary (skills get unknown) failure path, token cost vs fat skill, Linux/WSL/SSH resolver paths, or a head-to-head fat-skill control.

Ran real Claude Code and Codex against the hybrid stub install + production guide-serving orca binary. Worktrees lived only under demo-project (not the orca monorepo) and were removed after the run.

Setup:

  • Installed hybrid stub at ~/.agents/skills/orca-cli/SKILL.md (78-line discovery stub; fat skill backed up and restored after)
  • Guide binary: production orca (orca skills get orca-cli → full ~331-line version-matched guide)
  • Agents launched via orca worktree create --repo demo-project --agent {claude|codex} --prompt … --setup skip --no-parent

Claude Code (Opus 4.6)

Check Result
Explicit skill invoke Yes — Skill(orca-cli) / Skill tool (“Successfully loaded skill”)
Recognized form Thin discovery stub (not full guide)
Executable resolution Plain orca (macOS; ORCA_CLI_COMMAND / ORCA_DEV_REPO_ROOT unset)
Guide before first Orca cmd Yes — orca skills get orca-cli first
Task success status, worktree ps, worktree current, terminal list --worktree active all OK
Friction None (no guessing, wrong binary, or pre-guide fallback)

02-claude-hybrid-stub.png

Codex (gpt-5.6-sol high)

Check Result
Explicit skill load Yes — read installed ~/.agents/skills/orca-cli/SKILL.md
Recognized form Thin discovery stub
Executable resolution Bare orca (Darwin; no env overrides)
Guide before first Orca cmd Yes — orca skills get orca-cli was first Orca invocation
Task success status, worktree ps, terminal list --worktree active all OK
Friction Large worktree ps display only; no missing commands / wrong binary

01-codex-hybrid-stub.png

Notes

  • Both agents correctly treated the skill as a pointer, fetched the binary guide, then ran read-only Orca CLI commands without inventing subcommands from memory.
  • Explicit skill invocation remains effective: Claude via Skill tool; Codex via reading the installed skill file when asked to load $orca-cli.
  • Cleanup: demo-project validation worktrees removed; global skill install restored to the prior fat snapshot after the run.

Convert the installable orca-cli SKILL.md from a full fat guide into a
hybrid discovery stub: a safe CLI resolver, an `orca skills get orca-cli`
pointer, and a bounded read-only fallback for pre-guide binaries. The
version-matched command reference now lives only in the Orca binary
(embedded guide table, served by `orca skills get`), so the distributed
file can no longer drift from the binary that runs the commands.

- generator projects STUB_TOPICS from skill-stubs/<name>.md, reusing the
  guide's own frontmatter so the routing/description surface is unchanged;
  the embedded full guide (bundled-skill-guides.ts) is untouched.
- manifest regenerated: orca-cli releaseRevision 32->33 as an append-only
  snapshot; existing fat installs classify `outdated` and get the targeted
  `npx skills update` nudge (no in-app writes).
- tests: command-guidance assertions repointed to the guide source (their
  home now), plus stub-projection + safety coverage.

Only orca-cli converts; the other skills stay fat. Per
notes/skill-freshness-design.md, the E.3 pointer-compliance spike and the
E.5 RC window remain before any further thinning. allowed-tools is
intentionally not added yet (frontmatter kept byte-identical to the guide).
@coderabbitai

coderabbitai Bot commented Jul 17, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: dbeac594-bc9e-41bc-a252-4d3134216827

📥 Commits

Reviewing files that changed from the base of the PR and between 426bae2 and 07ee988.

📒 Files selected for processing (6)
  • config/scripts/orca-cli-skill-guidance.test.mjs
  • resources/skills/current-manifest.json
  • resources/skills/release-mapping.json
  • resources/skills/snapshot-registry.json
  • skill-stubs/orca-cli.md
  • skills/orca-cli/SKILL.md
🚧 Files skipped from review as they are similar to previous changes (4)
  • config/scripts/orca-cli-skill-guidance.test.mjs
  • skill-stubs/orca-cli.md
  • resources/skills/snapshot-registry.json
  • skills/orca-cli/SKILL.md

📝 Walkthrough

Walkthrough

The change adds an Orca CLI discovery stub and updates bundled skill generation to preserve canonical frontmatter while substituting stub content. Stub topics are validated, normalized, and covered by projection and CLI guidance tests, including line-ending checks. The Orca CLI guide now directs version-matched guide retrieval and bounded fallback commands. Manifest, release mapping, and snapshot metadata are refreshed for the generated artifact.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning It has useful detail, but it does not follow the required template and is missing the Screenshots, AI Review Report, and Security Audit sections. Reformat it to the repository template and add the missing sections, including Screenshots (or 'No visual change'), Testing, AI Review Report, Security Audit, and Notes.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title concisely describes the main change: converting orca-cli into a first-generation hybrid stub.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@AmethystLiang AmethystLiang self-assigned this Jul 18, 2026
@AmethystLiang
AmethystLiang self-requested a review July 18, 2026 16:47
@brennanb2025
brennanb2025 merged commit f8b430f into main Jul 20, 2026
14 checks passed
brennanb2025 added a commit that referenced this pull request Jul 20, 2026
…9607)

* feat(skills): convert the remaining bundled skills to hybrid stubs

Roll out the hybrid discovery-stub pattern (first shipped for orca-cli in
#9238) to the other seven bundled skills: computer-use, linear-tickets,
orca-emulator, orca-emulator-android, orca-linear, orca-per-workspace-env,
and orchestration. Each installable SKILL.md becomes a thin stub — a safe
CLI resolver, an `orca skills get <topic>` pointer, and a bounded read-only
fallback for pre-guide binaries — while the version-matched command
reference now lives only in the Orca binary (embedded guide table, served
by `orca skills get`). The distributed files can no longer drift from the
binary that runs the commands.

- generator: STUB_TOPICS now lists all eight canonical topics; each stub
  body lives in skill-stubs/<topic>.md and the projection reuses the
  guide's own frontmatter, so every routing/description surface is
  byte-identical. The embedded full guides (bundled-skill-guides.ts) are
  untouched.
- manifest regenerated: each converted skill's releaseRevision bumps +1 as
  an append-only snapshot (orca-cli stays at 33). Existing fat installs
  classify `outdated` and get the targeted `npx skills update` nudge with
  no in-app writes.
- tests: command-guidance assertions for computer-use, orca-linear, and
  orchestration repointed to their guide sources (their home now), plus
  per-topic stub-projection + safety coverage.

Per notes/skill-freshness-design.md this completes the fat->stub migration
across all bundled skills. The E.3 pointer-compliance spike and E.5 RC
window flagged on #9238 still apply; allowed-tools frontmatter is
intentionally not added yet (frontmatter kept byte-identical to guides).

* fix(skills): preserve useful hybrid fallbacks
enrick-dev pushed a commit to enrick-dev/orca that referenced this pull request Oct 15, 2026
…ai#9238)

* feat(skills): ship orca-cli as a first-generation hybrid stub

Convert the installable orca-cli SKILL.md from a full fat guide into a
hybrid discovery stub: a safe CLI resolver, an `orca skills get orca-cli`
pointer, and a bounded read-only fallback for pre-guide binaries. The
version-matched command reference now lives only in the Orca binary
(embedded guide table, served by `orca skills get`), so the distributed
file can no longer drift from the binary that runs the commands.

- generator projects STUB_TOPICS from skill-stubs/<name>.md, reusing the
  guide's own frontmatter so the routing/description surface is unchanged;
  the embedded full guide (bundled-skill-guides.ts) is untouched.
- manifest regenerated: orca-cli releaseRevision 32->33 as an append-only
  snapshot; existing fat installs classify `outdated` and get the targeted
  `npx skills update` nudge (no in-app writes).
- tests: command-guidance assertions repointed to the guide source (their
  home now), plus stub-projection + safety coverage.

Only orca-cli converts; the other skills stay fat. Per
notes/skill-freshness-design.md, the E.3 pointer-compliance spike and the
E.5 RC window remain before any further thinning. allowed-tools is
intentionally not added yet (frontmatter kept byte-identical to the guide).

* fix(skills): distinguish guide lookup failures

* chore(skills): refresh released skill mapping
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants