Skip to content
This repository was archived by the owner on Sep 7, 2026. It is now read-only.

ci: add endpoint-audit workflow (report-only) on main - #227

Merged
VascoSch92 merged 2 commits into
mainfrom
vasco/add-endpoint-audit-workflow
Jun 23, 2026
Merged

VascoSch92 merged 2 commits into
mainfrom
vasco/add-endpoint-audit-workflow

Conversation

@VascoSch92

Copy link
Copy Markdown
Member

What

Ports the endpoint audit from vasco/endpoint-audit to main so it runs on PRs (it currently lives only on that feature branch, which is why it doesn't run on #224/#225). Adds:

  • .github/workflows/endpoint-audit.yml — boots the pinned agent-server image, runs the audit, comments the report on the PR, uploads the .audit/ artifact
  • scripts/endpoint-audit.mjs — diffs the client's HTTP surface against the agent-server's live OpenAPI spec
  • endpoint-audit.config.json — config (gate spec + cloud classify spec)
  • package.json — config.agentServerImage pin (ghcr.io/openhands/agent-server:1.29.0-python) + audit:endpoints script

Report-only (not blocking)

This lands with gate.mismatch: false — the workflow runs, posts a drift report as a self-updating PR comment, and uploads the artifact, but never fails CI.

Why: the audit is a strict check that flags any client call not on the agent-server. The client still legitimately calls cloud/enterprise endpoints that are not (and won't be) on the agent-server — cloud-proxy, the 5 meta-profiles, and 2 shared-events. A blocking gate would therefore be red on main and every PR for reasons unrelated to the PR. (Confirmed: the gate fails on every run on the feature branches today.)

Running report-only surfaces the drift on every PR without blocking merges. To make it a blocking gate later — once the client is reconciled to agent-server-only calls (or cloud endpoints are split into a separate client) — flip gate.mismatch: true.

Notes

  • The audit needs a live agent-server, so it only runs in CI (boots the container on port 8010); there's no local/offline mode.
  • No dependency changes, so npm ci is unaffected; no src/ changes.

Ports the endpoint audit from vasco/endpoint-audit so it runs on PRs to
main. It boots the pinned agent-server image, diffs the client's HTTP
surface against the live OpenAPI spec, labels each off-contract call by the
backend that serves it (cloud vs no known backend), posts the report as a
self-updating PR comment, and uploads the .audit/ artifact.

Report-only: gate.mismatch=false, so the check never fails CI. The client
still legitimately calls cloud/enterprise endpoints that are not on the
agent-server (cloud-proxy, meta-profiles, shared-events), which a strict
gate would flag; running report-only surfaces drift without blocking
merges. Flip gate.mismatch=true to make it a blocking gate once the client
is reconciled to agent-server-only calls.

Files:
- .github/workflows/endpoint-audit.yml
- scripts/endpoint-audit.mjs
- endpoint-audit.config.json
- package.json: config.agentServerImage pin + audit:endpoints script
@github-actions

Copy link
Copy Markdown
Contributor

Endpoint audit

❌ 12 off-contract call(s) — not on the agent-server · classifiers: cloud

Category Count
❌ Off-contract (not on agent-server) 12
  ⛔ no known backend 7
  ↗️ served by cloud 5
➕ Missing API (agent-server has, client lacks) 12
agent-server endpoints 104
client endpoints 102

❌ Not on agent-server (gated, 12)

⛔ (no known backend) — served by no backend we can see (7)

  • DELETE /api/meta-profiles/{}
  • GET /api/meta-profiles
  • GET /api/meta-profiles/{}
  • POST /api/cloud-proxy
  • POST /api/meta-profiles/{}
  • POST /api/meta-profiles/{}/activate
  • POST /api/skills/installed/{}/update

↗️ served by cloud (5)

  • DELETE /api/keys/{}
  • GET /api/keys
  • GET /api/shared-events/search
  • POST /api/accept_tos
  • POST /api/keys

➕ Missing API — agent-server exposes it, client does not implement (12)

  • GET /
  • GET /api/bash/bash_events
  • GET /api/conversations
  • GET /api/conversations/{}/events
  • GET /api/conversations/{}/workspace
  • GET /api/conversations/{}/workspace/{}
  • GET /api/init
  • POST /api/conversations/{}/goal
  • POST /api/conversations/{}/goal/resume
  • POST /api/conversations/{}/goal/stop
  • POST /api/init
  • POST /api/skills/installed/{}/refresh

@VascoSch92
VascoSch92 requested a review from enyst June 23, 2026 13:39
- on: push only (drop pull_request trigger) per request.
- prettier --write scripts/endpoint-audit.mjs to satisfy format:check
  (the test job's 'Check formatting' step was failing on the unformatted
  copied file).
@VascoSch92
VascoSch92 merged commit 9a9f6a5 into main Jun 23, 2026
8 checks passed
@VascoSch92 VascoSch92 added the type: ci CI configuration changes label Jun 24, 2026
@openhands-release-bot openhands-release-bot Bot added the released: v1.27.0 Shipped in v1.27.0 label Jun 25, 2026
@openhands-release-bot

Copy link
Copy Markdown
Contributor

🚀 Released in v1.27.0.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

released: v1.27.0 Shipped in v1.27.0 type: ci CI configuration changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants