feat(roadmap): expose EpicDefinition status and ready work items through REST/MCP #333

Activity

1. mashkovd

   marked this as blocked on Sep 19, 2026

   • feat(roadmap-control-plane): derive dependency-aware ready work items from EpicDefinition .github#99
2. mashkovd

   marked this as blocking on Sep 19, 2026

   • feat(roadmap): start an exact DevLoop wave from the EpicDefinition ready set #334
3. mashkovd

   added a parent issue on Sep 19, 2026

   • roadmap(roadmap-control-plane): declarative EpicDefinition and GitHub graph reconciliation .github#66
4. mashkovd commented on Sep 23, 2026

   

   mashkovd

   on Sep 23, 2026

   MemberAuthor

   More actions

   Blocker: no distribution path for the canonical readiness evaluator (architecture decision needed)

   Checked on 2026-09-23 against mctl-api origin/main (c963963) and mctlhq/.github main (48e8140), before starting implementation. This issue cannot be done as a narrow change. The source-of-truth boundary it requires ("mctl-api MUST NOT reimplement the dependency algorithm") has no existing transport to rely on.

   Facts

   • mctl-api has no roadmap surface today. Apart from one prose line in docs/work-context-contract.md, there is nothing matching roadmap, RoadmapHealth, ReadySet or completion. The acceptance line "existing roadmap health/completion contracts remain backward compatible" can therefore only mean the .github schemas (roadmap-health.schema.json and its completion block), not an existing mctl-api API.
   • The readiness evaluator's output is never published. roadmap/scripts/ready.py (RoadmapReadySet / RoadmapReadySetList, roadmap.mctl.ai/v1alpha1) is CLI-only. roadmap-validate.yml replays fixtures offline and states that --live never runs in CI. There is no artifact upload, release, scheduled capture or committed snapshot, and the three JSONs under roadmap/fixtures/ are frozen test inputs. The #99 design (agents-state/.github/proposals/issue-99-…/design.md) concludes that "feat(roadmap): expose EpicDefinition status and ready work items through REST/MCP #333 consumes the published document", but it never defines where that document is published.
   • The mctl-api runtime cannot invoke the evaluator as it stands. The image is Go on alpine with only ca-certificates git openssh-client, while the evaluator is Python (jsonschema, PyYAML).
   • Stale reference: the issue body says it depends on .github#96, which is an unrelated CI fix. The readiness contract is .github#99, delivered in PR feat(alerts): fingerprint-based dedup for incident creation #107. The native blocked-by edge already points at fix(api): explicitly empty tenant allowlist now denies all skill enables #99, so only the prose is wrong.

   The choice that needs an owner

   Option	Shape	Cost
   (a) Invoke in-process	mctl-api keeps a shallow clone of mctlhq/.github (existing internal/gitops/reader.go, which already clones mctl-gitops the same way) and runs ready.py / health.py --live via os/exec with a read-only token. It adds the clone's git SHA as the revision, runs consistency_errors on the output, and wraps the result in GET /api/v1/roadmap/epics/{name}/status, /ready and mctl_get_epic_status / mctl_get_ready_work_items.	Python plus two pinned deps in the API image; live GitHub reads per call, or a short cache with capturedAt exposed.
   (b) Publish, then read	An Argo CWFT (via operations.Executor) or a scheduled .github job captures a RoadmapReadySetList + RoadmapHealthList, and mctl-api reads the published artifact.	A new workflow and storage location, plus explicit freshness semantics (a stale capture must be reported, never presented as live).
   (c) Read service	A small Python read service beside mctl-agents, called by mctl-api.	The most new infrastructure.

   Recommendation: (a). It is the only option in which "the same manifest + snapshot yields the same readiness regardless of client" holds by construction, because it runs the very evaluator .github CI tests. It also reuses an existing clone mechanism. Both (b) and (c) add a component whose staleness becomes a new failure mode that #333 would have to surface.

   Two contract gaps must be settled whichever option is chosen:

   1. The documents carry the manifest path and sha256 but no git revision, so mctl-api must add the SHA of the checkout it evaluated.
   2. lifecycle, title and goal are not in RoadmapHealth or RoadmapReadySet. They have to be read from the same pinned EpicDefinition YAML, not re-derived.

   Implementation is on hold until the option is chosen. #334 stays blocked behind this.
5. mashkovd commented on Sep 23, 2026

   

   mashkovd

   on Sep 23, 2026

   MemberAuthor

   More actions

   Proposed design (2026-09-23): one recommendation, for owner confirmation before any code

   Facts, re-verified on main

   • The canonical evaluator lives in mctlhq/.github: roadmap/scripts/ready.py and health.py, on top of github_graph.py / completion.py. compute() is pure (no clock, no I/O); only main() touches the network, and only with GET requests.
   • ready.py is a CLI. --live captures GitHub state and --capture writes the snapshot. Its output is schema-validated (roadmap/schemas/roadmap-ready-set.schema.json) and already carries, per epic, epic.manifest.{path, sha256}, source.{mode, capturedAt, apiBase}, and per-item state with typed blockers, including explicit unknown.
   • Nothing is published. roadmap-validate.yml only replays fixtures, and no workflow is scheduled or writes an artifact.
   • mctl-api has no evaluator and no roadmap code. Its image is alpine with git/openssh-client and no Python. The rejected alternative is mctl-api cloning .github and exec'ing ready.py.
   • An existing primitive fits the read side. mctl-api's internal/gitops.Reader already keeps a periodically refreshed git clone of an arbitrary repoURL+branch (NewReader(repoURL, branch, localPath, token, …)).
   • Every repository the roadmap binds is public (.github, mctl-api, mctl-agents, mctl-telegram, mctl-portal, mctl-docs, mctl-agent, mctl-gitops, mctl-claude-remote). The snapshot schema captures "identity and native relations only; issue prose is never captured". Publishing it exposes nothing that is not already public.

   Recommended boundary

   .github (Roadmap Control Plane)
     roadmap-publish.yml (schedule + push to roadmap/**)
       ready.py --live --capture   +   health.py over the same snapshot
         → roadmap-state branch of mctlhq/.github (orphan, data only):
             snapshot.json          GitHubGraphSnapshot (the observation)
             ready-set.json         RoadmapReadySetList
             health.json            RoadmapHealth per epic
             publication.json       envelope (below)
   mctl-api
     second gitops.Reader on https://github.com/mctlhq/.github.git @ roadmap-state
       internal/roadmap: parse + version-check + staleness, no evaluation
         REST  GET /api/v1/roadmap/epics, /epics/{name}, /ready
         MCP   mctl_get_epic_status, mctl_get_ready_work_items (read-only)
   

   Producer (.github). It is the only place that evaluates. It writes one commit per publication to its own repository with contents: write on that repo only: no cross-repo credential, no GitHub mutation, no issue writes. The git SHA of that commit is the publication version.

   Consumer (mctl-api). A read adapter only. It never calls GitHub for roadmap state and never re-derives readiness. It serves exactly what one publication says, so every consumer that reads the same publication sees the same readiness.

   Contract (publication.json, schema-versioned in .github/roadmap/schemas/)

   • apiVersion / kind: RoadmapPublication.
   • evaluator.revision: the .github commit the evaluator and manifests ran from (GITHUB_SHA).
   • manifests[]: {path, sha256}, already produced per epic.
   • observation: {source.mode, capturedAt, apiBase} and snapshot.sha256 (digest of snapshot.json).
   • outputs: digests of ready-set.json / health.json.

   The determinism guarantee is that ready.py --snapshot snapshot.json at evaluator.revision reproduces ready-set.json byte for byte, so a publication can be verified offline. Item states stay the evaluator's four (complete / ready / blocked / unknown) with typed blockers as the explanation, and unbound or unobserved items stay unknown, never defaulted. The consumer answers 503 when no publication exists or the apiVersion is unsupported. It exposes capturedAt and the publication age on every response so a client can judge freshness itself, and it serves nothing it cannot read.

   Why this mechanism and not the others

   • Actions artifacts expire, need an authenticated zip download, and have no stable "latest" address.
   • Object storage (MinIO) needs new CI→cluster credentials.
   • Committing to mctl-gitops needs a cross-repo App token and puts data churn into the deploy repository.
   • mctl-api running ready.py means Python in the image and a second evaluation site.

   The orphan-branch approach reuses the evaluator as is, the workflow token, and the existing gitops.Reader, so no new infrastructure is needed.

   Minimal decomposition

   1. .github issue A (new): publish a versioned RoadmapPublication. Covers the envelope schema, the publish workflow, the orphan roadmap-state branch, the byte-reproducibility test, and a commit policy of "on readiness change, or a heartbeat when the last capture is older than N minutes". The heartbeat keeps capturedAt honest without a commit per tick. This becomes a new required roadmap node, and feat(roadmap): expose EpicDefinition status and ready work items through REST/MCP #333 dependsOn it.
   2. mctl-api#333 (this issue): the read adapter. A second Reader, internal/roadmap, REST plus OpenAPI, and the two read-only MCP tools. The live DevLoop projection from .github#95 stays a separate optional field, as the issue already says.
   3. Unchanged: mctl-api#334 (governed wave start) consumes feat(roadmap): expose EpicDefinition status and ready work items through REST/MCP #333.

   A note on the dependency in the issue body: "Depends on .github#96" points at a CI fix. The readiness evaluator was .github#99 (closed), so issue A is the real prerequisite.

   Stopping here. No implementation, and no issue A filed, until the owner confirms this boundary.
6. mashkovd

   marked this as blocked on Sep 23, 2026

   • feat(roadmap-control-plane): publish a versioned RoadmapPublication to the roadmap-state branch .github#119
7. mashkovd

   added a commit that references this issue on Sep 23, 2026

   Merge pull request #356 from mctlhq/feat/333-roadmap-read-api

   ...

   2717a3e
8. mashkovd

   closed this as completedin #356on Sep 23, 2026
9. mashkovd commented on Sep 23, 2026

   

   mashkovd

   on Sep 23, 2026

   MemberAuthor

   More actions

   Closure evidence (closed by #356, merge commit 2717a3e):

   • mctl-api reads the RoadmapPublication from mctlhq/.github@roadmap-state via a second gitops.Reader and never evaluates. It verifies apiVersion, the sha256 of every derived file, the manifest↔ready-set↔health bijection, the ready list against item states, and epic identity/lifecycle (published since .github#123 657575c). Anything unverified answers 503, meaning "cannot say".
   • REST GET /api/v1/roadmap/{epics,epic-status,ready} (required_only defaults to true; without epic, every lifecycle: active epic), MCP mctl_get_epic_status / mctl_get_ready_work_items. Every answer carries provenance: state/evaluator/source revisions, capturedAt, age.
   • Tests read a real publication copied byte for byte (roadmap-state@a36af84). Served documents equal the published ones; ready items equal the published ready list; Enterprise MCP fix: raise OpenClaw tenant quota default/floor to limits.cpu=3 #35 serves ≥2 ready required items; no unbound or unknown item is served as ready. Tamper table, cache/no-stale, MCP handler and route-registration tests; every guard mutation-checked.
   • Reviews: Claude APPROVED on 639bdcc, agy PASS; Copilot could not review (quota).
   • Follow-up P3s: fix(roadmap): one epic per root issue; sized volume for the checkout #357. The MCP portal allowlist entries are enabled: false; exposing them on the shared portal is an owner decision. Not released or deployed.

   Next on the graph: #334 (governed wave start) consumes this.