Skip to content

feat(cognitive): emit TASK_PROGRESS at every step boundary — closes the streaming pipe - #20

Merged
flg77 merged 1 commit into
mainfrom
feat/agent-task-progress-emit
May 1, 2026
Merged

feat(cognitive): emit TASK_PROGRESS at every step boundary — closes the streaming pipe#20
flg77 merged 1 commit into
mainfrom
feat/agent-task-progress-emit

Conversation

@flg77

@flg77 flg77 commented May 1, 2026

Copy link
Copy Markdown
Owner

Summary

PR #19 wired the receive side (observer fan-out → channel callback → prompt-pane transcript). This PR adds the producer side so an operator using the prompt pane sees live progress lines without injecting synthetic events.

Pipeline (now end-to-end)

```
Operator types prompt → Send


TUIPromptChannel.send(..., on_progress=cb)
│ publishes TASK_ASSIGN + registers progress listener

Agent._handle_task receives TASK_ASSIGN
│ builds a sync callback that publishes TASK_PROGRESS

CognitiveCore.process_task(... progress_callback=cb)
step 1/6 Pre-reasoning gate ─┐
step 2/6 Building system prompt │
step 3/6 Calling LLM │ each fires callback
step 4/6 Post-reasoning governance │ → publishes TASK_PROGRESS
step 5/6 Persisting episode │
step 6/6 Drift scoring ─┘


dispatch_invocations(... progress_callback=cb)
step 1/N Calling skill:echo ─┐ same callback,
step 2/N Calling mcp:fs.read ─┘ same task_id


NATSObserver._route_task_progress fans out to per-task_id listeners


PromptScreen renders "→ step N/M — " lines in the transcript
```

What lands

File Change
`acc/cognitive_core.py` `process_task` gains optional `progress_callback` kwarg + an inline `_emit` helper. Six emits at the canonical step boundaries with `elapsed_ms` (since process_task entry) and post-LLM token counts. Exception-isolated.
`acc/capability_dispatch.py` `dispatch_invocations` gains optional `progress_callback` kwarg. One emit per invocation BEFORE dispatch with `current_step`/`total_steps_estimated` and `step_label="Calling :"`. Lazy-imports `ProgressContext`.
`acc/agent.py` `_handle_task` builds the agent-side callback when the inbound TASK_ASSIGN carries a `task_id`. Sync callback schedules an async publish via `asyncio.create_task` — fire-and-forget so cognitive pipeline never blocks on NATS. Same callback passed to both `process_task` and `dispatch_invocations` so the operator sees one continuous progress stream per task.
`tests/test_task_progress_emit.py` (new) 10 cases.

Tests (10 new, all green)

Tier Cases
Tier 1process_task emits exactly 6 emits in order with monotonic step counter; post-LLM emit carries non-zero token counts; elapsed_ms non-decreasing; None callback is zero overhead; misbehaving callback at step 2 doesn't stop steps 3-6
Tier 2dispatch_invocations emits one emit per invocation in order; emit fires BEFORE dispatch; None callback zero overhead; bad callback for invocation 1 doesn't skip invocation 2
Tier 3 — End-to-end across PR #19's receive pipe Full process_task → agent-callback → observer fan-out → consumer round-trip; payload shape matches what TUIPromptChannel expects (signal_type + task_id + agent_id + nested progress dict with every ProgressContext field)

Combined regression: 235 passed locally and on acc1
(test_task_progress_emit + test_task_progress_streaming + test_capability_dispatch_oversight + test_cognitive_core + test_prompt_screen_pilot + test_prompt_channel + test_observer_task_listener + test_redis_compat + test_capability_telemetry + test_config + test_role_store + test_guardrails + test_compliance, with the lighthouse-OpenSSL-impaired `TestEd25519Validation` deselected).

Test plan

  • `pytest tests/test_task_progress_emit.py -v --no-cov` → 10 passed
  • Combined regression on acc1 → 235 passed
  • Manual on acc1 post-merge:
    ```bash
    ./acc-deploy.sh rebuild # pull + no-cache build
    ./acc-deploy.sh down && ./acc-deploy.sh up
    acc-tui # press 7

    Send a prompt → watch transcript fill with live step lines

    WITHOUT injecting synthetic events.

    ```

Operational expectation post-merge

Operator sends a prompt → transcript renders 6 step lines for the LLM half + N step lines for the capability half, all under one task_id, all under one prompt round-trip:

```
14:32:01 operator → coding_agent
Generate a unit test for FizzBuzz

→ step 1/6 — Pre-reasoning gate (Cat-B setpoints)
→ step 2/6 — Building system prompt
→ step 3/6 — Calling LLM
→ step 4/6 — Post-reasoning governance
→ step 5/6 — Persisting episode + embedding output
→ step 6/6 — Drift scoring
→ step 1/1 — Calling skill:echo
✓ skill:echo

14:32:08 coding_agent-x latency=147ms
def test_fizzbuzz_basic():
assert fizzbuzz(15) == "FizzBuzz"
```

Out of scope (future enhancements)

  • Confidence-trend computation in CognitiveCore — currently fixed STABLE. Needs a small rolling-window of past confidence values.
  • Per-step deadline / over_budget reporting from Cat-B setpoint enforcement.
  • SlackPromptChannel honouring on_progress — would post follow-up Slack messages with the live step lines.

…s the streaming pipe end-to-end

PR #19 wired the receive side (observer fan-out → channel callback →
prompt-pane transcript) but no producer was emitting events.  This
PR adds the emitter half so the prompt pane shows live forward-motion
lines without operators having to inject synthetic events via acc-cli.

acc/cognitive_core.py
  process_task() gains optional progress_callback kwarg.  When
  supplied, it fires once at each of the 6 canonical step boundaries:

      1/6  Pre-reasoning gate (Cat-B setpoints)
      2/6  Building system prompt
      3/6  Calling LLM
      4/6  Post-reasoning governance       (carries token counts)
      5/6  Persisting episode + embedding output
      6/6  Drift scoring

  Each emit constructs a full ProgressContext with current_step,
  total_steps_estimated=6, step_label, elapsed_ms (since
  process_task entry), and (where known after step 4) llm_calls_so_far
  + tokens_in/out_so_far.  Exception-isolated — a misbehaving callback
  cannot break the cognitive pipeline.

  Module-level constant ``_PROCESS_TASK_TOTAL_STEPS = 6`` so the test
  suite can assert against the same number the runtime uses.

acc/capability_dispatch.py
  dispatch_invocations() gains optional progress_callback kwarg.
  Fires once per invocation BEFORE dispatch with current_step starting
  at 1, total_steps_estimated = len(invocations), step_label
  ``"Calling skill:echo"`` / ``"Calling mcp:fs.read"``.  Exception-
  isolated.  Lazy-imports ProgressContext to keep the dispatcher's
  import graph light.

acc/agent.py
  _handle_task() builds a per-task callback when the inbound TASK_ASSIGN
  carries a task_id (tasks without one — e.g. legacy synthetic
  injections — silently skip emission, keeping the pipeline lean).

  The callback is sync; it schedules an async publish via
  ``asyncio.create_task`` so the cognitive pipeline never blocks on
  NATS.  Fire-and-forget matches the operational tolerance for
  occasional lost progress events — operators care about forward
  motion, not exactly-once ordering.

  The same callback is passed to BOTH process_task and
  dispatch_invocations so the prompt pane sees a continuous progress
  stream (steps 1-6 of the LLM half + per-invocation steps for the
  capability half) under a single task_id.

Tests (tests/test_task_progress_emit.py — NEW, 10 cases all green):
  Tier 1 — process_task (5):
    * exactly 6 emits in order with monotonic current_step,
    * post-LLM emit carries non-zero token counts,
    * elapsed_ms is non-negative + monotonically non-decreasing,
    * progress_callback=None is zero overhead (no crash, normal result),
    * a misbehaving callback at step 2 does NOT stop steps 3-6 from
      firing or the pipeline from completing.
  Tier 2 — dispatch_invocations (4):
    * one emit per invocation in source order,
    * emit fires BEFORE dispatch (operator sees "Calling X" before
      the X outcome lands),
    * None callback is zero overhead,
    * exception in callback for invocation 1 doesn't skip invocation 2.
  Tier 3 — end-to-end across the receive pipe (1):
    * Full process_task → agent-callback → observer fan-out → consumer
      round-trip; payload shape matches what TUIPromptChannel expects
      (signal_type + task_id + agent_id + nested progress dict with
      every ProgressContext field).

Combined regression: 235 unit tests pass
(test_task_progress_emit + test_task_progress_streaming +
test_capability_dispatch_oversight + test_cognitive_core +
test_prompt_screen_pilot + test_prompt_channel + test_observer_task_listener
+ test_redis_compat + test_capability_telemetry + test_config +
test_role_store + test_guardrails + test_compliance,
TestEd25519Validation deselected for the lighthouse OpenSSL platform
limit as in earlier PRs).

Operational expectation post-merge: an operator using the prompt pane
(screen 7) sees a live transcript like:

    14:32:01  operator → coding_agent
      Generate a unit test for FizzBuzz
      → step 1/6 — Pre-reasoning gate (Cat-B setpoints)
      → step 2/6 — Building system prompt
      → step 3/6 — Calling LLM
      → step 4/6 — Post-reasoning governance
      → step 5/6 — Persisting episode + embedding output
      → step 6/6 — Drift scoring
      ✓ skill:echo
    14:32:08  coding_agent-x  latency=147ms
      def test_fizzbuzz_basic():
          assert fizzbuzz(15) == "FizzBuzz"

without injecting any synthetic events.

Out of scope (future enhancements):
* Confidence-trend computation in CognitiveCore (currently fixed
  STABLE).  Needs a small rolling-window of past confidence values.
* SlackPromptChannel honouring on_progress (would post follow-up
  Slack messages with the live step lines).
* Per-step deadline / over_budget reporting from CognitiveCore's
  Cat-B setpoint enforcement.
@flg77
flg77 merged commit 12dc15d into main May 1, 2026
@flg77
flg77 deleted the feat/agent-task-progress-emit branch May 1, 2026 15:01
flg77 added a commit that referenced this pull request May 2, 2026
…ional end-to-end (#24)

The ``mcps/echo_server`` manifest has shipped since Phase 4.2 as a
loader / TUI / role-allow-list fixture, but no actual server backed
the URL.  ``coding_agent``'s ``allowed_mcps=["echo_server"]``
declaration was a smoke target that 404'd at run time, and the
manual-smoke recipe in PR #20's prompt-pane help doc had to inject
synthetic events because the [MCP: echo_server.echo {...}] markers
the agent emits couldn't reach a real server.

This PR ships:

container/production/echo_mcp_server/main.py (NEW)
  Tiny diagnostic JSON-RPC 2.0 server.  stdlib http.server only —
  no Flask, aiohttp, mcp, or json-rpc deps.  Implements exactly the
  three methods :class:`acc.mcp.client.MCPClient` exercises:
    * initialize     → returns canonical protocolVersion + serverInfo
    * tools/list     → advertises one ``echo`` tool with valid
                       JSON Schema for its input
    * tools/call     → name=echo round-trips arguments.text in MCP's
                       canonical { content: [{type, text}] } envelope
  Listens on ACC_MCP_ECHO_HOST / _PORT (default 0.0.0.0:8080).
  Logs go through the standard logging pipeline at ACC_LOG_LEVEL
  (default INFO).  GET / returns a banner so curl /health-style
  probes don't 404.

container/production/Containerfile.echo-mcp-server (NEW)
  UBI10 python-312-minimal base, ~1 MB on top of the base layer
  (no pip deps).  Non-root UID 1001, EXPOSE 8080, USER 1001,
  CMD python3 /app/main.py.

container/production/podman-compose.yml
  + acc-mcp-echo service behind ``--profile mcp-echo``.  Container
    name matches the hostname referenced in the manifest's
    ``url: http://acc-mcp-echo:8080/rpc`` so agents resolve it on
    the acc-net bridge with zero manifest edits.  Healthcheck via
    GET / using stdlib urllib so we don't add curl to the image.
    Auto-built alongside the others by ``./acc-deploy.sh build``.

acc-deploy.sh
  + MCP_ECHO env var (default false) wires through to the
    --profile mcp-echo activation.  Header echoes the flag when set.
  + Profile auto-attaches on build/rebuild so a single
    ``./acc-deploy.sh rebuild`` produces the image without
    needing MCP_ECHO=true.
  + Header docstring documents the new env var.

mcps/echo_server/mcp.yaml
  Header rewritten — the manifest is no longer a "stub example", it
  describes a live diagnostic server.  Manual-smoke recipe added
  (MCP_ECHO=true ./acc-deploy.sh up + nats sub).

Tests (tests/test_echo_mcp_server.py — NEW, 8 cases all green):
  Pure JSON-RPC dispatch (7):
    * initialize returns protocolVersion + serverInfo,
    * tools/list advertises echo with valid JSON Schema,
    * tools/call name=echo round-trips text,
    * unknown tool name → JSON-RPC error with name in message,
    * tools/call with non-string text → param error -32602,
    * unknown method → method-not-found -32601,
    * response id always matches request id (across int / str / "" / None
      so ACC client's id-mismatch check never fires).
  End-to-end round-trip (1):
    * Spin up the real HTTPServer on 127.0.0.1:<ephemeral_port> in a
      background thread, drive it via :class:`acc.mcp.client.MCPClient`
      with a real ``MCPManifest``.  Confirms wire-format compatibility
      on both sides — server response shape passes the client's
      JSON-RPC envelope validation, MCPClient.list_tools + call_tool
      return the expected dicts.

Combined regression: 265 unit tests pass on Windows
(test_echo_mcp_server + test_mcp_stdio_transport + the four
coding_agent tier files + test_task_progress_* + the prompt-pane
suite + the redis-compat suite + telemetry + ecosystem +
file-picker + config + role-store + guardrails + compliance,
with TestEd25519Validation deselected for the lighthouse OpenSSL
platform limit).

acc1 verification deferred — host SSH still unreachable
(timeout to 10.199.12.91:22 since PR #21).  Tests are pure-Python
+ stdlib HTTP — local Windows result transfers cleanly when the
host is back.  Manual smoke recipe for acc1 is in mcps/echo_server/mcp.yaml.

Out of scope:
* Resources / prompts methods (the MCP spec covers more than tools;
  ACC only consumes tools today, so this server only implements the
  three methods the client exercises).
* HTTPS / mTLS — diagnostic server, host-network only.
* SSE / streamable HTTP transport — current ACC HTTPTransport posts
  one JSON-RPC envelope per request; SSE would require wiring on the
  client side too (separate PR if needed).
flg77 added a commit that referenced this pull request May 2, 2026
…ow reflect real movement (#25)

PR #20 wired the agent-side TASK_PROGRESS emitter, but every emit
hard-coded ``confidence_trend="STABLE"``.  The ↑/→/↓ arrows in the
prompt-pane transcript (PR #19) all showed → regardless of whether
the agent was actually doing well — operators couldn't read forward
motion from the trend column.

This PR derives the trend per emit from a rolling history of
confidence values and feeds each step a meaningful signal so the
arrows track the agent's real state.

acc/cognitive_core.py:
  process_task gains a closure-cell ``_conf_history: list[float]``.
  Each emit appends its confidence; the emit before the append
  inspects the previous value to compute the trend using the same
  ±0.05 epsilon convention as ``ProgressContext.next_step``:
      delta > +0.05 → RISING
      delta < −0.05 → FALLING
      otherwise     → STABLE
  First emit always reports STABLE (no prior to compare).

  Per-step confidence values now read real signal:
    1. Pre-reasoning gate           0.5  (neutral, no evidence yet)
    2. Building system prompt       0.5  (mechanical)
    3. Calling LLM                  0.55 (got past the guards)
    4. Post-reasoning governance    0.40..0.85, derived from
                                    deviation_score via a linear
                                    clamp 0.85 − 0.225·deviation
                                    (clean output → high; high
                                    deviation → low).
    5. Persisting episode           same as step 4 (no new evidence)
    6. Drift scoring                1 − drift_score clamped to
                                    [0.40, 0.95] (low drift → high
                                    confidence)

  Step 4 emit moved from before _post_reasoning_governance() to
  after, so the deviation_score is computable at emit time.  Same
  for step 6 (drift computed first, then emit).

  ``_conf_history`` accumulates even when ``progress_callback`` is
  None — cheap, and keeps the rolling-window state consistent across
  callback toggles in tests.

acc/tui/help/prompt.md:
  Updated example transcript with the new step labels + realistic
  confidence values + trend arrows so operators reading the help
  see what they'll actually see in the pane.

Tests (tests/test_progress_confidence_trend.py — NEW, 8 cases all green):
  Trend computation:
    test_first_emit_always_reports_stable_trend
      → guards a future "default to RISING" refactor.
    test_trend_rises_when_confidence_jumps_more_than_005
      → strict-greater-than semantics: +0.05 exactly = STABLE,
        +0.30 (post-gate clean output) = RISING.
    test_trend_falls_when_post_gate_deviation_high
      → 5x token budget → deviation 3.88 → confidence floor 0.40
        → step 4 falling vs step 3's 0.55.
  Per-step confidence shape:
    test_step_5_carries_step_4_value_so_trend_is_stable
      → guards a refactor that re-derives confidence from stale
        state at step 5.
    test_step_6_confidence_derived_from_drift_score
      → asserts the relationship shape (∈ [0.40, 0.95]) rather than
        an exact float — first-task drift depends on whether the
        role centroid has been seeded.
  Invariants:
    test_trend_label_is_always_one_of_three_legal_values
      → no free-form strings; prompt-pane render only handles the
        three labels.
    test_confidence_value_in_unit_interval_for_every_emit
      → catches map-function refactors that overflow on extreme
        deviation / drift inputs.
    test_progress_callback_none_path_does_not_corrupt_history
      → smoke test for the optimisation: history records without a
        callback, fresh ``_conf_history`` per process_task call.

Combined regression: 280 unit tests pass on Windows
(test_progress_confidence_trend + test_task_progress_emit +
test_task_progress_streaming + test_capability_dispatch_oversight +
test_cognitive_core + test_prompt_screen_pilot + test_echo_mcp_server
+ test_mcp_stdio_transport + the four coding_agent tier files +
test_redis_compat + test_capability_telemetry + test_config +
test_role_store + test_guardrails + test_compliance, with
TestEd25519Validation deselected for the lighthouse OpenSSL
platform limit).

acc1 verification still deferred — host SSH unreachable since PR #21
(5 PRs ago).  Tests are pure-Python with stub LLM/vector backends;
no platform-specific code.

Out of scope (future enhancements):
* Confidence signal from the LLM response itself (some backends
  return logprobs / confidence scores in the usage block — wire
  those into step 3/4 confidence when present).
* Per-invocation confidence in dispatch_invocations — currently
  hard-codes 0.5 / STABLE.  Could use the InvocationOutcome.ok
  flag to bump up/down by 0.10.  Separate PR if useful.
flg77 added a commit that referenced this pull request May 7, 2026
* feat(cluster): cluster_id propagation foundation (PR-1 of subagent clustering)

New acc/cluster.py module:
* ClusterPlan dataclass with field invariants (subagent_count >= 1,
  difficulty in [0, 1]).
* In-memory registry (register/lookup/unregister/list) with optional
  Redis mirror via redis_compat — edge-friendly: works without Redis.
* new_cluster_id() emits c-prefixed UUIDs to discriminate from task_id
  (plan-…) and agent_id (<role>-<hex>) in log lines / dashboards.
* fetch_cluster_async() backfills from Redis on local-cache miss.

Wire-protocol propagation:
* acc/plan.py:_publish_task_assign accepts optional cluster_id +
  target_agent_id kwargs; both attached only when supplied so legacy
  single-agent payloads stay byte-identical.
* acc/agent.py:_handle_task echoes inbound cluster_id on every
  outbound TASK_PROGRESS and TASK_COMPLETE so cluster fan-in
  aggregators see a complete event stream per cluster.

TUI fan-out:
* NATSObserver gains register_cluster_listener / unregister_cluster_listener
  + internal _fan_out_cluster helper. Every cluster-tagged
  TASK_PROGRESS / TASK_COMPLETE fans out to per-cluster_id callbacks
  with per-callback exception isolation (one buggy listener cannot
  starve others). Payloads without cluster_id are silently ignored.

21 new tests in tests/test_cluster_propagation.py covering:
dataclass invariants, registry round-trip, sync/async lookup miss
behaviour, TASK_ASSIGN cluster_id presence/absence, listener fan-out,
unregister idempotency, multi-listener support, exception isolation.

Foundation for PR-2 (estimator + sub-cluster spawn) and PR-4
(TUI cluster panel). No user-visible change in this PR.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(estimator): role-driven sub-cluster estimator + PlanExecutor fan-out (PR-2)

New acc/estimator.py:
* TaskComplexity dataclass — narrow input surface (tokens, task_type,
  required_skills, has_external_io).
* Estimator Protocol — pure callable, easy to fuzz + custom-implement.
* default_estimator: token-budget heuristic (base + ceil(tokens/per_n))
  with keyword-driven difficulty bumps.  Output clamped to
  [1, min(cap, role.max_parallel_tasks)] — defence in depth on top of
  Cat-A A-019.
* build_estimator() dispatcher: 'heuristic' (default) | 'fixed' |
  'module:dotted.path'.  Unknown / failing strategies log + fall back
  to heuristic — arbiter NEVER crashes on a buggy operator config.
* slice_skill_mix() round-robins skills across N sub-agents so no one
  member loads every skill prompt.
* derive_complexity() turns a raw step payload into TaskComplexity
  (token estimate via len(text)//4, [SKILL: ...] hint extraction).

acc/config.py:
* RoleDefinitionConfig gains max_parallel_tasks: int = 1 (legacy: no
  parallelisation) and estimator: dict[str, Any] = {}.  Schema is a
  free-form dict on purpose so 'module:' strategies don't need
  schema bumps.

acc/plan.py:
* PlanExecutor.__init__ accepts optional role_resolver / skill_resolver
  callbacks.  Without them, dispatch is byte-identical to PR-1.
* _maybe_build_cluster: consults the estimator, returns ClusterPlan or
  None for single-agent fallback.  All exceptions logged + downgraded
  to single-agent dispatch.
* _dispatch_cluster: fans one PLAN step out as N TASK_ASSIGN payloads
  sharing one cluster_id (PR-1 wire shape).
* on_task_complete: cluster aggregation — step transitions only after
  all members report.  COMPLETE if every member ok, FAILED if any one
  blocked.  Cluster auto-unregistered on transition.

regulatory_layer/category_a/constitutional_rhoai.rego:
* Bumped 0.4.0 → 0.5.0.
* Two new rules:
  - deny_cluster_oversize: subagent_count > role.max_parallel_tasks
  - deny_cluster_nonpositive: subagent_count < 1
  Both gate action='CLUSTER_SPAWN' so external (Gatekeeper) admission
  enforces the same invariant as the in-process clamp.

26 new tests in tests/test_estimator.py — heuristic shape, role-cap
clamp, [0,1] difficulty bound, skill_mix precedence, fixed strategy,
module: import + import-failure fallback, unknown-strategy fallback,
slice_skill_mix round-robin, derive_complexity SKILL-hint extraction,
PlanExecutor single-agent fallback, fan-out wire-shape, A-019 in-process
clamp, estimator-failure → single-agent fallback, cluster aggregation
all-members-must-report, any-blocked-fails-step.

47 passed across PR-1 + PR-2 test modules; 104 across all related
modules.

Foundation complete for PR-4 (TUI cluster panel) which subscribes via
the PR-1 register_cluster_listener and PR-3 (markdown role authoring,
independent of this PR).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(tui): cluster topology panel in prompt pane (PR-4 of subagent clustering)

acc/tui/widgets/cluster_panel.py (new):
* ClusterPanel widget — collapsible header + per-cluster body.
* Render-only: takes a snapshot dict, emits Rich-marked-up text.
* Driven manually via render_now(); no reactive watcher (Textual
  >=0.80 watcher path can re-enter layout, breaks Pilot tests).
* 30 s grace window after a cluster finishes — operator gets to read
  the final state before the row disappears.
* Skill name extracted heuristically from "Calling skill:<name>"
  step labels (capability_dispatch convention from PR #20).

acc/tui/models.py:
* CollectiveSnapshot.cluster_topology dict — keyed by cluster_id,
  populated from cluster-tagged TASK_PROGRESS / TASK_COMPLETE.

acc/tui/client.py:
* NATSObserver._fan_out_cluster also folds events into the snapshot
  via _update_cluster_topology — keeps panel rendering snapshot-driven
  without forcing every screen to register listeners.
* Member status transitions: running → complete | blocked.
* subagent_count tracked as running max of witnessed members.
* finished_at stamped when every observed member has reported, so
  the panel's grace-window filter can hide finished clusters.

acc/tui/screens/prompt.py:
* ClusterPanel mounted between target row and transcript.
* watch_snapshot pushes cluster_topology + calls panel.render_now()
  from a non-layout context.

11 new tests in tests/test_cluster_panel.py — aggregator fold-shape
(progress creates row, complete marks member done, finished_at stamped),
back-compat (no cluster_id → no row), skill_in_use extraction for
skill: + mcp:, panel render header counts + total members, expanded
member rows, grace-window filtering.

72 passed across PR-1/2/4 + streaming; 121 across all related modules.

Foundation for PR-5 (slash commands incl. /cluster show + /cluster
kill) which leverages the same registry + listener path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
flg77 added a commit that referenced this pull request May 7, 2026
* feat(cluster): cluster_id propagation foundation (PR-1 of subagent clustering)

New acc/cluster.py module:
* ClusterPlan dataclass with field invariants (subagent_count >= 1,
  difficulty in [0, 1]).
* In-memory registry (register/lookup/unregister/list) with optional
  Redis mirror via redis_compat — edge-friendly: works without Redis.
* new_cluster_id() emits c-prefixed UUIDs to discriminate from task_id
  (plan-…) and agent_id (<role>-<hex>) in log lines / dashboards.
* fetch_cluster_async() backfills from Redis on local-cache miss.

Wire-protocol propagation:
* acc/plan.py:_publish_task_assign accepts optional cluster_id +
  target_agent_id kwargs; both attached only when supplied so legacy
  single-agent payloads stay byte-identical.
* acc/agent.py:_handle_task echoes inbound cluster_id on every
  outbound TASK_PROGRESS and TASK_COMPLETE so cluster fan-in
  aggregators see a complete event stream per cluster.

TUI fan-out:
* NATSObserver gains register_cluster_listener / unregister_cluster_listener
  + internal _fan_out_cluster helper. Every cluster-tagged
  TASK_PROGRESS / TASK_COMPLETE fans out to per-cluster_id callbacks
  with per-callback exception isolation (one buggy listener cannot
  starve others). Payloads without cluster_id are silently ignored.

21 new tests in tests/test_cluster_propagation.py covering:
dataclass invariants, registry round-trip, sync/async lookup miss
behaviour, TASK_ASSIGN cluster_id presence/absence, listener fan-out,
unregister idempotency, multi-listener support, exception isolation.

Foundation for PR-2 (estimator + sub-cluster spawn) and PR-4
(TUI cluster panel). No user-visible change in this PR.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(estimator): role-driven sub-cluster estimator + PlanExecutor fan-out (PR-2)

New acc/estimator.py:
* TaskComplexity dataclass — narrow input surface (tokens, task_type,
  required_skills, has_external_io).
* Estimator Protocol — pure callable, easy to fuzz + custom-implement.
* default_estimator: token-budget heuristic (base + ceil(tokens/per_n))
  with keyword-driven difficulty bumps.  Output clamped to
  [1, min(cap, role.max_parallel_tasks)] — defence in depth on top of
  Cat-A A-019.
* build_estimator() dispatcher: 'heuristic' (default) | 'fixed' |
  'module:dotted.path'.  Unknown / failing strategies log + fall back
  to heuristic — arbiter NEVER crashes on a buggy operator config.
* slice_skill_mix() round-robins skills across N sub-agents so no one
  member loads every skill prompt.
* derive_complexity() turns a raw step payload into TaskComplexity
  (token estimate via len(text)//4, [SKILL: ...] hint extraction).

acc/config.py:
* RoleDefinitionConfig gains max_parallel_tasks: int = 1 (legacy: no
  parallelisation) and estimator: dict[str, Any] = {}.  Schema is a
  free-form dict on purpose so 'module:' strategies don't need
  schema bumps.

acc/plan.py:
* PlanExecutor.__init__ accepts optional role_resolver / skill_resolver
  callbacks.  Without them, dispatch is byte-identical to PR-1.
* _maybe_build_cluster: consults the estimator, returns ClusterPlan or
  None for single-agent fallback.  All exceptions logged + downgraded
  to single-agent dispatch.
* _dispatch_cluster: fans one PLAN step out as N TASK_ASSIGN payloads
  sharing one cluster_id (PR-1 wire shape).
* on_task_complete: cluster aggregation — step transitions only after
  all members report.  COMPLETE if every member ok, FAILED if any one
  blocked.  Cluster auto-unregistered on transition.

regulatory_layer/category_a/constitutional_rhoai.rego:
* Bumped 0.4.0 → 0.5.0.
* Two new rules:
  - deny_cluster_oversize: subagent_count > role.max_parallel_tasks
  - deny_cluster_nonpositive: subagent_count < 1
  Both gate action='CLUSTER_SPAWN' so external (Gatekeeper) admission
  enforces the same invariant as the in-process clamp.

26 new tests in tests/test_estimator.py — heuristic shape, role-cap
clamp, [0,1] difficulty bound, skill_mix precedence, fixed strategy,
module: import + import-failure fallback, unknown-strategy fallback,
slice_skill_mix round-robin, derive_complexity SKILL-hint extraction,
PlanExecutor single-agent fallback, fan-out wire-shape, A-019 in-process
clamp, estimator-failure → single-agent fallback, cluster aggregation
all-members-must-report, any-blocked-fails-step.

47 passed across PR-1 + PR-2 test modules; 104 across all related
modules.

Foundation complete for PR-4 (TUI cluster panel) which subscribes via
the PR-1 register_cluster_listener and PR-3 (markdown role authoring,
independent of this PR).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(tui): cluster topology panel in prompt pane (PR-4 of subagent clustering)

acc/tui/widgets/cluster_panel.py (new):
* ClusterPanel widget — collapsible header + per-cluster body.
* Render-only: takes a snapshot dict, emits Rich-marked-up text.
* Driven manually via render_now(); no reactive watcher (Textual
  >=0.80 watcher path can re-enter layout, breaks Pilot tests).
* 30 s grace window after a cluster finishes — operator gets to read
  the final state before the row disappears.
* Skill name extracted heuristically from "Calling skill:<name>"
  step labels (capability_dispatch convention from PR #20).

acc/tui/models.py:
* CollectiveSnapshot.cluster_topology dict — keyed by cluster_id,
  populated from cluster-tagged TASK_PROGRESS / TASK_COMPLETE.

acc/tui/client.py:
* NATSObserver._fan_out_cluster also folds events into the snapshot
  via _update_cluster_topology — keeps panel rendering snapshot-driven
  without forcing every screen to register listeners.
* Member status transitions: running → complete | blocked.
* subagent_count tracked as running max of witnessed members.
* finished_at stamped when every observed member has reported, so
  the panel's grace-window filter can hide finished clusters.

acc/tui/screens/prompt.py:
* ClusterPanel mounted between target row and transcript.
* watch_snapshot pushes cluster_topology + calls panel.render_now()
  from a non-layout context.

11 new tests in tests/test_cluster_panel.py — aggregator fold-shape
(progress creates row, complete marks member done, finished_at stamped),
back-compat (no cluster_id → no row), skill_in_use extraction for
skill: + mcp:, panel render header counts + total members, expanded
member rows, grace-window filtering.

72 passed across PR-1/2/4 + streaming; 121 across all related modules.

Foundation for PR-5 (slash commands incl. /cluster show + /cluster
kill) which leverages the same registry + listener path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(prompt): slash commands in the prompt input (PR-5 of subagent clustering)

Operator types '/<verb> <args>' in the prompt textarea; recognised
commands trigger an action without an LLM round-trip.  Non-slash
input flows through the legacy prompt-dispatch path unchanged
(full back-compat — every existing keystroke pattern still works).

acc/slash_commands.py (new):
* Pure-function parser — returns SlashIntent(kind, args, error).
* Verbs: /help, /cancel <task_id|cluster_id>, /cluster show|kill,
  /role list, /skills, /oversight pending|approve|reject.
* /cancel <c-…> auto-routes to KIND_CLUSTER_KILL via the c- prefix
  so operators can type either '/cancel <id>' or '/cluster kill <id>'.
* Unknown verbs return KIND_UNKNOWN with a helpful message — never
  raise.  Operators learn from typos in the transcript.
* HELP_TEXT documented + the test harness pins that every verb
  appears in it (no future verb additions will silently lack help).

acc/signals.py:
* SIG_TASK_CANCEL = "TASK_CANCEL".
* subject_task_cancel(cid) → "acc.{cid}.task.cancel" — distinct from
  subject_task so cancel handlers can subscribe cheaply.

acc/tui/screens/prompt.py:
* action_send branches on '/' prefix → _dispatch_slash().
* /cancel + /cluster kill publish TASK_CANCEL via the active
  observer (fire-and-forget; the agent's TASK_COMPLETE with
  blocked=True, block_reason='cancelled' is what the operator
  ultimately observes).
* /cluster show renders the current snapshot in the transcript.
* /role list + /skills query the local registry and append a
  system entry — no NATS round-trip.
* /oversight verbs are accepted by the parser and stubbed in the
  dispatch with a 'use Compliance screen' hint (full wiring lands
  in a follow-up).

22 new tests in tests/test_slash_commands.py — every verb routed,
required-arg enforcement (cancel without target, cluster kill without
id, oversight reject without reason), c-prefix routing, unknown verb
helpful message, HELP_TEXT covers every accepted verb, signal subject
format pinned.

82 passed across PR-1/2/4/5; 121 across all related modules.

The agent-side TASK_CANCEL subscriber + cooperative checkpoint inside
CognitiveCore.process_task land in a separate small follow-up so this
PR stays focused on the operator surface.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
flg77 added a commit that referenced this pull request Jun 5, 2026
…I variant

End-to-end runner that walks the operator through the five-phase
acc1 K8s hub smoke after PRs #20-#28 land.  Hermetic CI variant
exercises the same chain in-process against a file-mode catalog
so PR-time tests prove the wiring without a running cluster.

What ships:

  * tools/smoke-acc1-hub.sh (NEW):
    - Phase 0 preflight — checks cosign / kubectl / python / jq /
      curl / acc-pkg on PATH.
    - Phase 1 — applies gitops/acc-hub/ if not present; waits for
      rollout; curls /index.json.
    - Phase 2 — generates pilot cosign keypair via
      tools/cosign-pilot-keygen.sh if not on disk.
    - Phase 3 — builds pilot pkg, signs with cosign sign-blob,
      publishes via gitops/acc-hub/publish-to-hub.sh; verifies the
      hub now advertises the package via jq on the live index.
    - Phase 4 — downloads tarball + sig from live hub, runs
      acc-pkg install into a tmp sandbox, exercises cosign verify.
    - Phase 5 — RoleLoader resolves coding_agent from the
      installed-package path (proves the dual-source loader chain
      from PRs #21-#23).
    - Coloured logging + idempotent steps + smoke-specific exit
      codes (7 = hub deploy fail, 8 = roundtrip verification fail).

  * tests/pkg/test_live_smoke_hermetic.py (NEW):
    - Mirrors the bash script's Phase 3-5 in-process against a
      file-mode catalog with mocked cosign so CI exercises the
      chain without acc1 reachability.
    - 7 tests: build determinism, end-to-end install + load,
      idempotent re-install, signing-floor refusal, --allow-unsigned
      bypass, PROPOSE_INFUSE shares the same fetch_and_install seam,
      and smoke script wiring sanity (script references the right
      helpers).

  * tools/SMOKE.md (NEW):
    - Operator runbook: prerequisites, run command, what each
      phase does, exit codes, troubleshooting matrix.

Test growth: 2979/37 (PR #27 baseline) -> 422/1 pkg suite (this PR
adds +7 hermetic tests on top of the operator-only script).  Full
sweep impact is +7 (since #28 was Go-only, no Python tests).

Stage 1 close-out — every code path the eight sub-slices ship is
now exercised by a single hermetic test that proves they compose
correctly:

  Build (#20) -> Sign (#27) -> Publish (#27) ->
  Catalog resolve (#20) -> Verify (#20 + #26) ->
  Install (#20) -> Registry (#20) -> RoleLoader (#21) ->
  PROPOSE_INFUSE dispatch (#24) all hit the same code path.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant