feat(tui): capability invocation telemetry on Performance screen - #15
Merged
Conversation
PR-B added an ``invocations`` field to the TASK_COMPLETE payload
(``[{"kind", "target", "ok", "error"}, ...]``) but nothing was
consuming it. This PR wires the field end-to-end: the NATSObserver
folds each entry into the CollectiveSnapshot, and the Performance
screen renders two new panels — running per-(kind, target) totals
plus a recent-failures tail — so operators see at a glance which
skills + MCP tools are firing, at what success rate, and (when the
rate drops) what the most recent failure was.
acc/tui/models.py
+ CapabilityInvocationStats dataclass — kind, target, total, ok,
last_error, last_seen_ts. Properties .fail and .ok_rate.
Unfired tools report ok_rate=1.0 so the panel doesn't accuse a
never-invoked tool of failing.
+ CollectiveSnapshot.capability_stats: dict[str, ...] keyed by
f"{kind}:{target}" so a skill named "echo" and an MCP tool also
named "echo" don't collide.
+ CollectiveSnapshot.invocation_log: FIFO tail capped at
_MAX_INVOCATION_LOG (50).
+ record_invocation(invocation, *, agent_id, task_id, ts) — folds
one entry into capability_stats + appends to invocation_log.
Drops malformed entries silently (unknown kind, missing target).
acc/tui/client.py
NATSObserver._route_task_complete now iterates data["invocations"]
and calls snapshot.record_invocation for each well-formed entry.
Pre-PR-B agents that omit the field are no-ops (no crash, no stats).
Non-list invocations field is also skipped safely.
acc/tui/screens/performance.py
+ CAPABILITY INVOCATIONS DataTable — Kind / Target / Total / OK% /
Last error. Sorted by total desc so the busiest tools surface.
Empty state shows a single grey hint pointing at
docs/howto-skills.md. OK% colour-coded:
≥ 95 % green
≥ 80 % yellow
else red
Kind cell colour-coded cyan (skill) / magenta (mcp).
+ RECENT FAILURES Static — last 10 failures from invocation_log
(successes filtered out). Format: "ts kind:target agent_id"
+ indented error line. Empty state shows a grey hint.
Both panels live in the existing right-column Vertical alongside
TOKEN BUDGET / LATENCY PERCENTILES.
acc/tui/help/performance.md
Two new §sections describing the panels, including the colour
legend + sort order + empty-state behaviour.
Tests (tests/test_capability_telemetry.py — NEW, 13 cases all green):
CapabilityInvocationStats (2):
initial state clean; fail + ok_rate track correctly.
CollectiveSnapshot.record_invocation (5):
creates stats on first call,
accumulates across calls + tracks last_error,
keys kind:target independently (skill echo vs mcp echo),
drops malformed entries silently,
invocation_log enforces FIFO cap at 50 (60-call overflow test).
NATSObserver._route_task_complete (3):
folds invocations into snapshot end-to-end,
handles missing invocations field (pre-PR-B compat),
handles non-list invocations safely.
PerformanceScreen Pilot (3):
capability table sorted by total desc,
failures panel lists only failures (success row excluded),
empty snapshot renders the guidance row.
Combined regression: 166 existing tests pass unchanged
(test_config, test_role_store, test_guardrails, test_compliance,
test_observer_task_listener, test_prompt_channel,
test_ecosystem_screen_pilot, test_file_picker_pilot;
TestEd25519Validation deselected for the lighthouse OpenSSL platform
limit, as in earlier PRs).
Out of scope (planned follow-ups):
* TASK_PROGRESS streaming for the prompt channels
(TUIPromptChannel.supports_streaming() flips True).
* Telegram + WhatsApp PromptChannel adapters.
* Real echo MCP server image + compose entry.
3 tasks
flg77
added a commit
that referenced
this pull request
May 1, 2026
…ier B) (#22) Tier B of the coding_agent test plan (Tier A — schema invariants — landed in #21). Verifies that the canonical TUI surfaces consume the ``coding_agent`` role correctly: * Ecosystem ROLE LIBRARY contains the row. * ROLE DETAIL panel renders the role's purpose + persona on selection. * SKILLS / MCP SERVERS tables list echo + echo_server. * "Schedule infusion → Nucleus" button dispatches RolePreloadMessage with role_name='coding_agent'. * Prompt pane (screen 7) defaults the target_role Select to coding_agent; Send publishes a TASK_ASSIGN with target_role + optional target_agent_id pinning. * Performance screen (PR #15) renders the CAPABILITY INVOCATIONS table + RECENT FAILURES panel from synthetic TASK_COMPLETEs sourced from a coding_agent-* agent_id, including A-017 block reasoning. tests/test_coding_agent_tui_pilot.py (NEW, 10 cases all green): Ecosystem (5): test_ecosystem_role_library_row_for_coding_agent → ROLE LIBRARY contains the coding_agent row keyed by name. test_ecosystem_role_detail_renders_coding_agent_seed → on RowSelected, ROLE DETAIL Static carries the role.yaml purpose phrase verbatim + the "analytical" persona. test_ecosystem_skills_table_shows_echo_for_coding_agent_role test_ecosystem_mcps_table_shows_echo_server_for_coding_agent_role → both confirm PR-A's path-resolution fix loads the repo's skills/ + mcps/ correctly even when the test runner's cwd differs from the repo root. test_schedule_infusion_button_dispatches_role_preload_for_coding_agent → Send button posts RolePreloadMessage(role_name='coding_agent') the App routes to InfuseScreen.preload_from_role. Prompt pane (3): test_prompt_pane_target_role_defaults_to_coding_agent → guards a regression where the default Select value drifts. test_prompt_send_routes_task_assign_to_coding_agent → Send publishes TASK_ASSIGN with target_role='coding_agent', no target_agent_id (broadcast-by-role behaviour). test_prompt_send_with_target_agent_id_pins_to_specific_coding_agent → operator-supplied target_agent_id makes it onto the wire so PR-B's filter on the agent side only lets the named agent process the task. Performance screen (2): test_performance_telemetry_records_coding_agent_skill_invocation → coding_agent emits a skill:echo invocation in TASK_COMPLETE → CAPABILITY INVOCATIONS table renders one row keyed ``skill:echo``. test_performance_failures_panel_renders_a_017_block_from_coding_agent → coding_agent attempts shell.exec → Cat-A A-017 blocks → RECENT FAILURES panel shows shell.exec + A-017 + the coding_agent agent_id (truncated). Implementation patterns reused from earlier PRs: * ``_capture_static_updates`` — PR-A trick to read Static.update calls across Textual versions. * ``_StubObserver`` (channel-shape) — PR-B harness for prompt-pane pilot tests. * Synthetic TASK_COMPLETE delivery via ``observer._route_task_complete`` — PR #15 telemetry test pattern. * Synthetic ``DataTable.RowSelected`` event — PR-A pattern, more deterministic than ``pilot.press("enter")``. Combined regression: 238 unit tests pass on Windows (test_coding_agent_role + test_coding_agent_tui_pilot + test_task_progress_emit + test_task_progress_streaming + test_prompt_screen_pilot + test_prompt_channel + test_observer_task_listener + test_redis_compat + test_capability_telemetry + test_ecosystem_screen_pilot + test_file_picker_pilot + test_config + test_role_store + test_guardrails + test_compliance, with TestEd25519Validation deselected for the lighthouse OpenSSL platform limit). Out of scope (next two follow-up PRs): * Tier C — Live container integration via the coding-split profile. Opt-in via ACC_LIVE_TESTS=1 + a running stack so CI doesn't hang on a missing NATS. * Tier D — Negative paths: missing role dir → graceful EcosystemScreen degradation, target_agent_id mismatch → drop, HIGH-risk skill blocked by A-017 ceiling.
flg77
added a commit
that referenced
this pull request
Jun 2, 2026
…9) (#15) Two focused follow-ups to v0.3.48: * **webgui startup warning** — `resolve_auth_config` now logs a loud WARNING when `ACC_WEBGUI_AUTH_MODE=htpasswd` AND the configured `ACC_WEBGUI_HTPASSWD_PATH` is missing / unreadable / unset. Triggered by today's lighthouse smoke: the operator's host-path env (`/home/flg/...`) was propagated into the container where the file was mounted at `/app/...`, every login silently 401'd, and the only signal in `podman logs` was uvicorn's normal startup. Now the operator sees the mismatch in the first log line after the auth mode is set. * **ssh_exec skill** — HIGH-risk sibling to shell_exec for verifying remote outcomes from inside an agent. `requires_actions: [execute_ssh]`, argv-only (shlex.join'd into the remote command), key-auth only (`BatchMode=yes`), host allowlist via `ACC_SSH_HOST_ALLOWLIST` (localhost always allowed), default `accept-new` StrictHostKeyChecking, 60s/600s timeout, output cap. Flipped onto the same 9-role engineering family that already has shell_exec. Tests: +14 across 2 new files; full sweep 2561 passed, 38 skipped, 0 failed. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
flg77
added a commit
that referenced
this pull request
Jun 29, 2026
…cryptography (#141) Resolves the Python (pip) Dependabot alerts on flg77/acc-spearhead. Most were a stale uv.lock (last generated 2026-06-14) whose pins lagged the already-permissive pyproject constraints; one needed a constraint widening. Lockfile refresh (constraints already allowed the fix — `uv lock --upgrade-package`): aiohttp 3.14.0 → 3.14.1 (#15-22, 8 alerts) pyjwt 2.12.1 → 2.13.0 (#9-13, 5 alerts) starlette 1.0.1 → 1.3.1 (#24-27, 4 alerts; transitive via fastapi) msgpack 1.1.2 → 1.2.1 (#37, high) joserfc 1.6.5 → 1.7.2 (#39; transitive via authlib) Constraint widening (fix was outside the pin): cryptography 46.0.7 → 48.0.1 (#14/#23, two HIGH) — pyproject `<47`→`>=48.0.1,<49`. The Ed25519 arbiter sign/verify API is stable across 46→48; operator "pin and bump" one major at a time. The lock also catches up on the speech/turbovec optional-deps added since 06-14 (faster-whisper/piper-tts/ctranslate2/onnxruntime/av/turbovec) — no security content, just lock/pyproject reconciliation. No downgrades. Not fixed here (separate handling): transformers (#1) — vulnerable `Trainer` not in our execution path; ST<5.0 blocks the 5.0.0rc3 fix → dismissed on GitHub with that reason (intent already in pyproject). torch (#7, low) — no patched release exists → dismissed "no fix available". npm console-plugin (7 alerts) — need npm/node (absent here) → fleet hand-off. Verified with the bumped libs installed: signatures/spiffe 153 ✓, a2a/messenger/ slack/webgui/redis 168 ✓ (321 total, 0 failures). Co-authored-by: flg <flg@acc1.ic3net.internal> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Wires the `TASK_COMPLETE.invocations` field PR-B added all the way to the operator's eye: the NATSObserver folds each entry into the CollectiveSnapshot, and the Performance screen renders two new panels — running per-(kind, target) totals + a recent-failures tail — so operators see which skills + MCP tools are firing, at what success rate, and what the latest failure was.
Architecture
```
Agent ── publishes ─▶ TASK_COMPLETE { …, invocations: [{kind,target,ok,error}, …] }
│
▼
NATSObserver._route_task_complete
│
▼
CollectiveSnapshot.record_invocation
├── capability_stats[k]: {total, ok, last_error, last_seen_ts}
└── invocation_log: FIFO tail (cap 50)
│
▼
PerformanceScreen.watch_snapshot
├── _render_capability_invocations → DataTable (sorted by total desc)
└── _render_capability_failures → Static (last 10 failures)
```
Files
Visuals
```
CAPABILITY INVOCATIONS (skill / MCP tool)
┌──────────┬───────────────────────┬───────┬──────┬──────────────────────────┐
│ Kind │ Target │ Total │ OK% │ Last error │
├──────────┼───────────────────────┼───────┼──────┼──────────────────────────┤
│ mcp │ echo_server.echo │ 142 │ 100% │ — │
│ skill │ echo │ 87 │ 98% │ schema mismatch │
│ mcp │ fs.read │ 23 │ 78% │ A-018 blocked │
└──────────┴───────────────────────┴───────┴──────┴──────────────────────────┘
RECENT FAILURES (latest 10)
14:32:18 mcp:fs.read coding-aabb
A-018 blocked tool 'fs.read'@'fs_server': mcp_server 'fs_server' not in role.allowed_mcps
14:31:55 skill:echo coding-aabb
schema mismatch: 'text' must be string
```
Tests (13 new, all green)
Combined regression: 166 existing tests pass unchanged (config + role-store + guardrails + compliance + observer + prompt-channel + ecosystem + file-picker; TestEd25519Validation deselected for lighthouse OpenSSL platform limit).
Test plan
Out of scope (planned follow-ups)