Skip to content

fix: Show elapsed time and token count in REPL spinner row - #6

Merged
agentforce314 merged 1 commit into
mainfrom
fix/spinner-elapsed-tokens
Apr 29, 2026
Merged

fix: Show elapsed time and token count in REPL spinner row#6
agentforce314 merged 1 commit into
mainfrom
fix/spinner-elapsed-tokens

Conversation

@agentforce314

@agentforce314 agentforce314 commented Apr 29, 2026

Copy link
Copy Markdown
Owner

Summary

  • Default Python REPL spinner now renders (esc to interrupt · 12s · ↓ 1.2k tokens) to match typescript/src/components/Spinner/SpinnerAnimationRow.tsx (timer + tokens gated by the same 30s threshold as TS).
  • Two upstream bugs were dropping per-turn token usage on the floor — both fixed so /stats is also accurate now:
    • src/query/query.py:269 — AssistantMessage was constructed without usage=response.usage.
    • src/providers/openai_compatible.py — streaming requests didn't pass stream_options={"include_usage": True}, so OpenAI's SDK never emitted a usage chunk.
  • New src/utils/format.py ports formatDuration / formatNumber from typescript/src/utils/format.ts, including JS-style round-half-up so the Python output matches byte-for-byte.

Files changed

  • src/utils/format.py (new) — duration / number formatters.
  • src/repl/live_status.py — LiveStatus tracks elapsed time + exposes set_tokens(n); paused() preserves the timer.
  • src/repl/core.py — chat() accumulates per-turn tokens from AssistantMessage.usage and pushes them to the spinner.
  • src/query/query.py — pass usage through to AssistantMessage.
  • src/providers/openai_compatible.py — opt streaming calls into include_usage.
  • tests/test_format.py (new) — 30 parity tests for the new formatters.

Test plan

  • pytest tests/test_format.py (30 passed)
  • Manual: python -m src.cli --dangerously-skip-permissions, send a long prompt; after ~30s the bottom row shows ⠋ Thinking… (esc to interrupt · 42s · ↓ 1.2k tokens) and ticks live.
  • Confirmed token portion populates against an OpenAI streaming model (was stuck at 0 without the include_usage opt-in).

🤖 Generated with Claude Code

@agentforce314 @claude
fix: Show elapsed time and token count in REPL spinner row
6f29564 
Mirrors typescript/src/components/Spinner/SpinnerAnimationRow.tsx so the
default Python REPL renders ``(esc to interrupt · 12s · ↓ 1.2k tokens)``
under the active spinner once the 30s threshold is crossed, instead of
the previous static ``(esc to cancel · enter to queue)`` hint.

Three coupled fixes were required to flow per-turn token usage end-to-end:

- src/utils/format.py: new module porting ``formatDuration`` /
  ``formatNumber`` from typescript/src/utils/format.ts, including the
  JS-style round-half-up for the seconds field so output matches.
- src/repl/live_status.py: ``LiveStatus`` now tracks its own start time
  and exposes ``set_tokens(n)``. The spinner suffix is rebuilt every
  frame from the running elapsed and token total. ``paused()`` snapshots
  and restores the timer so foreground prompts don't reset it.
- src/repl/core.py: the ``chat()`` engine loop accumulates per-turn
  input+output tokens from each AssistantMessage.usage and pushes the
  running total to the spinner. ``self._stats_*`` cumulative counters
  are unchanged.
- src/query/query.py: ``AssistantMessage`` was being constructed without
  the ``usage`` field, dropping per-turn token totals on the floor.
  ``/stats`` was silently broken by the same omission.
- src/providers/openai_compatible.py: streaming requests now pass
  ``stream_options={"include_usage": True}`` so the OpenAI SDK emits a
  final usage chunk; without it ``chat_stream_response`` always
  returned ``usage={}`` and downstream consumers saw zero tokens.

Tests: new tests/test_format.py covers ``format_duration`` /
``format_number`` parity with the TS reference (30 cases, all green).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@agentforce314
agentforce314 merged commit f53573d into main Apr 29, 2026
@ericleepi314 ericleepi314 mentioned this pull request May 10, 2026
Merged
4 tasks
ericleepi314 added a commit that referenced this pull request May 11, 2026
@ericleepi314
Merge pull request #70 from agentforce314/feat/ch03-state-6-session-s…
e3dcdce 
…tart

ch03 state #6: session-start wiring for 1h cache eligibility
@agentforce314 agentforce314 mentioned this pull request Jun 10, 2026
Merged
peroxider pushed a commit to peroxider/clawcodex that referenced this pull request Jun 15, 2026
@agentforce314 @claude
feat(workflow): result delivery, resume location, retry, worktree, in…
d3eb41c 
…tegration test

Engine/runner/task polish (port-plan §10 remaining items, non-TUI):

- agentforce314#7 Result delivery: enqueue_workflow_notification delivers a run's terminal
  result to the model via the shared <task-notification> queue (notified guard).
- agentforce314#8 Run-file location: journals live under ~/.clawcodex/transcripts/workflows/
  (per-user session storage) via get_workflow_run_path, not the project tree.
- agentforce314#3 retry_workflow_agent: engine retry loop re-spawns a running agent (bounded);
  also fixes a latent bug where a single-agent skip propagated and aborted the run.
- agentforce314#4 ProgressTracker fed to finalize_agent_tool for accurate token totals.
- agentforce314#5 isolation="worktree": per-agent wf_<runId>-<idx> git worktrees, best-effort.
- agentforce314#6 Live integration test (fake provider) driving LiveAgentRunner -> run_agent ->
  the real query loop. It caught two real bugs, fixed here: tool DISPATCH resolves
  by name from the registry (so schema agents need a per-call registry where
  StructuredOutput is the validating tool), and the injected StructuredOutput was
  permission-blocked in the subagent (now explicitly allowed).

Full workflow suite: 144 passing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
peroxider pushed a commit to peroxider/clawcodex that referenced this pull request Jun 26, 2026
chadwweng MiniMax-M3 chadweng
fix(orchestrator): correct progress_sink import path for _task_update…
414edb2 
…_call

The `_task_update_call` function lives in `clawcodex_ext/tool_system/tools/tasks_v2.py`,
not `src/tool_system/tools/tasks_v2.py`. The wrong import caused every progress
heartbeat to log a warning and fail silently:

    WARNING progress_sink Failed to update task metadata for task X:
      cannot import name '_task_update_call' from 'src.tool_system.tools.tasks_v2'

This silent failure meant `query_runner.heartbeat` saw `seconds_since_last_event`
accumulate indefinitely, which triggered premature `exit_code=124` timeouts on
otherwise-healthy agent runs (e.g. agentforce314#4/agentforce314#6/agentforce314#7 had used 17/19/42 tools respectively
when killed). Fixing the import restores progress reporting so the heartbeat sees
real activity and lets agents finish their work.

This is an `extensions/`-layer fix — no upstream `src/` change — preserving the
Decoupling Mandate.

Co-Authored-By: MiniMax-M3 <MiniMax-AI@claude-code-best.win>
peroxider pushed a commit to peroxider/clawcodex that referenced this pull request Jun 26, 2026
chadwweng
feat(F-68): complete Feature Gate runtime toggle system with YAML con…
6690473 
…fig, dependency resolution, and hook integration

- Extend ConfigStore with YAML support and flexible config-file resolution
- Add circular dependency detection (detect_circular_deps) to FeatureRegistry
- Add transitive dependency resolution (get_transitive_deps, resolve_all_deps)
- Add full validation (validate_all) for deps, mutex, self-references
- Integrate feature-gated hook calls in query.py (_call_hooks_if_enabled)
- Gate HOOK_PRE_LLM and HOOK_POST_LLM phases behind feature flags
- Update workspace metadata for issue agentforce314#6
peroxider pushed a commit to peroxider/clawcodex that referenced this pull request Jun 26, 2026
chadwweng
feat(feature_gate): add src/services/feature_gate re-export shim
13ff0d6 
Add the upstream-facing re-export shim at src/services/feature_gate/__init__.py
so that Layer 0 (src/) code can import from the expected namespace without
duplicating the canonical Layer 1 implementation in clawcodex_ext/feature_gate/.

This completes the F-68 Feature Gate runtime toggle system integration:
- FeatureRegistry singleton with override/env/config/default resolution
- @feature_gated / @feature_gated_class decorators
- JSON/YAML config persistence via ~/.clawcodex/features.json
- CLI subcommand: clawcodex feature <list|get|set|reload|reset>
- Dependency and mutex validation with circular-dependency detection
- 114 tests passing

Refs: agentforce314#6
peroxider pushed a commit to peroxider/clawcodex that referenced this pull request Jul 2, 2026
chadwweng
!52 merge feat-multi-agent-mvp into dev-decoupling-refactor-0573f4c
889d232 
feat(orchestrator): enable multi-agent coordinator mode in headless flow

Created-by: qq_49552963
Commit-by: yeyunyao
Merged-by: chadwweng
Description:   - Orchestrator 新增支持 multi-agent 系统：完成。
  - 支持多种 agent 协作模式：完成，已实现并验证 single / pipeline / coordinator / debate。
  - 由 code agent 结合具体 issue 判断选择哪种模式：完成，分两类：
      - 有 mode:* label 时按 label 显式选择。
      - 没有显式 label 时走 LLM router，读取 issue 标题/描述后选择模式，并记录 reason。
   - `pipeline`：按多个 stage 顺序运行不同 agent，并通过 mailbox handoff 传递上下文
  - `coordinator`：由 coordinator 拆解任务并派发 worker agent，worker 的真实 Git diff 会被主
  session 识别并进入 MR 流程
  - `debate`：多个 proposer agent 并行生成方案，再由 judge agent 汇总决策
  - `router`：无显式 `mode:*` label 时，由 LLM router 根据 issue 内容自动选择模式


  代码改动范围：

  - 新增协作模式框架：
    - `single`
    - `pipeline`
    - `coordinator`
    - `debate`
    - LLM router / mode selector
  - 扩展 workflow 配置解析，支持：
    - enabled modes
    - default mode
    - pipeline stages / stage models / stage max turns
    - mailbox handoff
    - nested debate stage
    - debate proposer / judge / isolation 配置
    - 根据 issue label 或 router 选择 collaboration mode
    - coordinator 模式下启用工具过滤
    - pipeline / debate / coordinator 都走统一 session / registry / git sync 流程
  - 修复 coordinator 收尾判定：
    - worker 通过 Agent 工具改文件后，主 coordinator session 能识别真实 Git diff
    - 避免继续空转，能正常进入 git sync / MR 创建
  - 修复 headless prompt split 兼容问题：
    - workflow 没有 user-message marker 时，完整 prompt 会作为 user prompt 传入
    - 避免 headless 报 `no prompt provided`
  - 修复 orchestrator logging import 路径问题：
    - daemon 真启动轮询 GitCode issue 时不再因为 `logging_setup` 相对导入失败而崩溃
  - 补充/扩展 orchestrator 相关单测，覆盖 mode selection、pipeline、debate、coordinator、
  prompt builder、agent runner 等路径。


  真实 GitCode E2E：
  ![image.png](https://raw.gitcode.com/user-images/assets/9901179/636ed12f-74b6-4d4a-805b-03da5f4c30f6/image.png 'image.png')
  打开 `https://gitcode.com/qq_49552963/click/issues?state=open`。**5 个新 issue 都在最上面**，标题都带 `[E2E-R2 F1-single]` / `F2-pipeline` / `F3-coordinator` / `F4-debate` / `F5-router` 前缀。所有 issue 都挂有 `e2e-round-2` 标签 —— daemon 的 `require_any_labels: [e2e-round-2]` 就靠它把老 6 个测试 issue 隔离掉。

  - F1 single：completed，验证单 agent 基线处理和 MR 创建
  ![image.png](https://raw.gitcode.com/user-images/assets/9901179/4e7b3580-034f-491b-b0ca-6eb621670b10/image.png 'image.png')
  打开 `agentforce314/issues/7`。这是**最简单的场景**：F1 要求把 `click.termui.prompt()` docstring 里一个残缺的句子补完。标签 `e2e-round-2` + `mode:single` 显式指定单 agent 模式。issue 描述里明确写了 "Expected collaboration mode: single"——daemon 应该派 SingleModeRunner 一个 agent 处理。**截图下方能看到 daemon 发的"ClawCodex Run Summary"评论**——daemon 收到 issue 后自动回帖告诉你"我收到了"。
  
  
  - F2 pipeline：completed，验证 3 段 pipeline、stage model/max turn、mailbox handoff、nested
  debate stage
  ![image.png](https://raw.gitcode.com/user-images/assets/9901179/889d8e38-7a68-4db4-8fd6-46e2bb663949/image.png 'image.png')
  打开 `agentforce314/issues/8`。**这一个 issue 就演示 6 个功能**：
1. Pipeline 3 段（analyzer → implementer → tester）
2. `stage_models`：analyzer 用 deepseek-chat，implementer 用默认 deepseek-v4-flash，tester 又切回 deepseek-chat
3. `stage_max_turns`：8 / 6 / 4
4. `handoff: mailbox`：段之间通过 `.clawcodex/team.json` 传消息
5. **`stage_specs.implementer.kind: debate`**：中间那段本身就是 Debate（★ 嵌套派发）
6. Debate 内部又有 conservative + bold + synthesize judge
看下面的评论——**daemon 已经把这个 issue 跑完并链接了 MR agentforce314#7**。


  - F3 coordinator：completed，验证 coordinator mode、工具过滤、worker 派发、worker diff 被主
  session 识别、git sync、MR 创建
  ![image.png](https://raw.gitcode.com/user-images/assets/9901179/bcae1544-a8fb-43a7-934a-91bba7f97b68/image.png 'image.png')
  打开 `agentforce314/issues/9`。要求把散落在 termui.py / _termui_impl.py / style.py 的 ANSI 转义序列提取到新模块 `_ansi.py`。这是**跨多个文件的重构**——单 agent 干不动，coordinator 应该拆解成 (a) grep 所有用法 (b) 建新模块 (c) 逐 caller 改。截图能看到 daemon 已开始处理但 6-turn 太紧没完成。
  
  - F4 debate：completed，验证 proposer 并行、worktree 隔离、judge synthesize
  ![image.png](https://raw.gitcode.com/user-images/assets/9901179/5f20d69b-048c-4fc9-a4d5-4cc8ef95ab9c/image.png 'image.png')
  打开 `agentforce314/issues/10`。**核心争议题**：Click 9.x 是否砍 Python 3.7？两派立场：
- **Pragmatist（砍）**：用现代语法，缩 CI matrix
- **Optimist（留）**：长尾用户还在 3.7

要求两个 proposer 独立争，judge synthesize 决定 + 出实际的 `setup.cfg` 改动。截图下方能看到 **多条 daemon 评论**：每个 proposer 一条 + 最后 judge 综合决定一条。**MR agentforce314#6 就是 judge 的产物**。


  - F5 router：completed，验证无显式 mode label 时由 LLM router 根据 issue 内容选择模式
  - 最终 coordinator 复验 issue agentforce314#17：completed
    - `session_end_reason=task_complete`
    - `verification_status=passed`
    - MR: https://gitcode.com/qq_49552963/click/merge_requests/11
  ![image.png](https://raw.gitcode.com/user-images/assets/9901179/91923101-465a-4951-8202-063ed41054df/image.png 'image.png')
  打开 `agentforce314/issues/11`。**故意不打 mode 标签**（只有 `e2e-round-2`），让 LLMRouter 读 issue 描述自己判断。描述里说的是 "docstring 里 fg= 例子和 foreground 描述不一致，选一个改一下"——这是明显的单文件 docstring 修复。**router 判定结果**："single, reason=Simple docstring inconsistency in one file" ← 后面会在 registry.json 里看到这句话。

  结论：multi-agent modes 的核心调度、模式选择、pipeline/debate/coordinator 执行、worker 改动
  识别、Git sync 和真实 GitCode MR 创建链路均已验证通过。
# 功能验收表（每项都有 log 行证据）

| 功能                               | 证据（events-summary.log 原始行）                                                                                                                                      |
| -------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **repo_tracker 接 GitCode 真 API** | `httpx GET https://api.gitcode.com/api/v5/repos/qq_49552963/click/issues`                                                                                       |
| **daemon 真发评论回 issue**           | `httpx POST https://api.gitcode.com/api/v5/repos/qq_49552963/click/issues/10/comments`                                                                          |
| **daemon 真 push 分支**             | `remote: ... clawcodex-e2e2/10-... -> ...`                                                                                                                      |
| **daemon 真开 MR**                 | registry.json / 补跑 registry 里共 5 条 `pr_url`：MR agentforce314#6 / agentforce314#7 / agentforce314#8 / agentforce314#9 / agentforce314#10                                                                                          |
| **require_any_labels 过滤**        | 老 issue agentforce314#1~6 全部 `skip_issue_require_any have=<none>`                                                                                                            |
| **4 mode 都注册**                   | daemon 启动时 4 行 `Collaboration mode registered`                                                                                                                  |
| **Label 派对**                     | agentforce314#7~10 → `source=label`                                                                                                                                          |
| **Single（基线）**                   | agentforce314#7 走 SingleModeRunner；补跑 agentforce314#12 completed 并开 MR agentforce314#8                                                                                                                 |
| **Pipeline: 3 段**                | agentforce314#8 `stage=analyzer/implementer/tester` 依次 starting/finished                                                                                                     |
| **Pipeline: stage_models**       | agentforce314#8 `overriding workflow.agent.model deepseek-v4-flash → deepseek-chat`                                                                                          |
| **Pipeline: stage_max_turns**    | agentforce314#8 `overriding agent_runner.max_turns 6 → 8` (analyzer) / `6 → 4` (tester)                                                                                      |
| **Pipeline: mailbox handoff**    | agentforce314#8 4 次 `wrote .clawcodex/team.json for mailbox handoff (3 members)`                                                                                             |
| **★ Pipeline 嵌套 Debate**         | agentforce314#8 `stage=implementer starting (..., kind=debate ...)`                                                                                                          |
| **Coordinator 切换**               | agentforce314#9 `enabling coordinator_mode (was=False)` + `restored coordinator_mode=False`；补跑 agentforce314#15 completed 并开 MR agentforce314#10；最终复验 agentforce314#17 completed 并开 MR agentforce314#11                          |
| **Coordinator 工具过滤**             | agentforce314#9/agentforce314#15/agentforce314#17 `tool registry filtered to 6 tools: ['Agent','Read','SendMessage','TaskStop','WebFetch','WebSearch']`；agentforce314#15/agentforce314#17 worker 通过 `Agent` 实际 Edit 后进入 Git sync |
| **★ Debate 真并发**                 | agentforce314#10 `starting 2 proposers in PARALLEL` + pragmatist/optimist 同时启动时间戳                                                                                            |
| **Debate worktree 隔离**           | agentforce314#10 `.debate-worktree-10-pragmatist` + `.debate-worktree-10-optimist` 两个物理目录                                                                                    |
| **Debate judge_model 覆盖**        | agentforce314#10 `Debate judge: temporarily overriding workflow.agent.model deepseek-v4-flash → deepseek-chat`                                                               |
| **Debate judge_mode=synthesize** | daemon 启动时注册 `judge_mode=synthesize`                                                                                                                            |
| **★ LLMRouter 真 LLM 决策**         | agentforce314#11 `collaboration_mode=single (source=router, reason=LLMRouter: Simple docstring inconsistency in one file)`；补跑 agentforce314#14 completed 并开 MR agentforce314#9                         |






See merge request: chadwweng/clawcodex!52
singlaamitesh pushed a commit to singlaamitesh/clawcodex that referenced this pull request Jul 7, 2026
@agentforce314
Merge pull request agentforce314#6 from agentforce314/fix/spinner-ela…
fb2ba99 
…psed-tokens

fix: Show elapsed time and token count in REPL spinner row
singlaamitesh pushed a commit to singlaamitesh/clawcodex that referenced this pull request Jul 7, 2026
@ericleepi314
Merge pull request agentforce314#70 from agentforce314/feat/ch03-stat…
6e95e43 
…e-6-session-start

ch03 state agentforce314#6: session-start wiring for 1h cache eligibility
singlaamitesh pushed a commit to singlaamitesh/clawcodex that referenced this pull request Jul 7, 2026
@agentforce314 @claude
feat(workflow): result delivery, resume location, retry, worktree, in…
f481fe7 
…tegration test

Engine/runner/task polish (port-plan §10 remaining items, non-TUI):

- agentforce314#7 Result delivery: enqueue_workflow_notification delivers a run's terminal
  result to the model via the shared <task-notification> queue (notified guard).
- agentforce314#8 Run-file location: journals live under ~/.clawcodex/transcripts/workflows/
  (per-user session storage) via get_workflow_run_path, not the project tree.
- agentforce314#3 retry_workflow_agent: engine retry loop re-spawns a running agent (bounded);
  also fixes a latent bug where a single-agent skip propagated and aborted the run.
- agentforce314#4 ProgressTracker fed to finalize_agent_tool for accurate token totals.
- agentforce314#5 isolation="worktree": per-agent wf_<runId>-<idx> git worktrees, best-effort.
- agentforce314#6 Live integration test (fake provider) driving LiveAgentRunner -> run_agent ->
  the real query loop. It caught two real bugs, fixed here: tool DISPATCH resolves
  by name from the registry (so schema agents need a per-call registry where
  StructuredOutput is the validating tool), and the injected StructuredOutput was
  permission-blocked in the subagent (now explicitly allowed).

Full workflow suite: 144 passing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
peroxider pushed a commit to peroxider/clawcodex that referenced this pull request Jul 21, 2026
chadwweng
!107 merge SOP-tool3 into dev-decoupling-refactor-0573f4c
f39ce6c 
feat(SOP):resource catalog,composite tools与SDK隔离实现

Created-by: gkawuiq8
Commit-by: gkawuiq8;root
Merged-by: chadwweng
Description: ## 背景

  SOP Converter 将 SDK 接口转换为独立 Tool 后，创建型工具产生的内存对象无法跨 Tool 调用复用，导致“创建 Agent → 按 ID 调用”的生命周期链路断裂。同时，复合宏工具与 SDK 原子工具存在语义竞争，Agent 即使能够执行宏，也不一定能稳定检索并选择宏。

  本 PR 串联 F-55、F-56、F-57 和 F-157，补齐从资源创建、持久化、恢复、复合执行到 ToolSearch 分层选择的完整链路。
  
## 主要改动
### F-55：补齐工具生命周期依赖
让「创建 → 保存 → 再调用」变成系统认的固定套路，而不是靠模型自己猜下一步该调哪个工具。
  - 为 create/invoke 工具建立 `resource_type`、`produces`、`consumes` 和 `resource_ref` 契约。
  - 创建成功后写入可恢复记录，并通过 `created_persisted` 等稳定字段区分创建与持久化状态。
  - 为 invoke/run 工具增加 catalog fallback，资源不在当前进程时可自动查找并恢复。
  - 生成 `.clawcodex/tool-dependencies.yaml`，记录 create → persist → materialize → invoke 依赖链。
  - 将生命周期依赖写入 Skill frontmatter、Task Guide 和系统提示。
  - ToolSearch 接入生命周期排序与 `lifecycle-chain:` 查询，减少相似 SDK 工具之间的误选和空转。

 ### F-56：实现通用 SOP Resource Catalog
  给「创建出来的 Agent / 资源」做一个可跨会话复用的通讯录——按名字或 ID 能查到，敏感信息只记环境变量引用，不落明文 Key。

  - 新增 `ResourceRecord`、`ResourceCatalog` 和 `CatalogExecutionContext`。
  - 支持 bundle-local/user-local catalog、原子写入、幂等 upsert 和 legacy AgentCatalog 兼容。
  - 敏感配置仅保存 `env:` 引用，避免 API Key 明文落盘。
  - 新增 `ResourceHandler` 注册表，以 `resource_type` 作为扩展点；Agent 是首个生产级实现。
  - 支持按 ID、名称和 alias 查找资源，歧义时拒绝猜测。
  - 实现 Agent 的 materialize/invoke 恢复路径，并统一缺失、歧义、secret 缺失、版本不兼容等错误码。
  - 支持 `.clawcodex/resources.yaml` sidecar 显式覆盖资源类型和句柄字段。

 ### F-57：可执行复合工作流 + 手写宏
 把「多步流水线」做成真正可调用的一种 Tool（`workflow`），并支持你在仓库里手写 YAML 宏（例如文本/图像/多模态处理流水线）；Agent 一次调用宏，系统按步骤顺序跑完，而不是让模型自己拼一串原子工具。

#### 复合工作流运行时

- `workflow` 成为一等 Tool 调用类型，由主进程 `ToolRegistry` 调度。
- 支持顺序 step、输入输出绑定、资源绑定、递归深度限制、延迟工具激活，以及结构化 `trace`（哪一步成功/失败一目了然）。
- trusted private lane：Agent 实例等不能 JSON 化的对象不进入公开 Tool 输出。
- 内置宏 `invoke-existing-agent` 固定三步：`load_agent_record → materialize_agent → invoke_agent`（对应「从目录取出 → 在本进程恢复 → 真正调用」）。
- 完善 output schema、类型重建与 JSON-safe 输出，保证宏返回值可被后续步骤和下一次 Tool 消费。

#### 手写宏

除了 convert 自动生成的工具，你还可以在 `sop-macros/` 里手写「业务宏」YAML；convert 时装进 bundle，运行时能被 ToolSearch 召回并一键执行。

- 支持从源码树 `sop-macros/`、`--macros-dir` 或 `--macro-manifest` 加载手写宏定义。
- convert 阶段完成 schema 校验、原子写入 bundle（如 `.clawcodex/macros/`），并注册为 `call_type=workflow` 的可调工具。
- 宏可声明对外入参（如 `input_path` / `output_path`）、内部 step 编排（如调用 `execute-pipeline`），以及对外输出契约（如非空 `session_id` + `summary`）。
- 与 MacroCatalog / MacroRoute 打通：自然语言或 `select:宏名` 可稳定命中手写宏，而不是落到裸原子工具。
- 验收侧典型路径：AscendDataForge 文本/图像/多模态手写宏 → ToolSearch 命中 → 一次 `tool_use` 跑完整流水线 → 产物落盘。

 ### F-157：实现宏工具/原子工具分层检索
 搜索工具时「能走整包宏就不要拆零件」——有合适宏就优先（甚至独占）展示宏；宏不可用时再把原子工具放回来，避免 Agent 被一堆相似小工具带偏。

- 宏路由增加 `intent_key`、`covered_tools`、`unavailable_policy`，标明「这个宏覆盖了哪些原子工具」。
- `sop convert` 生成 `.clawcodex/tool-retrieval.yaml`，显式记录 macro / atomic / neutral 层级与覆盖关系。
- `RetrievalPlan` 支持 verified exclusive、prefer、普通检索三种策略。
- exclusive 命中且预检通过后，从搜索结果和当前可用工具里隐藏被覆盖的原子工具。
- 宏预检失败时，同一次 ToolSearch 内撤销隐藏并恢复原子候选，避免「宏挂了又搜不到退路」。
- 若仍去调已被宏盖住的陈旧原子工具，返回 `tool_shadowed_by_macro`，引导改调推荐宏。
- 评分接近时优先同意图宏，同时保留「用户精确点名原子工具」时的优先级。
- ToolSearch 结果带上 intent、selection、被抑制工具、preflight、reason codes，方便对照验收。

 ### SDK 依赖隔离与转换健壮性
 convert 时给每个 bundle 准备独立 venv 并装好 SDK 依赖；真正跑 Tool 时不切换进程解释器（避免把 Agent/REPL 进程 exec 掉），而是把 bundle 的 site-packages 挂进当前进程，形成「软隔离」。
  - 为转换后的 bundle 建立独立虚拟环境并解析 SDK requirements。
  - 支持 Windows/WSL 路径、模块导入、wrapper 重入和 bundle 上下文传播。
  - 完善 CLI handler、异步接口、复杂参数 schema 和 SDK 类型恢复。

 ## 核心链路

  ```text
  create resource
    → ResourceCatalog upsert
    → ToolSearch 召回复合宏
    → catalog lookup
    → materialize
    → invoke
    → 返回 JSON-safe output + workflow trace
  ```

  ## 验证

  聚焦回归结果：

  164 passed, 8 subtests passed

  覆盖范围包括：

  - F-55 生命周期排序、catalog fallback 和 create → invoke E2E。
  - F-56 catalog、secret 引用、资源类型扩展及 Agent 重建。
  - F-57 workflow dispatch、宏装载、参数绑定、trace 和 output schema。
  - F-157 exclusive suppression、preflight 回滚、shadow guard、结构偏置和索引校验。

  手工验证已确认 Agent 创建后写入 catalog，以及 invoke-existing-agent 完成
  load → materialize → invoke 三步调用。F-157 自然语言选择效果仍需按手工验收清单复验。

## 兼容性与边界

  - 保留 legacy AgentCatalog 双写及读取 fallback。
  - 历史未分类工具默认为 neutral，不会因名称相似被隐藏。
  - 当前生产级资源恢复保证以 Agent 为主；第二种真实 SDK 资源和通用 resume-resource 宏不在本 PR 范围。
  - Session 宏注册、trace-to-macro 和宏提升属于后续可选能力。
  - 外部 Ray/worker 是否使用 bundle 解释器仍受目标 SDK 运行环境影响，需单独进行端到端验收。


## 一、F-56，F-57特性验证

会话根目录：

`\\wsl$\Ubuntu-18.04\root\.clawcodex\sessions\3efca96f-4e92-4580-9506-8f479371b255\`

落盘 catalog：

`D:\projects\clawcodex\.clawcodex\JiuwenAgent_v7.18\.clawcodex\resource-catalog.json`

### 1) F-56：创建并写入 catalog

**文件：** `subagents\agent-ark8z37bg.jsonl`

| 行号（约） | 流程 | 详情 |
|-----------|--------|----------|
| L0 | `tool_use` Skill | `core_merged-skill` |
| L4 | `tool_use` ToolSearch | `select:openjiuwen-core-application-llm-agent-create-llm-agent` |
| L7 | `tool_use` create 工具 | `openjiuwen-core-application-llm-agent-create-llm-agent`，`agent_config.id=verify-bot` |
| **L9** | **`tool_result`（核心）** | 见下表字段 |

**L9 `tool_result` 里的 F-56 字段：**

```text
created_persisted = true
resource_catalog_reason = "f56_resource_catalog"
resource_catalog_path = ".../JiuwenAgent_v7.18/.clawcodex/resource-catalog.json"
catalog_reason = "bundle-local"
agent_id = "verify-bot"
resource_ref = "verify-bot"
callable_by_agent_id / callable_by_resource_ref = true
*_call_contract = "catalog_persisted"
```
![image-89.png](https://raw.gitcode.com/user-images/assets/9901179/6dcdd193-8ec6-41a3-b1c7-e2e1ab304439/image-89.png 'image-89.png')

**落盘对照：** 打开 `resource-catalog.json`，`records` 中有 key 含 `verify-bot`，且：

- `resource_id`: `verify-bot`
- `source_tool`: `openjiuwen-core-application-llm-agent-create-llm-agent`
- `materializer` 含 `init_kwargs.agent_config`（api_key 应是 `env:DEEPSEEK_API_KEY`，不是明文）

![F-56+57测试2.PNG](https://raw.gitcode.com/user-images/assets/9901179/94cce083-a48c-4922-b2ba-93d5645a9d5f/F-56_57测试2.PNG 'F-56+57测试2.PNG')

### 2) F-57：宏召回 + catalog→materialize→invoke

**文件：** `subagents\agent-a6d96va9i.jsonl`

| 行号（约） | 流程 | 详情 |
|-----------|--------|----------|
| L0 | Skill | `core_merged-skill` |
| L4 | ToolSearch | `select:invoke-existing-agent` |
| **L6** | ToolSearch 结果 | `matches: ["invoke-existing-agent"]`（宏被召回，不是原子 `llmagent-invoke`） |
| L7 | 调宏 | `invoke-existing-agent`，入参 `agent_ref=verify-bot`, `query=ping` |
| **L9** | **宏结果（核心）** | workflow `steps` 三步全 success |

![image-88.png](https://raw.gitcode.com/user-images/assets/9901179/9755b6b5-217a-4e9d-b273-ccf2f669dfde/image-88.png 'image-88.png')

![image-90.png](https://raw.gitcode.com/user-images/assets/9901179/1cad3ab1-e03e-4b6b-8412-c770f0cb1470/image-90.png 'image-90.png')

![image-91.png](https://raw.gitcode.com/user-images/assets/9901179/2dfc8fca-9b59-4d8e-bd89-bc0ab4ccd686/image-91.png 'image-91.png')

**L9 `steps`（F-57 主路径）：**


```text
load_agent_record   kind=catalog  status=success
materialize_agent   kind=python   status=success
invoke_agent        kind=python   status=success
agent_id = verify-bot
```
![image-92.png](https://raw.gitcode.com/user-images/assets/9901179/3874ad3b-b4b3-4b8a-9686-ba5d73d79e92/image-92.png 'image-92.png')

父会话 `transcript.jsonl` 只能看到 overview 委派了 `Agent(core_merged-agent)`；**F-56/57 细节只在上述两个 subagent jsonl + resource-catalog.json**。

---

## 二、F-56 / F-57 验收对话用例

前置：`--agent` 指向已 convert 的 JiuwenAgent bundle；`DEEPSEEK_API_KEY` 已设置；每次新开 session 更清晰。

### A. 主路径（必过）

| ID | 用户输入 | 期望工具链 | 验收点（日志/文件） |
|----|----------|------------|-------------------|
| A1 | 用 JiuwenAgent SDK 创建名为 `verify-bot` 的 LLMAgent；provider=deepseek；api key 用 `env:DEEPSEEK_API_KEY`；model=`deepseek-v4-flash` | Skill → ToolSearch → `create-llm-agent` | `created_persisted=true`；`resource_catalog_reason=f56_resource_catalog`；catalog 文件有 `verify-bot`；密钥非明文 |
| A2 | 用 `verify-bot` 回复 `请原样输出：PING_OK_56`，把原文返回给我 | Skill → ToolSearch(`select:invoke-existing-agent` 或自然语言命中宏) → `invoke-existing-agent` | ToolSearch `matches` 含宏；steps=`load_agent_record→materialize_agent→invoke_agent` 全 success；output 含 `PING_OK_56` |
| A3 | （新开同 bundle 的新 session，不重建）只用已有 `verify-bot` 回复 `PING_CROSS_SESSION` 原文返回 | 同上，**禁止**再 create | 证明跨会话读 catalog（F-56§9 agentforce314#2） |

![F-56+57验收测试4.PNG](https://raw.gitcode.com/user-images/assets/9901179/d390563c-8296-4643-9a23-9b239a74004b/F-56_57验收测试4.PNG 'F-56+57验收测试4.PNG')

![image-93.png](https://raw.gitcode.com/user-images/assets/9901179/3205ee3b-9c90-41aa-b76f-e0dc7f6e826e/image-93.png 'image-93.png')

### B. 引用形态（F-56 契约 / F-57 兼容入参）

| ID | 用户输入 | 期望 | 验收点 |
|----|----------|------|--------|
| B1 | 用 resource_ref=`verify-bot` 调用，消息=`REF_OK`，原文返回 | `invoke-existing-agent`，`agent_ref` 或等价字段 | steps 全 success；output 含 `REF_OK` |
| B2 | 用 agent_id=`verify-bot` 调用，消息=`ID_OK`，原文返回 | 同上（legacy 兼容） | 成功，不落到 `llmagent-invoke` |

![image-94.png](https://raw.gitcode.com/user-images/assets/9901179/69b54d29-9d48-4166-b49d-17dbea0b2be3/image-94.png 'image-94.png')

![image-95.png](https://raw.gitcode.com/user-images/assets/9901179/3062b5dd-6637-4c85-89f8-8ed4aeb8cb4f/image-95.png 'image-95.png')

### C. 幂等 / 更新（F-56§9 agentforce314#4）

| ID | 用户输入 | 期望 | 验收点 |
|----|----------|------|--------|
| C1 | 再次创建同名 `verify-bot`（同配置） | create 成功或明确 upsert | catalog 仍一条主记录；可再 invoke |
| C2 | 创建后立刻 invoke 一次 | 宏成功 | 说明 upsert 未破坏 materializer |

![image-96.png](https://raw.gitcode.com/user-images/assets/9901179/22e678e1-4cd7-4680-9706-377a0e8affca/image-96.png 'image-96.png')

![image-97.png](https://raw.gitcode.com/user-images/assets/9901179/7e9ead6b-f37e-4696-a623-b5aac18d59b4/image-97.png 'image-97.png')

### D. 负向错误码（F-56§9 agentforce314#5/agentforce314#6 + F-57 透传）

| ID | 用户输入 | 期望 | 验收点 |
|----|----------|------|--------|
| D1 | 用 `no-such-agent-xyz` 回复 `ping` | `invoke-existing-agent` 失败 | `error_code` 含 `resource_catalog_missing`（或等价）；steps 在 catalog 步失败 |
| D2 | （若环境可构造重名）按模糊名调用 | 拒绝猜测 | `resource_catalog_ambiguous` |
| D3 | 临时去掉 `DEEPSEEK_API_KEY` 再 invoke | materialize/invoke 失败 | `resource_secret_missing` 或明确 secret 错误，**日志无明文 key** |

![image-98.png](https://raw.gitcode.com/user-images/assets/9901179/a96e3cab-3aec-4f23-b78e-195c1448eb91/image-98.png 'image-98.png')

### E. 宏 vs 原子工具（F-57 路由，不含 F-157 隐藏验收）

| ID | 用户输入 | 期望 | 验收点 |
|----|----------|------|--------|
| E1 | 调用已经创建的 Agent（例如让 verify-bot 回复 ping） | 优先 `invoke-existing-agent` | ToolSearch/实际调用是宏，不是 `llmagent-in

See merge request: chadwweng/clawcodex!107
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@agentforce314