Repository navigation
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 741fa88588
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| } else if (info.type == InstanceType::DECODE && | ||
| peer.role == InstanceType::PREFILL) { | ||
| result.emplace_back(info.rpc_address, *peer.info); |
There was a problem hiding this comment.
Send decode unlinks to the surviving prefill peers
When a dedicated DECODE instance is deregistered, deregister_instance() calls this with removing=true, but this branch still builds (info.rpc_address, *peer.info). The resulting UnlinkInstance is sent to the decode endpoint being removed and identifies the prefill as its peer; if that decode is failed or unreachable, the call fails and every surviving prefill retains the stale decode link. For removal, reverse the tuple so the surviving prefill receives the removed decode metadata.
Useful? React with 👍 / 👎.
重构目的
原来的
scheduler/同时承担选路、实例注册与健康管理、etcd watch、引擎连接、请求存储及输出处理,InstanceMgr和Scheduler的职责较集中。本 PR 按这些职责拆分模块,将scheduler/调整为router/,并把实例管理拆为cluster/与coordinator/,方便分别演进路由策略、实例控制和部署管理。这是阶段性的架构重构草案,先以 draft 形式讨论目录边界、依赖和生命周期。继续使用现有 CLI、HTTP/RPC 协议、etcd 注册格式与 RR/CAR/SLO_AWARE 策略;CAR 的评分公式保持不变。JSON/YAML 拉起引擎、进程托管、自动扩缩容和 Kubernetes 控制器尚未在本 PR 中实现。
目录与文件组织
以下路径均相对于
xllm_service/;.h/.cpp表示同名声明和实现文件。目录命名备选
当前代码采用
coordinator/和kv_conductor/,以下名字保留为评审时的备选:coordinator/membership/coordinator/etcd/coordinator/coordination/kv_conductor/kv_directory/kv_conductor/统筹 KV 元数据的订阅、恢复、索引和查询。内部保留KVDirectory、kv_index、kv_event_decoder等具体名称。配置中的discovery字段继续表示KV 事件端点的自动发现规则,与源码目录命名独立。
应用入口与配置
main.cppservice_app.h/.cppconfig/service_config.hcommon/options.h整理而来。config/config_loader.h/.cppServiceConfig,集中处理配置读取边界。router/:选路与请求生命周期router.h/.cpprouting_types.hrouting_policy.hRoutingPolicy接口。策略消费状态快照,不直接访问 etcd 或调用引擎。policies/round_robin.h/.cpppolicies/cache_aware_routing.h/.cpppolicies/slo_aware.h/.cpprouting_state.h/.cpprequest_manager.h/.cppRequestManager保留在router/,因为它持有服务端请求生命周期和调度预留;request/放请求数据、输入预处理和有序输出分发。cluster/:实例事实、控制和通信分离instance_types.hinstance_state.h/.cppinstance_controller.h/.cppinstance_client.h/.cppInstanceState记录引擎和注册系统提供的事实;RoutingState记录服务自己产生的预留与预测。二者独立,避免心跳更新覆盖服务侧预留,也避免实例状态锁承担路由策略计算。coordinator/:注册发现与服务成员etcd_client.h/.cppinstance_discovery.h/.cppservice_membership.h/.cpp请求、KV 索引与协议适配
request/request.hrequest/chat_message.hrequest/request_metrics.h、request_output.hrequest/request_preprocessor.h/.cpprequest/output_dispatcher.h/.cppkv_conductor/kv_event_decoder.h/.cppkv_conductor/kv_index.h/.cppkv_conductor/kv_directory.h/.cpphttp_service/service.h/.cpphttp_service/response_handler.h/.cppscheduler/移入,绑定客户端输出回调,构造 OpenAI/Anthropic 响应。http_service/chat_json_parser.*、anthropic_adapter.*、anthropic_stream_encoder.*、usage_proto_adapter.*rpc_service/service.h/.cppGenerations,分别转交实例控制、实例状态和请求管理模块。rpc_service/disagg_generation_adapter.h/.cpprpc_service/client.h/.cpp数据流与并发边界
迁移与文档
Scheduler拆为Router、RequestPreprocessor、RequestManager和OutputDispatcher。InstanceMgr拆为InstanceState、InstanceController、InstanceClient、InstanceDiscovery、ServiceMembership和RoutingState。master.*拆为main.cpp与service_app.*;各模块的 CMake 依赖同步调整。cluster/、router/、request/和http_service/重新归类,覆盖代际替换、准入/下线竞争、有序输出、预留释放与选路。docs/zh/architecture.md,更新中英文 README 的目录结构、KV events 文档和 sidecar 的源码引用。验证
git diff --check通过。2。