[APIServer][BugFix] Fix gunicorn fork deadlock and multi-dp pipe issue in get_save_output_v1

[liyonghua0910]

Motivation

💡 If this PR is a Cherry Pick, the PR title needs to follow the format by adding the [Cherry-Pick] label at the very beginning and appending the original PR ID at the end. For example, [Cherry-Pick][CI] Add check trigger and logic(#5191)

💡 如若此PR是Cherry Pick，PR标题需遵循格式，在最开始加上[Cherry-Pick]标签，以及最后面加上原PR ID，例如[Cherry-Pick][CI] Add check trigger and logic(#5191)

This PR fixes two issues related to process forking in the API server:

1. Gunicorn fork deadlock: When workers=1, gunicorn still forks a worker process, which can cause deadlock due to inherited resources (e.g., zmq sockets, CUDA contexts) not being fork-safe. When there is only one worker, uvicorn should be used directly to avoid the fork entirely.
2. Multi-DP pipe block/data loss in get_save_output_v1: Non-rank-0 token_processors skip receiving from their zmq pipes under multi-DP, causing pipes to block or lose data.

Modifications

• fastdeploy/entrypoints/openai/api_server.py:

  • Changed server launch logic: use StandaloneApplication (gunicorn) when workers > 1, use uvicorn.run directly when workers == 1, avoiding unnecessary fork that can lead to deadlock.
  • Added llm_engine._exit_sub_services() call in the lifespan shutdown path for the non-async branch, ensuring sub-services exit properly in sync mode.

• fastdeploy/output/token_processor.py:

  • Removed the rank_id-based conditional check in get_save_output_v1, so that all token_processors receive and process data from their respective zmq pipes regardless of rank, ensuring pipes are properly consumed in multi-DP scenarios.

Usage or Command

No extra configuration needed. Single-worker deployments will automatically use uvicorn instead of gunicorn.

Accuracy Tests

This PR does not modify model computation logic and does not affect accuracy.

Checklist

• Add at least a tag in the PR title.
  • Tag list: [[FDConfig],[APIServer],[Engine], [Scheduler], [PD Disaggregation], [Executor], [Graph Optimization], [Speculative Decoding], [RL], [Models], [Quantization], [Loader], [OP], [KVCache], [DataProcessor], [BugFix], [Docs], [CI], [Optimization], [Feature], [Benchmark], [Others], [XPU], [HPU], [GCU], [DCU], [Iluvatar], [Metax]]
  • You can add new tags based on the PR content, but the semantics must be clear.
• Format your code, run pre-commit before commit.
• Add unit tests. Please write the reason in this PR if no unit tests.
• Provide accuracy results.
• If the current PR is submitting to the release branch, make sure the PR has been submitted to the develop branch, then cherry-pick it to the release branch with the [Cherry-Pick] PR tag.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 30.00000% with 14 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@17b9039). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
fastdeploy/output/token_processor.py	0.00%	7 Missing ⚠️
fastdeploy/worker/metax_model_runner.py	0.00%	3 Missing ⚠️
fastdeploy/worker/xpu_model_runner.py	0.00%	3 Missing ⚠️
fastdeploy/entrypoints/openai/api_server.py	75.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             develop    #8088   +/-   ##
==========================================
  Coverage           ?   67.49%           
==========================================
  Files              ?      475           
  Lines              ?    66919           
  Branches           ?    10321           
==========================================
  Hits               ?    45169           
  Misses             ?    18870           
  Partials           ?     2880           

Flag	Coverage Δ
GPU	77.51% <42.85%> (?)
XPU	6.95% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[PaddlePaddle-bot]

🤖 Paddle-CI-Agent | pr_review | 2026-07-01 22:26:38

📋 Review 摘要

PR 概述：修复 OpenAI APIServer 单 worker 启动路径，并调整 get_save_output_v1 在多 DP 下的 ZMQ 消费。
变更范围：fastdeploy/entrypoints/openai/、fastdeploy/output/、fastdeploy/worker/、tests/entrypoints/
影响面 Tag：[APIServer] [DataProcessor] [XPU] [Metax]

问题

未发现新的非历史阻塞性问题。

历史 Findings 修复情况

Finding	问题	状态
F1	单 worker 分支需要同步更新 launcher 单测	✅ 已修复
F2	Gunicorn 多 worker 退出时误清理共享 LLMEngine 子服务	⚠️ 仍存在
F3	XPU DP×TP 多个 TP rank 输出推到同一个 DP socket	⚠️ 仍存在
F4	单 worker Uvicorn 路径未继承 --timeout-graceful-shutdown	⚠️ 仍存在

📝 PR 规范检查

已修复。当前 PR 描述已补充 Motivation、Modifications、Usage or Command、Accuracy Tests 等实质内容，历史模板占位问题已不再存在。

总体评价

本轮未发现新的可定位代码问题。需要注意的是，三个历史问题仍能从当前代码中复现：多 worker 生命周期清理仍无 worker 数量保护，XPU DP×TP 的 get_save_output_v1 socket 命名/发送关系仍未修正，单 worker uvicorn.run 仍未传入 graceful shutdown 超时参数。

[PaddlePaddle-bot]

🤖 Paddle-CI-Agent | ci_status_monitor | 2026-07-06 15:18:02 UTC+08:00

CI报告基于以下代码生成（30分钟更新一次）:
PR commit: 1892433 | Merge base: 17b9039 (branch: develop)

---

1 Required任务 : 8/10 通过

总执行（rerun次数）	总任务	✅ 通过	❌ 失败	⏳ 运行中	⏸️ 等待中	跳过
46(0)	46	41	4	0	0	1

任务	错误类型	置信度	日志
Run FastDeploy Unit Tests and Coverage / run_tests_with_coverage	PR问题：diff coverage 42%，未达80%	高	Job
Approval	环境问题：需要人工 Approval	高	Job

2 失败详情

🔴 Run FastDeploy Unit Tests and Coverage / run_tests_with_coverage — PR问题（置信度: 高）

分析器: 通用分析(fallback)
失败用例: diff coverage 阈值检查

用例	错误摘要
Verify Code Coverage Threshold (80%)	单测通过，但 PR diff 覆盖率为 42%，低于 80% 阈值

关键日志:

Failure. Coverage is below 80%.
Diff Coverage
fastdeploy/entrypoints/openai/api_server.py (75.0%): Missing lines 323
fastdeploy/output/token_processor.py (0.0%): Missing lines 427-430,432,434-435
GPU Patch Coverage Details: total_percent_covered=42, total_num_violations=8
TEST_EXIT_CODE: 0
COVERAGE_EXIT_CODE: 9
Process completed with exit code 9.

• 根因摘要: diff coverage 42%，新增行未覆盖

PR 新增的同步 shutdown 分支 llm_engine._exit_sub_services() 未被测试覆盖；同时 process_sampling_results_use_zmq() 中移除 rank 条件后的 ZMQ 接收、reschedule、batch process、postprocess 路径未被测试覆盖。现有新增测试只覆盖了 launch_api_server() 的 workers == 1 分支，未覆盖上述新增/变更行，因此 diff-cover 以退出码 9 阻断 job。

修复建议:

1. 在 tests/entrypoints/openai/test_api_server.py 补充 lifespan 的同步分支 shutdown 测试，mock envs.FD_ENABLE_ASYNC_LLM=False、llm_engine._exit_sub_services()、EngineClient 和 handler 构造，进入并退出 lifespan 后断言第 323 行被调用。
2. 在 tests/output/test_token_processor.py 或 tests/output/test_get_save_output_v1.py 增加 process_sampling_results_use_zmq() 正常路径测试：mock zmq_server.recv_pyobj() 返回 list，再通过受控异常跳出循环，并断言第 427-435 行的 _reschedule_preempt_task_use_zmq()、_process_batch_output_use_zmq()、postprocess() 均被调用。

关联变更: fastdeploy/entrypoints/openai/api_server.py:323，fastdeploy/output/token_processor.py:427，fastdeploy/output/token_processor.py:428，fastdeploy/output/token_processor.py:429，fastdeploy/output/token_processor.py:430，fastdeploy/output/token_processor.py:432，fastdeploy/output/token_processor.py:434，fastdeploy/output/token_processor.py:435

🔴 Approval — 环境问题（置信度: 高）

该 Job 需要人工 Approval，完成审批后 CI 才会继续执行。

• 根因摘要: 需要人工 Approval

修复建议:

1. 请通过人工审批后重新触发相关 workflow。

关联变更: 无

[paddle-bot]

Thanks for your contribution!