[Feature] console print statistical metrics

[CSWYF3634076]

Motivation

增加控制台打印scheduler的统计信息，方便实时查看当前服务的情况，对标vllm和sglang
控制台效果图

Add the ability to print statistical information of the scheduler from the console, facilitating real-time monitoring of the current service status, benchmarking against vllm and sglang
Effect diagram of console

Modifications

• 新增SchedulerMetricsLogger用于统计信息，在EngineService中初始化
• schedule 方法中计算并调用metrics函数打印到控制台
• TokenProcessor的_process_batch_output中计算token量累计值

---

• Added SchedulerMetricsLogger for statistical information, initialized in EngineService
• Calculate and call the print metrics function in the schedule method
• Calculate the cumulative value of token quantity in the _process_batch_output of TokenProcessor

Usage or Command

no

Accuracy Tests

性能无影响

main

============ Serving Benchmark Result ============
Successful requests:                     190       
Failed requests:                         10        
Maximum request concurrency:             512       
Request rate configured (RPS):           20.00     
Benchmark duration (s):                  61.22     
Total input tokens:                      486400    
Total generated tokens:                  48299     
Request throughput (req/s):              3.10      
Output token throughput (tok/s):         788.91    
Peak output token throughput (tok/s):    2160.00   
Peak concurrent requests:                190.00    
Total token throughput (tok/s):          8733.71   
---------------Time to First Token----------------
Mean TTFT (ms):                          27649.77  
Median TTFT (ms):                        24970.23  
P99 TTFT (ms):                           46783.65  
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          28.44     
Median TPOT (ms):                        30.44     
P99 TPOT (ms):                           71.20     
---------------Inter-token Latency----------------
Mean ITL (ms):                           28.17     
Median ITL (ms):                         17.53     
P99 ITL (ms):                            189.81    
==================================================

this PR

============ Serving Benchmark Result ============
Successful requests:                     190       
Failed requests:                         10        
Maximum request concurrency:             512       
Request rate configured (RPS):           20.00     
Benchmark duration (s):                  60.73     
Total input tokens:                      486400    
Total generated tokens:                  48299     
Request throughput (req/s):              3.13      
Output token throughput (tok/s):         795.29    
Peak output token throughput (tok/s):    2062.00   
Peak concurrent requests:                190.00    
Total token throughput (tok/s):          8804.33   
---------------Time to First Token----------------
Mean TTFT (ms):                          27556.45  
Median TTFT (ms):                        24871.24  
P99 TTFT (ms):                           46258.72  
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          27.95     
Median TPOT (ms):                        25.26     
P99 TPOT (ms):                           72.67     
---------------Inter-token Latency----------------
Mean ITL (ms):                           27.67     
Median ITL (ms):                         18.04     
P99 ITL (ms):                            192.37    
==================================================

Checklist

• Add at least a tag in the PR title.
  • Tag list: [[FDConfig],[APIServer],[Engine], [Scheduler], [PD Disaggregation], [Executor], [Graph Optimization], [Speculative Decoding], [RL], [Models], [Quantization], [Loader], [OP], [KVCache], [DataProcessor], [BugFix], [Docs], [CI], [Optimization], [Feature], [Benchmark], [Others], [XPU], [HPU], [GCU], [DCU], [Iluvatar], [Metax]]
  • You can add new tags based on the PR content, but the semantics must be clear.
• Format your code, run pre-commit before commit.
• Add unit tests. Please write the reason in this PR if no unit tests.
• Provide accuracy results.
• If the current PR is submitting to the release branch, make sure the PR has been submitted to the develop branch, then cherry-pick it to the release branch with the [Cherry-Pick] PR tag.

[paddle-bot]

Thanks for your contribution!

[codecov-commenter]

Codecov Report

❌ Patch coverage is 52.87356% with 41 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@6563b83). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
...astdeploy/engine/sched/scheduler_metrics_logger.py	44.06%	33 Missing ⚠️
fastdeploy/output/token_processor.py	36.36%	6 Missing and 1 partial ⚠️
fastdeploy/engine/sched/resource_manager_v1.py	92.30%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             develop    #6339   +/-   ##
==========================================
  Coverage           ?   68.26%           
==========================================
  Files              ?      390           
  Lines              ?    52026           
  Branches           ?     8102           
==========================================
  Hits               ?    35513           
  Misses             ?    13936           
  Partials           ?     2577           

Flag	Coverage Δ
GPU	68.26% <52.87%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[kevincheng2]

1. PD分离场景这里可能不准，后边可以看下
2. 使用local schedule调度时，waiting队列会不准，后边也可以看下

[CSWYF3634076]

好的，后续对一下