[Graph Optimization] remove static_op_get_block_shape_and_split_kv_block from cudagraph

[DrRyanHuang]

Motivation

由于 step_use_cudagraph 的控制，动态图+CUDAGraph 只在 Decode 的时候开启 CUDAGraph(代码如下)，而 SOT 动转静也是一样的，纯 Prefill + Prefill+Decode混合Batch 都跑在了纯动态图。因此本PR打开开关，当是动转静且子图切分模式时，step_use_cudagraph 始终为 True

FastDeploy/fastdeploy/worker/gpu_model_runner.py

Lines 1582 to 1590 in 0e0eaa1

	# Update Batch type for cuda graph for only_prefill_batch
	only_prefill_use_cudagraph = self.use_cudagraph and self.cudagraph_only_prefill and self.only_prefill()
	# When support capture both prefill-only and decode-only, this will use [only_prefill_use_cudagraph or only_decode_use_cudagraph]
	self.forward_meta.step_use_cudagraph = (
	only_prefill_use_cudagraph
	if self.cudagraph_only_prefill
	else only_decode_use_cudagraph and self.forward_meta.ids_remove_padding.shape[0] > 0
	)

但此时出现了精度问题，纯 Prefill + Prefill+Decode混合Batch 存在精度问题

是 get_block_shape_and_split_kv_block 这个算子存在 prefill 与 decode 不统一的情况，所以在子图切分的时候，也要把这个算子从 CUDAGraph 中移出去

FastDeploy/custom_ops/gpu_ops/append_attn/get_block_shape_and_split_kv_block.cu

Lines 313 to 315 in 0e0eaa1

	// decoder
	if (max_dec_len_this_time > 0) {
	if (mla_backend) {

FastDeploy/custom_ops/gpu_ops/append_attn/get_block_shape_and_split_kv_block.cu

Lines 388 to 390 in 0e0eaa1

	// encoder
	if (max_enc_len_this_time > 0) {
	const uint32_t max_tile_size_per_bs_kv =

Modifications

• 打开 CUDAGraph 开关，当是动转静且子图切分模式时，step_use_cudagraph 始终为 True
• 将 get_block_shape_and_split_kv_block 添加到 FLAGS_cuda_graph_blacklist 中
• 与 [Executor]CUDAGraph support Speculate Decode #3769 同，将 paddle.empty -> paddle.zeros

Usage or Command

rm -rf log/*

MODEL=/root/paddlejob/tmpspace/huangzihao01/MODEL/ERNIE-4.5-0.3B-Paddle
# MODEL=/root/paddlejob/tmpspace/huangzihao01/MODEL/ERNIE-4.5-21B-A3B-Paddle
# MODEL=/root/paddlejob/tmpspace/huangzihao01/MODEL/ERNIE-4.5-300B-A47B-Paddle

PORT=39899
let MPPPPP=PORT+1
let EWQPPP=PORT+2

# export FLAGS_print_ir=1
export CUDA_VISIBLE_DEVICES=2,3,4,5
# export GLOG_v=6
# export FLAGS_call_stack_level=2

python -m fastdeploy.entrypoints.openai.api_server \
       --model $MODEL \
       --port $PORT \
       --metrics-port $MPPPPP \
       --engine-worker-queue-port $EWQPPP \
       --tensor-parallel-size 4 \
       --max-model-len 32768 \
       --max-num-seqs 128 \
       --graph-optimization-config '{"graph_opt_level": 1, "use_cudagraph":true, "full_cuda_graph": false}' \
       --quantization wint4 \

Accuracy Tests

NO NEED

Checklist

• Add at least a tag in the PR title.
  • Tag list: [[FDConfig],[APIServer],[Engine], [Scheduler], [PD Disaggregation], [Executor], [Graph Optimization], [Speculative Decoding], [RL], [Models], [Quantization], [Loader], [OP], [KVCache], [DataProcessor], [BugFix], [Docs], [CI], [Optimization], [Feature], [Benchmark], [Others], [XPU], [HPU], [GCU], [DCU], [Iluvatar], [Metax]]
  • You can add new tags based on the PR content, but the semantics must be clear.
• Format your code, run pre-commit before commit.
• Add unit tests. Please write the reason in this PR if no unit tests.
• Provide accuracy results.
• If the current PR is submitting to the release branch, make sure the PR has been submitted to the develop branch, then cherry-pick it to the release branch with the [Cherry-Pick] PR tag.

[paddle-bot]

Thanks for your contribution!

[codecov-commenter]

Codecov Report

❌ Patch coverage is 71.42857% with 2 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@0e0eaa1). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
...l_executor/layers/attention/append_attn_backend.py	0.00%	1 Missing ⚠️
fastdeploy/worker/gpu_model_runner.py	0.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             develop    #6081   +/-   ##
==========================================
  Coverage           ?   67.42%           
==========================================
  Files              ?      350           
  Lines              ?    45323           
  Branches           ?     6968           
==========================================
  Hits               ?    30561           
  Misses             ?    12527           
  Partials           ?     2235           

Flag	Coverage Δ
GPU	67.42% <71.42%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[DrRyanHuang]

• 有个单测报错：tests/distributed/test_chunked_moe.py
  ERROR: test_case (main.TestChunkedMoE)

   Traceback (most recent call last):
     File "/workspace/FastDeploy/tests/distributed/chunked_moe.py", line 185, in test_case
       self.run_model_runner()
     File "/workspace/FastDeploy/tests/distributed/chunked_moe.py", line 156, in run_model_runner
       self.model_runner.initialize_forward_meta()
     File "/workspace/FastDeploy/fastdeploy/worker/gpu_model_runner.py", line 1593, in initialize_forward_meta
       if self.graph_opt_config.graph_opt_level > 0 and not self.graph_opt_config.full_cuda_graph:
   AttributeError: 'GPUModelRunner' object has no attribute 'graph_opt_config'

[gongshaotian]

改成 zeros 的目的是什么

[DrRyanHuang]

和这个一样：https://github.com/PaddlePaddle/FastDeploy/pull/3769/files#r2379415270

[gongshaotian]

LGTM

[EmmonsCurse]

LGTM~