[Cherry-Pick] [Feature] support v1 update/clear api for RL (#6761)

[liyonghua0910]

Motivation

This PR upgrades the weight clearing and updating flow for RL scenarios.

The legacy control path mainly relied on shared memory to synchronize state across the engine, worker, and cache-related components. While functional, the signal path was not explicit enough, and it was difficult to trace how failed requests were handled across components. In addition, the old workflow usually cleared weights through clear_load_weight first, even though residual requests could still exist, and then relied on a manual reset_scheduler call to clean up the scheduler queue. This made the lifecycle less explicit and introduced risks of inconsistent states during asynchronous resource recycling.

The goal of this PR is to move the control flow to an explicit control-request path and replace the legacy weight clear/reload flow with the new sleep/wakeup workflow, so state transitions and troubleshooting become more straightforward.

Modifications

• Switch weight-clear/update control signals to a dedicated ControlRequest/ControlResponse path, so each control request has its own request ID and can be traced through logs end to end.
• Add /v1/sleep and /v1/wakeup, with tags support to specify which part of GPU memory should be offloaded or reloaded. Enable these APIs by export FD_ENABLE_V1_UPDATE_WEIGHTS=1.
• Keep /clear_load_weight and /update_model_weight for compatibility:
  • When FD_ENABLE_V1_UPDATE_WEIGHTS=0, /clear_load_weight and /update_model_weight still rely on shared memory for control and multi-process synchronization.
  • When FD_ENABLE_V1_UPDATE_WEIGHTS=1, /clear_load_weight and /update_model_weight switch to the new control path, using the engine worker queue, engine cache queue, and FMQ for request dispatch and response collection.
• Extend /v1/pause and /v1/resume with cache-transfer-manager coordination to support multi-level cache and KV-cache-backend scenarios.

Usage or Command

Export the following environment variable when starting server:

export FD_ENABLE_V1_UPDATE_WEIGHTS=1

Send control requests:

# Legacy-compatible endpoints
curl -i http://<IP>:<PORT>/clear_load_weight
curl -i http://<IP>:<PORT>/update_model_weight

# New control endpoints
curl -X POST http://<IP>:<PORT>/v1/sleep
curl -X POST http://<IP>:<PORT>/v1/wakeup

Accuracy Tests

• N/A. This PR focuses on the RL weight-control path, request/control coordination, and interface changes. No model-output accuracy result is included in the current description.

Checklist

• Add at least a tag in the PR title.
• Format your code, run pre-commit before commit.
• Add unit tests. Please write the reason in this PR if no unit tests.
• Provide accuracy results.
• If the current PR is submitting to the release branch, make sure the PR has been submitted to the develop branch, then cherry-pick it to the release branch with the [Cherry-Pick] PR tag.

[paddle-bot]

Thanks for your contribution!

[codecov-commenter]

Codecov Report

❌ Patch coverage is 21.03321% with 428 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (release/2.5@49c2310). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
fastdeploy/cache_manager/cache_transfer_manager.py	16.93%	152 Missing ⚠️
fastdeploy/engine/common_engine.py	10.63%	125 Missing and 1 partial ⚠️
fastdeploy/worker/gpu_model_runner.py	11.29%	55 Missing ⚠️
fastdeploy/rl/dynamic_weight_manager.py	13.95%	37 Missing ⚠️
fastdeploy/worker/worker_process.py	26.92%	14 Missing and 5 partials ⚠️
fastdeploy/entrypoints/engine_client.py	6.66%	14 Missing ⚠️
fastdeploy/entrypoints/openai/api_server.py	76.92%	7 Missing and 2 partials ⚠️
fastdeploy/utils.py	36.36%	6 Missing and 1 partial ⚠️
fastdeploy/entrypoints/openai/utils.py	25.00%	3 Missing ⚠️
fastdeploy/config.py	0.00%	1 Missing and 1 partial ⚠️
... and 2 more

Additional details and impacted files

@@              Coverage Diff               @@
##             release/2.5    #6974   +/-   ##
==============================================
  Coverage               ?   68.95%           
==============================================
  Files                  ?      390           
  Lines                  ?    54071           
  Branches               ?     8519           
==============================================
  Hits                   ?    37282           
  Misses                 ?    14087           
  Partials               ?     2702           

Flag	Coverage Δ
GPU	68.95% <21.03%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.