feat: Deepseek Support

[yfw]

This reverts commit 427548f.

What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

Issues

List issues that this PR closes (syntax):

Usage

• You can potentially add a usage example below

# Add a code snippet demonstrating how to use this 

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
• Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

• Sample run (140 steps):
  
• To reproduce this run, first follow these instructions to create a BF16 Hugging Face checkpoint. Then, use the following script (replace /path/to/hf_checkpoint with the newly created checkpoint):

uv run python examples/run_grpo_math.py --config=examples/configs/grpo_math_1B_megatron.yaml \
    grpo.val_batch_size=2 \
    policy.model_name=/path/to/hf_checkpoint \
    cluster.num_nodes=64 \
    cluster.gpus_per_node=8 \
    policy.megatron_cfg.pipeline_model_parallel_size=8 \
    policy.megatron_cfg.tensor_model_parallel_size=1 \
    policy.megatron_cfg.expert_tensor_parallel_size=1 \
    policy.megatron_cfg.sequence_parallel=False \
    policy.megatron_cfg.expert_model_parallel_size=64 \
    policy.megatron_cfg.num_layers_in_first_pipeline_stage=7 \
    policy.megatron_cfg.num_layers_in_last_pipeline_stage=6 \
    policy.megatron_cfg.activation_checkpointing=True \
    policy.megatron_cfg.apply_rope_fusion=False \
    policy.max_total_sequence_length=512 \
    checkpointing.enabled=False \
    checkpointing.save_period=20 \
    grpo.val_period=20 \
    grpo.max_val_samples=16 \
    grpo.val_batch_size=4 \
    checkpointing.keep_top_k=100 \
    checkpointing.checkpoint_dir=results/dsv3 \
    grpo.val_at_start=False \
    grpo.max_val_samples=16 \
    policy.generation.vllm_cfg.async_engine=False \
    policy.generation.vllm_cfg.tensor_parallel_size=64 \
    grpo.max_num_steps=1000000 \
    grpo.num_prompts_per_step=32 \
    grpo.num_generations_per_prompt=16 \
    policy.train_global_batch_size=512 \
    policy.train_micro_batch_size=1 \
    policy.sequence_packing.enabled=False

• Currently refit is still very slow for the large dsv3 model (just under 500 seconds). We will address this in a follow-up PR.