[Feature] Add TD3Trainer with config system integration

[bsprenger]

Description

This commit adds a TD3Trainer, implementing the TD3 algorithm (twin delayed DDPG) via the trainer framework.

Motivation and Context

The trainer framework already implements many of the most common algorithms. However, TD3 is notably missing.

This commit therefore adds:

• the TD3Trainer class
• config dataclasses for the trainer and loss function, integrated with the hydra-based config framework
• A SOTA example (HalfCheetah-v4)

Running the SOTA example with the default configuration produces results shown in the following GIF:

• I have raised an issue to propose this change (required for new features and bug fixes)

Closes: #3558

Types of changes

• New feature (non-breaking change which adds core functionality)
• Example (update in the folder of examples)

Checklist

• I have read the CONTRIBUTION guide (required)
• My change requires a change to the documentation.
• I have updated the tests accordingly (required for a bug fix or a new feature).
• I have updated the documentation accordingly.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3557

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures, 1 Cancelled Job, 1 Unrelated Failure

As of commit ccac61d with merge base 4e2e787 ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / CPU Pytest benchmark (gh)
  Process completed with exit code 1.
• Continuous Benchmark / CPU Pytest benchmark (gh)
  TypeError: expected Tensor()
• Generate documentation / build-docs (3.12, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 685f4e5188ed7d1e7ab46c0838753d05b0dc04cc8eaa4b19c216e3d185527eb3 /exec failed with exit code 2
• Unit-tests on Linux / test-setup-minimal (3.10) / linux-job (gh)
  RuntimeError: Command docker exec -t a60c016a9473e0ea333b2dbf31ab71f54ff45b66d740b5ce0a8bc992afcb3a1c /exec failed with exit code 1
• Unit-tests on Linux / test-setup-minimal (3.14) / linux-job (gh)
  RuntimeError: Command docker exec -t c6ac26ec0932d1e2ccb65f7859e4f0716759b98356bb5c938caee8e712ed06ca /exec failed with exit code 1
• Unit-tests on Linux / tests-optdeps (3.12, 13.0) / linux-job (gh)
  RuntimeError: Command docker exec -t 8fd2a750a3a0c27cfe069e9ee774442557a4766d4cc4a82441b6efc965d6262c /exec failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  Workflow failed! Resource not accessible by integration

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Unit-tests on Windows / unittests-cpu (3.10, windows.4xlarge, cpu) / windows-job (gh) (trunk failure)
  Process completed with exit code 1.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

⚠️ PR Title Label Error

Unknown or invalid prefix [Trainer].

Current title: [Trainer] Add TD3Trainer with config system integration

Supported Prefixes (case-sensitive)

Your PR title must start with exactly one of these prefixes:

Prefix	Label Applied	Example
[BugFix]	BugFix	[BugFix] Fix memory leak in collector
[Feature]	Feature	[Feature] Add new optimizer
[Doc] or [Docs]	Documentation	[Doc] Update installation guide
[Refactor]	Refactoring	[Refactor] Clean up module imports
[CI]	CI	[CI] Fix workflow permissions
[Test] or [Tests]	Tests	[Tests] Add unit tests for buffer
[Environment] or [Environments]	Environments	[Environments] Add Gymnasium support
[Data]	Data	[Data] Fix replay buffer sampling
[Performance] or [Perf]	Performance	[Performance] Optimize tensor ops
[BC-Breaking]	bc breaking	[BC-Breaking] Remove deprecated API
[Deprecation]	Deprecation	[Deprecation] Mark old function
[Quality]	Quality	[Quality] Fix typos and add codespell

Note: Common variations like singular/plural are supported (e.g., [Doc] or [Docs]).

[vmoens]

LGTM thanks for this!