[Doc] Document LSTM is_init after slice sampling

[YeonwooSung]

Description

Document the is_init contract after slice sampling, and add a CPU regression that hidden state leaks across concatenated mid-episode slices when that contract is skipped.

InitTracker only marks episode starts. A SliceSampler window can start mid-episode, so every stored is_init flag in that window is False. Concatenating those slices looks like one long trajectory to LSTMModule: hidden state from the last step of slice A is fed into the first step of unrelated slice B.

The contract: after slice sampling, mark the first timestep of each slice as is_init=True. SliceSampler already does this by default (init_key="is_init"). Hand-built batches can OR the first timestep of each slice, or shift ("next", "truncated") onto is_init (do not OR truncated in place; it sits at slice ends). Sequential mode then zeros the incoming hidden with torch.where(is_init, zeros, hidden); recurrent mode splits on is_init and restarts from the stored hidden at each split.

No LSTM internals are changed.

Motivation and Context

Why is this change required? What problem does it solve?
If it fixes an open issue, please link to the issue here.
You can use the syntax close #15213 if this solves the issue #15213

• I have raised an issue to propose this change (required for new features and bug fixes)

close #3147

Types of changes

What types of changes does your code introduce? Remove all that do not apply:

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds core functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (update in the documentation)
• Example (update in the folder of examples)

Checklist

Go over all the following points, and put an x in all the boxes that apply.
If you are unsure about any of these, don't hesitate to ask. We are here to help!

• I have read the CONTRIBUTING and Agents.md docs
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I have updated the documentation related to my changes

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/4474

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• linux.dgx.b200 runner queuing exceeds 4 hours and dgxb200-05 "CUDA not available"

⚠️ 16 Awaiting Approval

As of commit 8683d34 with merge base 84d4602 ():

AWAITING APPROVAL - The following workflows need approval before CI can run:

• Build Aarch64 Linux Wheels (gh)
• Build Linux Wheels (gh)
• Build M1 Wheels (gh)
• Build Windows Wheels (gh)
• Continuous Benchmark (PR) (gh)
• Examples Tests on Linux (gh)
• Generate documentation (gh)
• Habitat Tests on Linux (gh)
• Libs Tests on Linux (gh)
• Lint (gh)
• LLM Tests on Linux (gh)
• Push Binary Nightly (gh)
• SOTA Tests on Linux (gh)
• Tutorials Tests on Linux (gh)
• Unit-tests on Linux (gh)
• Validate Test Partitioning (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[itwasabhi]

Hi @YeonwooSung thanks for picking up this request! The docs are quite helpful.