Add environment contribution docs

docs/contribute/development-setup.md

@@ -47,7 +47,7 @@ All contributions must pass these automated checks:
 
 **Required Checks**:
 - **Unit Tests**: All existing tests must pass
-- **Build Docs**: Documentation must build without errors  
+- **Build Docs**: Documentation must build without errors
 - **Copyright Check**: All files must have proper copyright headers
 - **DCO Signing**: All commits must be signed off
 - **Pre-commit Hooks**: Code formatting and linting
@@ -238,7 +238,7 @@ pre-commit autoupdate             # Update hook versions
 
 **Common fixes**:
 - `ruff check --fix .` for linting
-- `ruff format .` for formatting  
+- `ruff format .` for formatting
 - Add copyright headers to new files
 
 ---

docs/contribute/environments/index.md

@@ -0,0 +1,41 @@
+(contribute-environments)=
+
+# Contribute Environments
+
+Help advance RL training for the community by contributing new environments. There are two main ways to get involved:
+
+---
+
+::::{grid} 1 2 2 2
+:gutter: 1 1 1 2
+
+:::{grid-item-card} {octicon}`server;1.5em;sd-mr-1` New Environments
+:link: new-environment
+:link-type: doc
+Build novel training environments for coding, reasoning, tool use, games, and more.
++++
+{bdg-secondary}`new-environment`
+:::
+
+:::{grid-item-card} {octicon}`repo;1.5em;sd-mr-1` Benchmark Integrations
+:link: https://github.com/NVIDIA-NeMo/Gym/tree/main/responses_api_agents/mini_swe_agent
+:link-type: url
+Integrate existing benchmarks (SWE-Bench, Tau Bench, and so on) into NeMo Gym.
+
+→ View mini-swe-agent with SWE-Bench as a reference implementation
+
+*More detailed guidance coming soon.*
++++
+{bdg-secondary}`benchmark`
+:::
+
+::::
+
+```{toctree}
+:caption: Contributing Environments
+:hidden:
+:maxdepth: 1
+
+New Environment <new-environment>
+```
+

docs/contribute/environments/new-environment.md

@@ -0,0 +1,224 @@
+(contribute-new-environment)=
+
+# Add a New Environment
+
+An environment consists of three components: Agents, Models, and Resources. Most contributors will create new resource servers while using existing agent server and model server implementations. If you need to create custom agents or models, you can reference the implementations in `responses_api_agents/` and `responses_api_models/`.
+
+This guide focuses on the resource server contribution process.
+
+:::{tip}
+For a guide to building your first resource server, see {doc}`/tutorials/creating-resource-server`.
+:::
+
+## Required Files
+
+Your resource server must include these files:
+
+```{list-table}
+:header-rows: 1
+:widths: 30 70
+
+* - File
+  - Description
+* - `app.py`
+  - Main server implementation with `verify` function
+* - `configs/*.yaml`
+  - Configuration with valid `domain` field
+* - `tests/test_app.py`
+  - At least one unit test
+* - `data/example.jsonl`
+  - At least five example inputs
+* - `data/example_rollouts.jsonl`
+  - Pre-generated rollouts from example data (generate before submitting PR)
+* - `requirements.txt`
+  - Python dependencies
+* - `README.md`
+  - Documentation with licensing information
+```
+
+
+## Contribution Workflow
+
+Contributing a resource server follows this sequence:
+
+```{list-table}
+:header-rows: 1
+:widths: 10 30 60
+
+* - Step
+  - Phase
+  - Description
+* - 1
+  - Curate Tasks
+  - Collect or generate training tasks and create example data
+* - 2
+  - Implementation
+  - Build resource server with verification logic
+* - 3
+  - Testing
+  - Write and run unit tests
+* - 4
+  - Example Rollouts
+  - Generate example rollouts to verify functionality
+* - 5
+  - Reward Profiling
+  - Validate reward distribution with inference runs
+* - 6
+  - Training Validation
+  - Train with GRPO to ensure meaningful training signal
+* - 7
+  - Submit PR
+  - Submit pull request with all required information
+* - 8
+  - Review
+  - Address reviewer feedback and verify reproducibility
+```
+
+## Detailed Steps
+
+### 1. Curate Training Tasks
+
+Prepare the dataset for your environment:
+
+- Collect or generate prompts/tasks for your environment
+- Create `data/example.jsonl` with at least 5 representative task examples
+
+### 2. Resource Server Implementation
+
+Build your resource server:
+
+- Run `ng_init_resources_server +entrypoint=resources_servers/my_server` to scaffold the new resource server
+- Follow the {doc}`/tutorials/creating-resource-server` guide to implement your specific logic
+- Implement verification logic for your tasks by defining the `verify()` function
+- Set the `domain` field in your resource server configuration (see {py:class}`~nemo_gym.config_types.Domain`).
+- Complete the auto-generated `README.md` with licensing information
+
+### 3. Testing
+
+Write and run tests for your resource server:
+
+- At least one test per server is required for PR approval
+- You are responsible for ensuring your tests adequately cover your server's functionality
+
+### 4. Generate Example Rollouts
+
+Verify basic functionality and generate example rollouts:
+
+- Document the command used to start your server, for example, `ng_run +entrypoint=resources_servers/my_server`
+- Generate rollouts and save 5 example outputs to `data/example_rollouts.jsonl` to demonstrate correct reward signals
+
+### 5. Reward Profiling
+
+Run inference to validate reward distribution:
+
+- Use a ~500 sample subset (minimum)
+- Use Qwen3-4B, Qwen3 30B A3B, or equivalent model
+- Generate 16 responses per prompt
+- Report reward distribution
+- **For tool calling**: Provide tool call metrics and correlation with rewards
+
+### 6. Training-Based Validation
+
+Validate with actual training:
+
+- Train with GRPO on Qwen3-4B, Qwen 30B A3B Instruct, or equivalent model
+- Include training accuracy curve
+- Include test benchmark accuracy curve (if applicable)
+
+### 7. Submit PR
+
+Include the following in your pull request description:
+
+- Description of the environment
+- Description of the verification logic
+- Description of the prompts/tasks: What is the source? Which domain does it cover?
+- Provide relevant license information for data and software. If models were used for synthetic data generation, note this in your PR description
+
+### 8. PR Review Process
+
+After submitting your PR:
+
+1. A team member will be assigned to review and reproduce your environment
+2. The reviewer will verify all steps and check correctness of the 5 example rollouts
+3. The reviewer will re-run the procedure to ensure reproducibility
+4. Address any feedback from reviewers
+5. Once approved, maintainers will merge your contribution
+
+## Recommended Technical Design
+
+For optimal performance and scalability, we recommend following these design patterns:
+
+### Async-First Design
+
+Endpoint handlers should be asynchronous to handle concurrent requests efficiently during training:
+
+```python
+# Recommended: async function
+async def verify(self, body: BaseVerifyRequest) -> BaseVerifyResponse:
+    return BaseVerifyResponse(**body.model_dump(), reward=1.0)
+```
+
+Avoid spawning additional threads or processes unless necessary. A single Gym instance can handle tens of thousands of concurrent requests when properly implemented.
+
+### NeMo Gym OpenAI Client
+
+We recommend using the NeMo Gym OpenAI client from `nemo_gym.openai_utils`:
+
+```python
+from nemo_gym.openai_utils import (
+    NeMoGymAsyncOpenAI,
+    NeMoGymResponse,
+    NeMoGymResponseCreateParamsNonStreaming,
+)
+```
+
+The NeMo Gym client is optimized for scale and provides consistent behavior. External clients like LiteLLM often preprocess or postprocess inputs and outputs in ways that can interfere with training data collection.
+
+### Pydantic Models
+
+Consider using Pydantic models for request and response validation by extending base classes from `nemo_gym.base_resources_server`:
+
+```python
+from pydantic import BaseModel
+from nemo_gym.base_resources_server import BaseVerifyRequest, BaseVerifyResponse
+
+class MyVerifyRequest(BaseVerifyRequest):
+    expected_result: str
+    difficulty: int
+```
+
+### Error Handling
+
+Tool execution errors should be propagated back to the model rather than crashing the server, enabling the model to learn from mistakes:
+
+```python
+async def execute_tool(self, path: str, body: ToolRequest) -> ToolResponse:
+    try:
+        result = self.tool_functions[path](**body.model_dump())
+        return ToolResponse(output=result)
+    except Exception as e:
+        # Return error to model so it can correct itself
+        return ToolResponse(output=f"Error executing tool '{path}': {str(e)}")
+```
+
+### Configuration
+
+Pass configuration through NeMo Gym config files rather than environment variables for better reproducibility:
+
+```yaml
+# configs/my_server.yaml
+host: 0.0.0.0
+port: 8000
+domain: agent
+```
+
+### Multi-Step Rollouts
+
+For multi-step scenarios, the model returns training information on response messages (`prompt_token_ids`, `generation_token_ids`, `generation_log_probs`). When constructing messages for subsequent model calls, propagate this information from previous responses to maintain the training data chain.
+
+
+## Reference
+
+- {doc}`/tutorials/creating-resource-server` - Introductory tutorial for creating your first resource server
+- {doc}`/about/concepts/task-verification` - Verification and reward concepts
+- {doc}`/about/concepts/core-components` - Resource server architecture

docs/contribute/index.md

@@ -14,6 +14,8 @@ New environments expand Gym's training capabilities.
 
 - Novel training environments (coding, reasoning, tool use, games, and so on)
 - Benchmark integrations (such as SWE-Bench and Tau Bench)
+
+→ Refer to {ref}`contribute-environments` for guidance.
 :::
 
 :::{dropdown} {octicon}`workflow;1.5em;sd-mr-1` Integrate RL Frameworks

docs/contribute/rl-framework-integration/index.md

@@ -77,7 +77,7 @@ The typical integration workflow follows this sequence:
   - Description
 * - 1
   - Generation backend
-  - Expose your generation engine (vLLM, SGLang) as an OpenAI-compatible HTTP server
+  - Expose your generation engine, such as vLLM or SGLang, as an OpenAI-compatible HTTP server
 * - 2
   - On-policy corrections
   - Implement token ID fixes to prevent re-tokenization and re-templating issues

docs/index.md

@@ -148,12 +148,20 @@ Contribute to NeMo Gym development.
 ::::{grid} 1 2 2 2
 :gutter: 1 1 1 2
 
-:::{grid-item-card} {octicon}`workflow;1.5em;sd-mr-1` Integrate Gym into RL Frameworks
+:::{grid-item-card} {octicon}`server;1.5em;sd-mr-1` Contribute Environments
+:link: contribute/environments/index
+:link-type: doc
+Contribute new environments or integrate existing benchmarks.
++++
+{bdg-primary}`environments`
+:::
+
+:::{grid-item-card} {octicon}`workflow;1.5em;sd-mr-1` Integrate RL Frameworks
 :link: contribute/rl-framework-integration/index
 :link-type: doc
 Implement NeMo Gym integration into a new training framework.
 +++
-{bdg-primary}`advanced`
+{bdg-primary}`training-integration`
 :::
 
 ::::
@@ -196,16 +204,6 @@ tutorials/offline-training-w-rollouts
 tutorials/rl-training-with-nemo-rl
 ```
 
-```{toctree}
-:caption: Contribute
-:hidden:
-:maxdepth: 1
-
-Overview <contribute/index>
-Development Setup <contribute/development-setup>
-Integrate RL Frameworks <contribute/rl-framework-integration/index>
-```
-
 ```{toctree}
 :caption: Reference
 :hidden:
@@ -224,3 +222,14 @@ apidocs/index.rst
 
 troubleshooting/configuration.md
 ```
+
+```{toctree}
+:caption: Contribute
+:hidden:
+:maxdepth: 1
+
+Overview <contribute/index>
+Development Setup <contribute/development-setup>
+Environments <contribute/environments/index>
+Integrate RL Frameworks <contribute/rl-framework-integration/index>
+```