[GDPVal] Fix long-context budgeting and compaction

benchmarks/gdpval/config.yaml

@@ -103,6 +103,15 @@ gdpval_stirrup_agent:
       agent_max_turns: 250
       concurrency: 32
       temperature: 1.0
+      # GDPVal tasks routinely need multi-thousand-token scripts. A 1,024-token
+      # completion cannot form a useful tool call and can create a sticky
+      # code_exec({}) loop after long reasoning turns.
+      context_window_tokens: 262144
+      min_completion_tokens: 8192
+      # Reject the observed near-empty compaction outputs that erase progress
+      # on long GDPVal sessions. Generic Stirrup agents accept any non-empty
+      # summary for backward compatibility.
+      min_compaction_summary_words: 50
       system_prompt_template: ${oc.env:SYSTEM_PROMPT_TEMPLATE,null}
       user_prompt_template: ${oc.env:USER_PROMPT_TEMPLATE,null}
       gdpval_container_path: ${oc.env:GDPVAL_CONTAINER_PATH,null}

responses_api_agents/stirrup_agent/README.md

@@ -131,6 +131,10 @@ The agent reads its Hydra config at `configs/stirrup_agent.yaml`. Notable keys:
 | `rerun_incomplete` | `false` | If true, skip the rollout for tasks that already **finished** (a finish marker is cached) and only re-run the ones that didn't (see [Task Re-run Mode](#task-re-run-mode)). Composes with `execute_only` and `judge_only`. |
 | `model_id` | `null` | HF model id or local path used to load a tokenizer for dynamic output sizing. |
 | `completion_token_buffer` | `1000` | Safety margin (in tokens) reserved when sizing `max_completion_tokens` per call. |
+| `context_window_tokens` | `262144` | Model context-window size used for dynamic completion budgeting. Independent of a request's `max_output_tokens`. |
+| `min_completion_tokens` | `1024` | Target useful per-call completion floor. The hard cap always applies; estimated remaining context is also a strict bound when the tokenizer successfully renders the complete prompt. Approximate fallbacks preserve this floor even when their estimate exceeds context. GDPVal raises it to `8192`. |
+| `prompt_estimator_truncate_history_thinking` | `null` | Optional estimator-only switch for templates that omit completed historical reasoning. It is never sent to the provider. |
+| `min_compaction_summary_words` | `1` | Minimum summary length accepted during context compaction. GDPVal uses `50`; invalid summaries retry up to three times and never replace the live history. |
 
 Env vars honored: `TAVILY_API_KEY`, `HF_TOKEN`, `OPENAI_API_KEY`.
 
@@ -155,13 +159,28 @@ The wrapper ships a `DynamicMaxTokensChatCompletionsClient`
 
 Set `model_id` to the same checkpoint (or HF id) you are serving via
 vLLM and the tokeniser match is exact.  Leave it unset and the client
-falls back to a conservative character-count estimate — slower to
-allocate completion budget but always safe.  `completion_token_buffer`
+falls back to a character-count estimate. When
+`prompt_estimator_truncate_history_thinking=true`, that fallback mirrors
+Nemotron's prior-assistant transformation without changing the retained
+trajectory or provider request. `completion_token_buffer`
 absorbs the residual gap between our estimate and the exact prompt the
 server renders (chat-template wrappers, tool-schema injection).  The
 default `1000` works in practice; raise it (e.g. 2000–5000) if you see
 sporadic HTTP 400 responses at the vLLM proxy.
 
+When the loaded tokenizer successfully renders the complete prompt, estimated
+remaining context is a strict output bound and a prompt with no remaining room
+fails before dispatch. If full rendering is unsupported, JSON and character
+estimates can substantially overcount retained reasoning, so the client still
+dispatches the configured completion floor; the hard completion cap applies in
+both modes.
+
+GDPVal uses an `8192`-token minimum because a 1024-token truncation cannot
+reliably produce a complete script or tool call. Generic Stirrup agents keep
+the historical 1024-token default. Launchers serving a local checkpoint should
+still pass that checkpoint as `model_id`; the fallback is a resilience path,
+not a replacement for template-exact tokenization.
+
 ## Advanced Features
 
 NeMo Gym splits a GDPVal evaluation into two halves — *executing* the task

responses_api_agents/stirrup_agent/app.py

@@ -475,7 +475,7 @@ async def _run_stirrup_agent(
     api_key: str = "dummy",
     max_turns: int = 250,
     temperature: float = 0.6,
-    max_tokens: int = 262144,
+    context_window_tokens: int = 262144,
     reference_files: Optional[list] = None,
     reference_file_urls: Optional[list] = None,
     exec_provider_class: Optional[str] = None,
@@ -489,7 +489,10 @@ async def _run_stirrup_agent(
     top_p: float = 0.95,
     enable_thinking: bool = True,
     max_completion_tokens_cap: int = 64000,
+    min_completion_tokens: int = 1024,
+    prompt_estimator_truncate_history_thinking: Optional[bool] = None,
     truncation_recovery: bool = True,
+    min_compaction_summary_words: int = 1,
     tavily_api_key: Optional[Union[str, List[str]]] = None,
     tavily_max_sweeps: int = 1,
 ) -> Dict[str, Any]:
@@ -502,6 +505,9 @@ async def _run_stirrup_agent(
     a tokenizer that sizes ``max_completion_tokens`` dynamically per call
     (see ``DynamicMaxTokensChatCompletionsClient``).  When unset, a
     character-count fallback is used.
+
+    *context_window_tokens* describes model capacity. It is deliberately
+    separate from the outer Responses API request's ``max_output_tokens``.
     """
     from stirrup.tools import DEFAULT_TOOLS
     from stirrup.tools.code_backends.base import SHELL_TIMEOUT, CodeExecToolProvider, CommandResult
@@ -550,13 +556,15 @@ async def run_command(self, cmd: str, *, timeout: int = SHELL_TIMEOUT) -> Comman
         model=model_name,
         base_url=model_base_url,
         api_key=api_key,
-        max_tokens=max_tokens,
+        max_tokens=context_window_tokens,
         model_id=model_id,
         completion_token_buffer=completion_token_buffer,
         temperature=temperature,
         top_p=top_p,
         enable_thinking=enable_thinking,
         max_completion_tokens_cap=max_completion_tokens_cap,
+        min_completion_tokens=min_completion_tokens,
+        prompt_estimator_truncate_history_thinking=prompt_estimator_truncate_history_thinking,
         truncation_recovery=truncation_recovery,
     )
 
@@ -609,6 +617,7 @@ async def run_command(self, cmd: str, *, timeout: int = SHELL_TIMEOUT) -> Comman
         "tools": tools,
         "tool_response_as_user": True,
         "skip_input_file_listing": is_gdpval,
+        "min_compaction_summary_words": min_compaction_summary_words,
     }
     if system_prompt:
         agent_kwargs["system_prompt"] = system_prompt
@@ -969,8 +978,8 @@ class StirrupAgentWrapperConfig(BaseResponsesAPIAgentConfig):
         default=None,
         description="HuggingFace model ID (or local checkpoint path) used to load a tokenizer "
         "for dynamic max_completion_tokens sizing. E.g. 'Qwen/Qwen3-Coder-30B-A3B-Instruct'. "
-        "When None, a character-count fallback is used (conservative, but slightly over-allocates "
-        "input tokens). See ``nemo_client.DynamicMaxTokensChatCompletionsClient``.",
+        "When None, an approximate character-count fallback is used; it can substantially "
+        "overcount retained reasoning. See ``nemo_client.DynamicMaxTokensChatCompletionsClient``.",
     )
     completion_token_buffer: int = Field(
         default=1000,
@@ -979,6 +988,13 @@ class StirrupAgentWrapperConfig(BaseResponsesAPIAgentConfig):
         "(messages + tool-schema JSON) and the exact prompt the server sees after chat-template "
         "rendering. See ``nemo_client.DynamicMaxTokensChatCompletionsClient``.",
     )
+    context_window_tokens: int = Field(
+        default=262144,
+        ge=1,
+        description="Model context-window size used for dynamic per-call completion budgeting. "
+        "This is independent of the Responses API request's max_output_tokens, which limits "
+        "output rather than describing model capacity.",
+    )
     top_p: float = Field(
         default=0.95,
         description="Top-p sampling cutoff for the policy model. Forwarded to the LLM client.",
@@ -990,10 +1006,25 @@ class StirrupAgentWrapperConfig(BaseResponsesAPIAgentConfig):
     )
     max_completion_tokens_cap: int = Field(
         default=64000,
+        ge=1,
         description="Hard ceiling on per-call ``max_completion_tokens``. Dynamic sizing computes "
         "context_window - input_tokens - completion_token_buffer, then caps to this value. "
         "Set to match the training-side response-length budget for RL.",
     )
+    min_completion_tokens: int = Field(
+        default=1024,
+        ge=1,
+        description="Target minimum per-call ``max_completion_tokens``. The hard cap always applies; "
+        "estimated remaining context is also a strict bound when the loaded tokenizer successfully "
+        "renders the complete prompt. The "
+        "approximate fallback preserves this floor even when its estimate exceeds context. "
+        "Long-horizon benchmarks can opt into a larger usable floor.",
+    )
+    prompt_estimator_truncate_history_thinking: Optional[bool] = Field(
+        default=None,
+        description="Optional prompt-estimator setting for checkpoints whose template omits historical "
+        "reasoning before the last user turn. This is never forwarded to the model request.",
+    )
     truncation_recovery: bool = Field(
         default=True,
         description="When a model call spends its entire completion budget without emitting a "
@@ -1004,6 +1035,13 @@ class StirrupAgentWrapperConfig(BaseResponsesAPIAgentConfig):
         "deliverable writes (median 34.6k tokens). The instruction is sent to the server but not "
         "recorded in the trajectory; set False for RL rollouts that require an unsteered policy.",
     )
+    min_compaction_summary_words: int = Field(
+        default=1,
+        ge=1,
+        description="Minimum whitespace-delimited word count accepted from context compaction. "
+        "The generic default rejects only empty output; long-horizon benchmarks can require a "
+        "more complete summary.",
+    )
     tavily_api_key: Optional[Union[str, List[str]]] = Field(
         default=None,
         description="Tavily API key(s) for the ``web_search`` / ``fetch_web_page`` tools. "
@@ -1163,7 +1201,10 @@ async def responses(
 
         model_name = getattr(body, "model", None) or "default"
         temperature = getattr(body, "temperature", None) or self.config.temperature
-        max_tokens = getattr(body, "max_output_tokens", 262144) or 262144
+        requested_output_tokens = getattr(body, "max_output_tokens", None)
+        max_completion_tokens_cap = self.config.max_completion_tokens_cap
+        if requested_output_tokens is not None:
+            max_completion_tokens_cap = min(max_completion_tokens_cap, requested_output_tokens)
 
         exec_provider = self.task_strategy.get_exec_provider(task_info, self.config)
         exec_provider_class = None
@@ -1183,7 +1224,7 @@ async def responses(
             "api_key": "dummy",  # pragma: allowlist secret
             "max_turns": self.config.agent_max_turns,
             "temperature": temperature,
-            "max_tokens": max_tokens,
+            "context_window_tokens": self.config.context_window_tokens,
             "reference_files": task_info.get("reference_files") if self.config.task == "gdpval" else None,
             "reference_file_urls": task_info.get("reference_file_urls") if self.config.task == "gdpval" else None,
             "exec_provider_class": exec_provider_class,
@@ -1196,8 +1237,11 @@ async def responses(
             "completion_token_buffer": self.config.completion_token_buffer,
             "top_p": getattr(body, "top_p", None) or self.config.top_p,
             "enable_thinking": self.config.enable_thinking,
-            "max_completion_tokens_cap": self.config.max_completion_tokens_cap,
+            "max_completion_tokens_cap": max_completion_tokens_cap,
+            "min_completion_tokens": self.config.min_completion_tokens,
+            "prompt_estimator_truncate_history_thinking": (self.config.prompt_estimator_truncate_history_thinking),
             "truncation_recovery": self.config.truncation_recovery,
+            "min_compaction_summary_words": self.config.min_compaction_summary_words,
             "tavily_api_key": self.config.tavily_api_key,
             "tavily_max_sweeps": self.config.tavily_max_sweeps,
         }