Epic: System Prompting & Context Trimming vNext

[mattKorwel]

Overview

This epic covers the overhaul of the Gemini CLI system prompting and context management to optimize for Gemini 3.0 and address performance/coherence issues. The current prompting is heavily reliant on 2.5-specific "handholding," which is counterproductive for newer models that are more proficient with terminal-centric workflows. Managing context rot and token bloat is identified as the single most important factor for improving performance on long-running tasks and benchmarks like SWEBench.

Key Objectives

System Prompting & Gemini 3.0

• Gemini 3.0 Optimization: Transition from handholding-heavy 2.5 prompts to flexible instructions that leverage Gemini 3's Bash proficiency.
• Terminal-Centric Approach: Leverage the newer model's Bash proficiency, allowing it to perform more exploratory actions (e.g., using piping or redirection to limit output) rather than relying on strictly gated tool parameters.

Context Management

• Rolling Context Window: Implement a rolling window for context compression/pruning, targeting a stable pool (e.g., ~50k tokens) to prevent model incoherence over time.
• Context Pruning vs. Compression: Distinguish between compression (pulling down when limits are hit) and pruning (actively maintaining a focused area by removing content and replacing with file references).
• Temporary Directory Redirection: Offload heavy outputs (build logs, large search results, pre-flights) to the session temporary directory to isolate them from the primary context window.
• Improved Benchmark Performance: Utilize context trimming as the primary lever to increase success rates on high-token scenarios.

Requirements & Technical Tasks

• Temp Dir Read/Write: Enable the model to both read from and write to the temporary directory.
• Output Visibility: Implement feedback/streaming for redirected output to ensure users don't experience "hanging" during long operations.
• Structured Interpolations: Implement structured interpolations for prompt variants (sub-agents, experimental features) as suggested in Support template syntax for system prompts #13757.
• Safety & Security: Incorporate a comprehensive security and safety section in the prompts.
• Context Telemetry: Add telemetry to track context usage percentages across different components (prompt, tools, extensions) to identify "debt" offenders.

Behavioral Evals & Metrics

• Establish behavioral evals to prevent overactions and ensure consistent tool use.
• Coherence Hill-Climbing: Use a "long scenario" metric (e.g., fixing 700+ linter errors) to measure the point at which the agent loses coherence and loops.
• Repetitive Task Reliability: Ensure reliable execution of multi-file, independent changes as noted in related issue Gemini CLI often performs badly when the context window gets large #9791.
• Track operational health metrics separate from data science metrics.

---

Based on the planning session: Planning: Make Gcli smarter

Sub Issues By Category

I've categorized the 19 open sub-issues for Epic #15328 to help you decide what to tackle next:

🤖 System Prompt & Model Instructions

• [System Prompt] Isolated system prompts for Gemini 3 and Gemini 2.5 #17161: Isolated system prompts for Gemini 3 and Gemini 2.5
• Refine system prompt instructions for save_memory tool #15678: Refine instructions for save_memory tool
• Support template syntax for system prompts #13757: Support template syntax for system prompts

🧹 Context & Resource Management

• [Shell Redirection #4] ReadFolder and other FS tools cannot access project temp directory #17164: Fix FS tools (like ReadFolder) access to project temp directory
• [Shell Redirection #5] Shorten temp directory #17190: Shorten temp directory paths
• [Context Mgmt] Rolling Tool Output Pruning #16955: Rolling Tool Output Pruning
• [Context Mgmt] Lower default tool output truncation thresholds for Gemini 3 #17037: Lower tool output truncation thresholds for Gemini 3
• [Context Mgmt] Refactor GEMINI.md Context Loading to be Hierarchical #16999: Hierarchical GEMINI.md context loading

🧠 Agent Logic & Workflow

• Gemini 3 (pro/flash) doesn't understand that a question shouldn't automatically lead to action #17028: Prevent Gemini 3 from auto-acting on simple questions
• [Behavior Mgmt] Gemini CLI should build and lint after every file change #17108: Auto-build and lint after every file change
• [Behavior Mgmt] Agent should have an explicit 'review changes' step #17109: Explicit 'review changes' step
• Agent should run changes to validate app #17110: Auto-run changes to validate application
• [Behavior Mgmt] Gemini CLI should *briefly* describe what was done after doing "large" changes. #17111: Briefly describe "large" changes after completion
• [Behavior Mgmt] Gemini CLI should prefer using automated linting and formatting tools vs. manually making changes #15032: Prefer automated linting/formatting over manual changes

⚡ UX & Benchmarking

• [Shell Redirection #2] Auto-accept file redirection for current session #17188 & [Shell Redirection #3] Shift+Tab (AUTO_EDIT) mode should auto-accept shell redirections #17189: Auto-accept file/shell redirections
• [Mendel] Add experiment-based model control #17162: Experiment-based model control (Mendel)
• [Benchmarks] Verify scores for new system prompt #17163: Verify benchmark scores for new prompts
• Re-enable write_todos tool for Gemini 3 family #17035: Re-enable write_todos for Gemini 3