Research: Plan Mode / Ask Mode System Prompt & Architecture Analysis

[kilo-code-bot]

This issue tracks research into how Plan Mode (and Ask Mode) work in the Kilo CLI codebase — specifically focusing on the system prompts, permission architecture, and potential improvements.

See the comment below for detailed findings.

[kilo-code-bot]

Research Findings: Ask Mode in the Kilo CLI Codebase

1. Where the Ask Mode System Prompt is Defined

The Ask mode prompt is defined in a dedicated text file:

File: packages/opencode/src/agent/prompt/ask.txt

You are a knowledgeable technical assistant focused on answering questions and providing information about software development, technology, and related topics.

Guidelines:
- Answer questions thoroughly with clear explanations and relevant examples
- Analyze code, explain concepts, and provide recommendations without making changes
- Use Mermaid diagrams when they help clarify your response
- You may run read-only bash commands (ls, cat, grep, git log, git diff, etc.) to gather information
- You must NOT modify files, run write commands, or execute code — you are read-only
- MCP tools are available if configured — each call requires user approval
- If a question requires implementation, suggest switching to a different agent

This prompt is imported in packages/opencode/src/agent/agent.ts:

import PROMPT_ASK from "./prompt/ask.txt"

2. Where the Mode is Determined and How It's Passed

Agent Registration (agent.ts, lines 303–337)

The Ask agent is registered as a named agent in the agent state factory:

ask: {
  name: "ask",
  description: "Get answers and explanations without making changes to the codebase.",
  prompt: PROMPT_ASK,
  options: {},
  permission: PermissionNext.merge(
    defaults,
    user,
    PermissionNext.fromConfig({
      "*": "deny",           // deny ALL tools by default
      bash: readOnlyBash,    // only read-only shell commands
      read: { "*": "allow", "*.env": "ask", "*.env.*": "ask", "*.env.example": "allow" },
      grep: "allow",
      glob: "allow",
      list: "allow",
      question: "allow",
      webfetch: "allow",
      websearch: "allow",
      codesearch: "allow",
      codebase_search: "allow",
      external_directory: { [Truncate.GLOB]: "allow" },
      ...mcpRules,
    }),
    user.filter((r) => r.action === "deny"),
  ),
  mode: "primary",
  native: true,
},

Mode Flow Through the System

The user selects an agent (mode) which is stored on MessageV2.User.agent. This flows through SessionPrompt.loop() → Agent.get(lastUser.agent) → LLM.stream().

3. How the System Prompt Communicates Ask Mode Constraints

The system prompt is assembled in packages/opencode/src/session/llm.ts (lines 74–89):

const system = []
system.push(
  [
    // Soul - core identity
    ...(isCodex ? [] : [SystemPrompt.soul()]),
    // Agent prompt OR provider prompt (mutually exclusive!)
    ...(input.agent.prompt ? [input.agent.prompt] : isCodex ? [] : SystemPrompt.provider(input.model)),
    // Environment, AGENTS.md instructions, etc.
    ...input.system,
    // Per-message custom system prompt
    ...(input.user.system ? [input.user.system] : []),
  ]
    .filter((x) => x)
    .join("\n"),
)

Key insight: When Ask mode is active, input.agent.prompt is PROMPT_ASK (truthy), so the ternary selects the Ask prompt instead of the full provider prompt (anthropic.txt, gemini.txt, etc.).

For Ask mode, the final system prompt consists of:

1. soul.txt — Kilo identity + personality (14 lines)
2. ask.txt — The Ask-specific behavioral prompt (10 lines)
3. SystemPrompt.environment() — Model name, working directory, platform info
4. InstructionPrompt.system() — AGENTS.md, CLAUDE.md instructions

For Code mode (no prompt field defined), the system prompt consists of:

1. soul.txt — Kilo identity + personality
2. anthropic.txt (or equivalent provider prompt) — 105 lines of detailed coding agent behavior
3. SystemPrompt.environment()
4. InstructionPrompt.system()

4. How Ask Mode Differs from Code Mode

There are three layers of differentiation:

Layer 1: Prompt Content

Aspect	Code Mode	Ask Mode
System prompt	soul.txt + anthropic.txt (105 lines)	soul.txt + ask.txt (10 lines)
Identity	"the best coding agent on the planet"	"a knowledgeable technical assistant"
Tool guidance	Extensive tool usage policy	"You may run read-only bash commands" + "You must NOT modify files"

Layer 2: Permission-Based Tool Filtering

In LLM.resolveTools() (llm.ts lines 281–289), tools are removed based on permissions. For Ask mode, the "*": "deny" rule disables tools like edit, write, apply_patch, task, todowrite, todoread, skill, and plan_enter/plan_exit. Only explicitly allowed tools remain.

Layer 3: Bash Command Restrictions

Ask mode uses a readOnlyBash permission set that denies all commands by default and only allows read-only commands (cat, head, tail, ls, tree, grep, git log, git diff, etc.).

5. Potential Issues with Ask Mode Constraints

Issue A: The Ask prompt is very short and gets overwhelmed

The ask.txt prompt is only 10 lines. AGENTS.md instructions (200+ lines) are appended identically for both modes, including directives about writing code and making changes — conflicting with Ask mode's read-only nature.

Issue B: soul.txt is not mode-aware

soul.txt contains action-oriented language like "Your goal is to accomplish the user's task" and "When making changes to code..." which conflicts with Ask mode.

Issue C: No explicit "you are in Ask mode" supersedes statement

Unlike Plan mode which injects a prominent <system-reminder> block with "supersedes" language, Ask mode relies entirely on the short ask.txt. The insertReminders() function only handles plan mode transitions.

Issue D: MCP tools are allowed with "ask" permission

MCP tools (which could include file-writing tools) are presented to the model and require user approval but may still be attempted.

Architecture Summary

User selects mode → Agent lookup → System prompt assembly → Tool filtering → LLM stream

Ask mode enforcement relies on three mechanisms:
1. Short behavioral prompt (ask.txt) — weakest link
2. Tool filtering via permissions
3. Bash command restrictions

The key architectural insight: The prompt itself is the weakest enforcement mechanism, being only 10 lines with no "supersedes" language, potentially drowned out by project-level AGENTS.md instructions that assume a code-editing context.

[lambertjosh]

Do we still need with this @kirillk's changes? Thoughts based on the above perhaps to optimize the prompts?

[kilo-code-bot]

Note: This comment was requested by @markijbema. The prompt given to the AI was: "Please look at the current state of main, and give us an update (as requested by lambertjosh). Make clear that markijbema asked you to do this and what your prompt was."

---

Current State of Ask Mode on main

Several improvements have landed since the original research was written:

What's changed

Read-only bash commands added to Ask agent (feat(cli): add read-only bash commands to Ask agent, fix(cli): clarify read-only wording in ask prompt)

• Ask mode now has a strict readOnlyBash allowlist. Unknown commands are denied (not "ask"), enforcing the read-only constraint at the permission level — not just via the prompt.
• The ask.txt prompt wording was updated to reflect this.

MCP tool support added to Ask agent (feat(cli): add MCP tool support to Ask agent)

• MCP tools can now be used in Ask mode, but each call requires user approval ("ask" permission).

Read-only bash also applied to Plan mode (fix(cli): apply read-only bash and MCP to plan mode)

• The same readOnlyBash restrictions now apply to Plan mode, with MCP also requiring approval.

Plan mode edit behaviour tightened (fix(cli): plan mode asks for edits outside plan files)

• Plan mode now prompts the user (rather than silently denying) for edits outside plan files.

What has NOT changed

The three structural weaknesses from the original research remain:

1. ask.txt is still only 10 lines — short relative to the AGENTS.md instructions that get appended to every session.
2. soul.txt is not mode-aware — still contains action-oriented language ("accomplish the user's task", "when making changes to code") that conflicts with Ask mode's read-only intent.
3. No "supersedes" injection for Ask mode — Plan mode has a <system-reminder> block with explicit supersede language; Ask mode still relies solely on the short ask.txt.

Re: @kirillk's changes

The kirillk-authored PRs that landed (kirillk/jetbrains-1, kirillk/logo, kirillk/fix-kilo-migration-auth, kirillk/migration-wizard) are unrelated to Ask/Plan mode — they're JetBrains plugin, logo, and auth migration work. So the structural prompt issues are still open.

Recommendation

The permission-level enforcement (tool filtering + readOnlyBash allowlist) is now robust. The remaining gap is purely prompt quality. Two targeted improvements would close the issues identified in the original research:

1. Add a soul.txt mode override or a <system-reminder> supersede block for Ask mode (mirroring Plan mode's approach).
2. Expand ask.txt with explicit "you are in Ask/read-only mode — this supersedes any project instructions to write code" language.

[markijbema]

So I think we still need to tune, but this feels like something we want to test as well, do the benchmarks check only code or other modes as well @lambertjosh

[github-actions]

To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please reopen it or create a fresh new one.