fix(prompts): stop showing example ids the model cannot legitimately send

[yh928]

Summary

• Six tool schemas showed an example value for an id-shaped argument. Each now names the call that issues the real id instead.
• The extract_from_result handle loses its example entirely: no example of a runtime-minted handle can ever be valid.

Problem

A sample is the shape a model reproduces, so the model sends the sample.

extract_from_result documented result_id as res_1. Two live runs called it with the literal res_1 against a cache that had never issued a handle, and both got a cache-miss instead of the data they wanted.

The same shape appeared in memory_goals edit/delete ((e.g. 'g1') — ids are sequential, so a guess deletes a real goal), the memory query source / cover window / memory-tree retrieval schemas ((e.g. \slack:#eng`)— source ids onlytree.list_sources can produce), and the browser element selector (@ref (e.g. @e1)` — refs change every navigation).

Solution

Each description now names where the real id comes from. No prohibition rules and no parser tolerance were added — the fix is what the schema shows. A rule saying "do not guess" competes with an example that shows a guessable value, and the example wins; widening the parser or the cache to accept a fabricated id would turn a wasted turn into a wrong result.

The extract_from_result case carries a code comment recording the two live runs, so the example is not reintroduced as a readability improvement.

Submission Checklist

• Tests added or updated (happy path + at least one failure / edge case)
• Diff coverage ≥ 80% — prompt-text only; the changed lines are schema strings exercised by the existing tool-schema tests
• N/A: behaviour-only change, no feature row added/removed/renamed — Coverage matrix updated
• No new external network dependencies introduced
• N/A: no release-cut surface touched — Manual smoke checklist updated
• Linked issue closed via Closes #NNN in the ## Related section

Impact

• No runtime behaviour changes; these are description strings the model reads.
• Platform-neutral. No migration.

Related

Closes #5316

Summary by CodeRabbit

• Documentation

  • Clarified that result handles, memory source IDs, goal IDs, and browser references must be copied from previously returned data.
  • Added guidance to rerun tools when result handles expire.
  • Warned that guessed goal IDs may affect goals that were not previously viewed.

• Tests

  • Added coverage to prevent fabricated result-handle examples and verify expiration guidance.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: e4f3628e-17d8-4082-b47b-7e355fc36709

📥 Commits

Reviewing files that changed from the base of the PR and between 8e65c40 and 6a8918a.

📒 Files selected for processing (8)

• src/openhuman/agent/harness/subagent_runner/extract_tool.rs
• src/openhuman/agent/harness/subagent_runner/extract_tool_tests.rs
• src/openhuman/memory/goals/schemas.rs
• src/openhuman/memory/query/cover_window.rs
• src/openhuman/memory/query/query_source.rs
• src/openhuman/memory/tools/goals.rs
• src/openhuman/memory/tree/retrieval/schemas.rs
• src/openhuman/tools/impl/browser/browser_part_03.rs

🚧 Files skipped from review as they are similar to previous changes (5)

• src/openhuman/memory/goals/schemas.rs
• src/openhuman/memory/query/cover_window.rs
• src/openhuman/memory/tree/retrieval/schemas.rs
• src/openhuman/memory/query/query_source.rs
• src/openhuman/agent/harness/subagent_runner/extract_tool.rs

Included review availability: Your plan provides up to 10 included reviews per hour; 7 remain after this review.

---

📝 Walkthrough

Walkthrough

Tool schemas now direct callers to copy runtime-issued identifiers from prior results, source listings, goal lists, or page snapshots. Fabricated identifier examples were removed, and extraction-tool tests verify the updated documentation.

Changes

Runtime identifier documentation

Layer / File(s)	Summary
Result handle documentation src/openhuman/agent/harness/subagent_runner/extract_tool.rs, src/openhuman/agent/harness/subagent_runner/extract_tool_tests.rs	Extraction-tool documentation now requires previously emitted, non-evicted handles. Tests reject fabricated handle literals and verify expiry guidance.
Memory identifier documentation src/openhuman/memory/goals/..., src/openhuman/memory/query/..., src/openhuman/memory/tools/goals.rs, src/openhuman/memory/tree/retrieval/schemas.rs	Memory schemas now require goal IDs from read goal lists and source IDs copied from tree.list_sources.
Browser reference documentation src/openhuman/tools/impl/browser/browser_part_03.rs	The selector schema now requires @ref values copied from the latest page snapshot.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: senamakel

Merge Risk: ⚪ Minimal · up to 6a891

This change replaces misleading example identifiers with guidance to obtain real runtime-issued values, without changing runtime behavior; no actionable merge-blocking risk remains beyond normal checks and review.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 44.44% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 18 functions across 10 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the primary change: removing misleading example IDs from tool prompts and schemas.
Linked Issues check	✅ Passed	The pull request replaces fabricated examples with references to valid runtime ID sources across extract handles, goal IDs, memory source IDs, and browser references. The added tests verify that fabri…
Out of Scope Changes check	✅ Passed	All changes support the linked issue. The schema updates, helper extraction, and focused tests are related to preventing models from copying fabricated runtime IDs.

Full details: Linked Issues check

Explanation

The pull request replaces fabricated examples with references to valid runtime ID sources across extract handles, goal IDs, memory source IDs, and browser references. The added tests verify that fabricated handle examples are absent.

---

_{A rabbit reads the schemas bright,
No guessed handles hop in sight.
From lists and snapshots, IDs appear,
Expired trails say, “rerun here.”
Clean tool words now guide the way.}

_{Comment @coderabbitai help to get the list of available commands.}

[greptile-apps]

Greptile Summary

This PR fixes a class of model hallucination bugs caused by example identifiers in tool schema descriptions. When a schema shows result_id="res_1" or id='g1', the model treats those literals as templates and reproduces them verbatim — causing cache misses on extract_from_result (documented by two live production runs), silent mutations of wrong goals on memory_goals edit/delete, and stale selector refs in browser interactions.

• extract_from_result: removes the res_1 example entirely and adds a code comment recording the two live misuse runs; the description now instructs the model to copy the handle from an actual placeholder.
• memory_goals edit/delete (both schemas.rs and tools.rs): replaces (e.g. 'g1') with "taken from a goals list you have read"; the tools.rs version also adds an explicit warning that sequential IDs mean a guessed value mutates a real goal.
• memory/query and memory/tree/retrieval: replaces the slack:#eng / gmail:abc examples with "copied from tree.list_sources" across four description sites.
• browser.rs: changes @ref (e.g. @e1) to "@ref copied from the latest page snapshot", reflecting that refs are navigation-scoped and cannot be guessed.

Confidence Score: 5/5

Safe to merge — all changes are description strings with no runtime logic modifications.

Every changed line is a description or comment string that the model reads but the runtime never parses. No control flow, data model, or API surface is touched. The fix is narrowly scoped and directly addresses two documented production failures.

Files Needing Attention: No files require special attention. The minor wording difference between schemas.rs (no sequential-ID risk note) and tools.rs (includes the warning) is an intentional style difference between the RPC controller schema and the agent tool schema surfaces.

Important Files Changed

Filename	Overview
src/openhuman/agent/harness/subagent_runner/extract_tool.rs	Removes the res_1 example from the result_id parameter description; adds a code comment recording two live production misuse runs to prevent the example from being re-introduced.
src/openhuman/memory/goals/schemas.rs	Replaces "(e.g. 'g1')" with "taken from a goals list you have read" for edit and delete id fields; the sequential-ID risk warning present in tools.rs is not repeated here.
src/openhuman/memory/goals/tools.rs	Replaces "(e.g. 'g1')" with instructional text plus an explicit "Ids are sequential, so a guessed one lands/deletes a real goal you never looked at" warning for both edit and delete.
src/openhuman/memory/query/cover_window.rs	Replaces the slack:#eng / gmail:abc example with "copied from tree.list_sources" for the source_id field.
src/openhuman/memory/query/query_source.rs	Same source_id description fix as cover_window.rs — example replaced with "copied from tree.list_sources".
src/openhuman/memory/tree/retrieval/schemas.rs	Two source_id FieldSchema comment strings updated — both drop the slack:#eng / gmail:abc examples in favour of "copied from tree.list_sources".
src/openhuman/tools/impl/browser/browser.rs	Changes @ref (e.g. @e1) to "@ref copied from the latest page snapshot" in the selector description, preventing the model from guessing stale element references across navigations.

Sequence Diagram

sequenceDiagram
    participant M as Model
    participant S as Schema description
    participant T as Tool runtime
    participant C as Cache / Store

    Note over M,C: Before this PR
    S-->>M: result_id handle example res_1
    M->>T: extract_from_result result_id res_1
    T->>C: lookup res_1
    C-->>T: cache-miss
    T-->>M: Error

    Note over M,C: After this PR
    S-->>M: Copy handle from earlier result
    M->>T: some_tool returns placeholder result_id res_abc42
    M->>T: extract_from_result result_id res_abc42
    T->>C: lookup res_abc42
    C-->>T: hit
    T-->>M: extracted answer

Loading

_{Reviews (2): Last reviewed commit: "fix(prompts): stop showing example ids t..." | Re-trigger Greptile}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/openhuman/agent/harness/subagent_runner/extract_tool.rs`:
- Line 192: Update the description of the oversized tool output placeholder
handle to state that emitted handles may expire after cache eviction; when a
handle is no longer valid, rerun the original tool to obtain a new result
instead of sending the expired handle.
- Around line 170-183: Add focused semantic regression tests for the
runtime-identifier model-facing descriptions: in
src/openhuman/agent/harness/subagent_runner/extract_tool.rs:170-183 verify exact
placeholder-handle copying and no fabricated handles, and at 192 verify no-stash
guidance; test source_id guidance in cover_window at
src/openhuman/memory/query/cover_window.rs:43 and query_source at
src/openhuman/memory/query/query_source.rs:31; test the query_source and
cover_window controller schemas in
src/openhuman/memory_tree/retrieval/schemas.rs:106 and 168; test edit and delete
goal-ID guidance in src/openhuman/memory_goals/schemas.rs:94 and 118; test
GoalsEditTool and GoalsDeleteTool in src/openhuman/memory_goals/tools.rs:133 and
185; and test snapshot references plus CSS/text selector support in
src/openhuman/tools/impl/browser/browser.rs:773.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 4f6e103a-c320-4966-9c52-24c5a1e06a35

📥 Commits

Reviewing files that changed from the base of the PR and between a52a599 and ef5be20.

📒 Files selected for processing (7)

• src/openhuman/agent/harness/subagent_runner/extract_tool.rs
• src/openhuman/memory/query/cover_window.rs
• src/openhuman/memory/query/query_source.rs
• src/openhuman/memory_goals/schemas.rs
• src/openhuman/memory_goals/tools.rs
• src/openhuman/memory_tree/retrieval/schemas.rs
• src/openhuman/tools/impl/browser/browser.rs

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

[greptile-apps]

yh928 has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

[yh928]

Markdown Link Check is red on this head, and it is not this branch: the run reports 0 errors and 10 timeouts against external hosts, and the branch touches no markdown at all (seven .rs files). It needs a re-run rather than a change here — I don't have permission to trigger one.

Everything else on this head is green, and both review threads are addressed in e4df37cbe.

[greptile-apps]

yh928 has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

[yh928]

Correction to my earlier comment: the link check has since passed on this head, so no re-run is needed. It was a transient timeout against external hosts (the failing run reported 0 errors and 10 timeouts), and the branch touches no markdown.

[M3gA-Mind]

Maintainer review (review only — no changes pushed to your branch).

The change is right and I would like to see it land. The argument in the description is the correct one: a schema description is a prompt, an example id in it is an instruction, and adding a "do not guess" rule alongside a guessable sample loses to the sample. Not widening the parser or the cache to accept a fabricated id is also the right call — that converts a wasted turn into a wrong result. The extract_from_result code comment recording the two live res_1 runs is what stops the example being reintroduced later as a readability improvement, and the two new tests pin both halves of the contract (no handle-shaped literal in model-facing text; the text still says where a real handle comes from and that it expires).

It is CONFLICTING and needs a rebase. I resolved it locally to see exactly what is involved so I could tell you precisely, then discarded that work rather than pushing to your fork. Your branch is untouched at 2a4f0ae4b. Two conflicts, both from the same cause — #5856/#5857 split large files into _part_NN.rs fragments stitched with include!():

1. src/openhuman/tools/impl/browser/browser.rs — the whole body moved out; the file is now a header plus include!("browser_part_01.rs")…_03. Your one-line selector change now belongs at:

src/openhuman/tools/impl/browser/browser_part_03.rs:36

The string is identical there, so it is a straight re-target, no re-wording needed.

2. src/openhuman/agent/harness/subagent_runner/extract_tool.rs — main moved the inline mod tests { … } out to extract_tool_tests.rs. Keep main's form:

#[cfg(test)]
#[path = "extract_tool_tests.rs"]
mod tests;

and move only your two new tests — no_model_facing_text_shows_a_fabricated_handle and the_handle_argument_says_where_a_handle_comes_from_and_that_it_expires — into extract_tool_tests.rs, de-indented one level. Do not carry the three chunk_budget_* tests across: they already exist in extract_tool_tests.rs on main (lines 7, 23, 36), so bringing them along duplicates them. extract_tool_tests.rs opens with use super::*;, and your description_text() / parameters_schema_json() helpers are in extract_tool.rs proper, so they stay in scope with no import changes.

Everything else in the PR — memory/goals/schemas.rs, memory/query/cover_window.rs, memory/query/query_source.rs, memory/tree/retrieval/schemas.rs — merges cleanly. Note memory/goals/tools.rs is now memory/tools/goals.rs; git follows that rename on its own.

After the rebase the net diff is 8 files, +90/−30. Its last CI run was green, so I would expect it green again once rebased. Not approving — a maintainer will review and merge.

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

[yh928]

Rebased onto main (8e65c4008) and pushed — 6a8918a04. Thank you for resolving it locally to tell me precisely what was involved, and for discarding it rather than pushing to the fork — the two re-targets you named were exactly right, and there was a third you couldn't have hit.

The three moves:

your diff	new home
tools/impl/browser/browser.rs:770	browser_part_03.rs:36 — identical string, straight re-target as you said
subagent_runner/extract_tool.rs inline mod tests	extract_tool_tests.rs, keeping main's #[path] form
memory/goals/tools.rs	memory/tools/goals.rs — the file moved; git followed the rename on its own

Everything else (memory/goals/schemas.rs, query/cover_window.rs, query/query_source.rs, tree/retrieval/schemas.rs) applied unchanged.

One thing worth noting about the test move. The auto-merge silently resolved the test module to main's #[path] form and dropped both new tests on the floor — the rebase looked clean and the contract they pin was simply gone. I re-extracted them into extract_tool_tests.rs; no_model_facing_text_shows_a_fabricated_handle and the_handle_argument_says_where_a_handle_comes_from_and_that_it_expires both run and pass. Flagging it because that class of loss is invisible in a conflict-free rebase.

Verified locally: cargo test --lib extract_tool — 6/6 pass. cargo fmt --check, layout gate, and the full pre-push suite clean.

The substance is unchanged, and the extract_from_result comment recording the two live res_1 runs is still there — which, as you noted, is what stops the example being reintroduced later as a readability improvement.

[tinysweeper]

tinysweeper found nothing blocking. Approving.

          $0.0091 · 119,479 in / 934 out · 23,552 cached (20%) · openrouter/openai/text-embedding-3-small, deepseek/deepseek-v4-flash · 429 embedded
critique: $0.0036 · 58,591 in  / 329 out · 23,552 cached (40%) · deepseek/deepseek-v4-flash
security: $0.0042 · 46,716 in  / 300 out · 0 cached (0%)       · deepseek/deepseek-v4-flash
tests:    $0.0013 · 14,172 in  / 305 out · 0 cached (0%)       · deepseek/deepseek-v4-flash

[tinysweeper]

How this change flows

2 changed behaviours across 7 relationships. 6 surrounding behaviours are shown (60 graph nodes walked). 27 further behaviours left out to keep the diagram readable.

flowchart LR
  n0["schemas<br/>changed"]:::changed
  n1["schemas<br/>changed"]:::changed
  n2["vec"]:::impacted
  n3["format"]:::impacted
  n4["join"]:::impacted
  n5["openhuman"]:::impacted
  n6["caps_tools_ws"]:::impacted
  n7["expansion_tools_for"]:::impacted
  n0 -->|calls| n2
  n1 -->|calls| n2
  n4 -->|calls| n3
  n6 -->|calls| n3
  n6 -->|calls| n4
  n7 -->|calls| n2
  n7 -->|uses| n5
  classDef changed fill:#0d4429,stroke:#238636,color:#e6edf3
  classDef impacted fill:#161b22,stroke:#6e7681,color:#c9d1d9
  classDef flagged fill:#5a1e02,stroke:#d93f0b,color:#ffffff
  classDef blocking fill:#67060c,stroke:#f85149,color:#ffffff

Loading

_{Green: changed behaviour. Grey: surrounding behaviour. Arrows name the call, use, implementation, or test relationship. Orange: has findings. Red: has a finding that blocks the merge.}

[chatgpt-codex-connector]

You have reached your Codex usage limits for security reviews. Please try again later.

[chatgpt-codex-connector]

You have reached your Codex usage limits for security reviews. Please try again later.

[chatgpt-codex-connector]

You have reached your Codex usage limits for security reviews. Please try again later.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8ff1958a80

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

You have reached your Codex usage limits for security reviews. Please try again later.

[chatgpt-codex-connector]

You have reached your Codex usage limits for security reviews. Please try again later.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 055a644e98

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bb12219f19

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Add source listing to the memory agent's allowlist

Fresh evidence after the earlier native-surface report is that this commit registers the wrapper globally, but the deep-retrieval agent_memory still has a named allowlist containing memory_tree but not memory_tree_list_sources (src/openhuman/memory/agent/agent/agent.toml:24-36). The subagent runner constructs allowed_names solely from that resolved scope (src/openhuman/agent/harness/subagent_runner/ops/runner.rs:1270-1279), so delegated memory runs neither see nor can execute this new tool. When retrieval requires an exact provider-specific source_id, the agent therefore still cannot obtain it despite the updated schemas directing it to this name; include the tool in the memory agent's allowlist.

Useful? React with 👍 / 👎.