Skip to content

fix(kiro): prioritize tool search results within catalog budget - #2475

Merged
lidge-jun merged 1 commit into
lidge-jun:devfrom
mchlkim:codex/fix-kiro-tool-search-priority
Aug 25, 2026
Merged

lidge-jun merged 1 commit into
lidge-jun:devfrom
mchlkim:codex/fix-kiro-tool-search-priority

Conversation

@mchlkim

@mchlkim mchlkim commented Aug 24, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Prioritize definitions restored from tool_search when Kiro's outbound catalog exceeds its count or byte budget.
  • Keep the tool_search gateway immediately behind restored definitions so deferred discovery remains available.
  • Preserve caller order within each priority tier and keep the existing 48-tool / 96 KiB limits unchanged.
  • Add a regression with 68 ordinary tools, one search gateway, and one restored definition.

Fixes #2407

Verification

  • bun test tests/kiro-adapter.test.ts — 56 passed, 0 failed (Bun 1.3.14)
  • bun test tests/server-kiro-completion-e2e.test.ts — 4 passed, 0 failed (unrestricted loopback E2E)
  • bun run typecheck — passed
  • bun run privacy:scan — passed
  • git diff --check upstream/dev...HEAD — passed
  • bun run test in the sandbox is not representative: loopback E2E setup is denied with EADDRINUSE / EPERM before test logic runs. An unrestricted full-suite result is still required to tick the first readiness item.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed. (No documentation change needed; this implements the priority contract already documented on OcxTool.loadedFromToolSearch.)
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults.

Review readiness checklist

This PR stays in draft until every box below is ticked. Tick all four boxes once the requirements are met:

  • All CI tests are green on my local testing. (Focused and Kiro server E2E checks pass; unrestricted full-suite verification remains pending.)
  • I pushed my PR to the latest dev commit.
  • I resolved all correct Codex and CodeRabbit findings.
  • My PR is ready for review.

Summary by CodeRabbit

  • New Features
    • Improved tool selection when catalogs exceed size or count limits.
    • Prioritizes search-loaded tools and the search gateway while preserving existing ordering for other tools.
    • Reports omitted tools more accurately when limits are reached.

@coderabbitai

coderabbitai Bot commented Aug 24, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 5db66203-717b-4537-8ae4-d8bab84314a3

📥 Commits

Reviewing files that changed from the base of the PR and between bb89eaf and 404151f.

📒 Files selected for processing (2)
  • src/adapters/kiro-tools.ts
  • tests/kiro-adapter.test.ts

📝 Walkthrough

Walkthrough

The Kiro adapter now prioritizes tools loaded by tool_search, then the tool_search gateway, before ordinary tools when count or byte limits require truncation. Tests verify ordering, limits, and omission reporting.

Changes

Kiro catalog budgeting

Layer / File(s) Summary
Priority-based catalog selection
src/adapters/kiro-tools.ts
The adapter ranks loaded tool-search tools first, the tool_search gateway second, and ordinary tools last. It preserves original order within each priority group and applies the ordering only when count or byte limits are exceeded.
Catalog prioritization regression coverage
tests/kiro-adapter.test.ts
The test confirms that prioritized tools occupy the available slots, ordinary tools retain their order, the catalog count limit is enforced, and omission notices list only omitted ordinary tools.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: ingwannu, lidge-jun

✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

✅ Deterministic PR hygiene checks passed.

@github-actions github-actions Bot added the bug Something isn't working label Aug 24, 2026
@github-actions

github-actions Bot commented Aug 24, 2026 •

Copy link
Copy Markdown
Contributor

⏳ DRAFT

  • review readiness checklist open (2/4 boxes ticked).

What to do

  • Tick all four boxes in the PR description once you're done (currently 2/4).

Review readiness checklist

  • ⬜ All CI tests are green on my local testing.
  • ✅ I pushed my PR to the latest dev commit.
  • ✅ I resolved all correct Codex and CodeRabbit findings.
  • ⬜ My PR is ready for review.

2/4 boxes ticked.

Automatic draft conversion failed. Please convert this pull request to a draft manually until every box above is ticked.

@lidge-jun

Copy link
Copy Markdown
Owner

리뷰 · 우선순위 64 / 80

설명: 이 풀 리퀘스트는 키로가 도구 검색으로 되살린 정의를 카탈로그 한도 안에서 앞에 두는 고침이다. 작성자는 mchlkim 이다. 오늘 열일곱 시 삼 분에 열렸다. 초안이다. 점검 네 칸이 비어 있다. 위생은 통과다. 베이스는 지금 HEAD c44e43f00 다. 파일 둘이다. src/adapters/kiro-tools.ts 더하기 20 빼기 9, tests/kiro-adapter.test.ts 더하기 35 이다. Fixes 2407 이다. 지금 CURRENT dev HEAD 는 c44e43f00 이다. 이번 시간에 origin/dev 는 그대로다. 새 머지는 없다. package.json 은 2.27.0 이다. src/config.ts 는 3238줄이다. src/runtime 폴더는 지금 HEAD 에 없다. combo-stream-preflight.ts 는 171줄이고 HEAD 에 있다. gui/src/combo-workspace-data.ts 는 589줄이다. src/providers/default-aliases.ts 와 model-presets.ts 는 아직 없다. src/codex/history-provider.ts 는 1557줄이다. src/server/responses/fetch-helpers.ts 는 137줄이다. src/codex/history-manifest.ts 는 112줄이다.

지금 HEAD 를 열었다. src/responses/parser.ts 743줄은 검색으로 되살린 정의에 loadedFromToolSearch 를 붙인다. 그 표시는 이미 있다. 구멍은 키로 변환이다. src/adapters/kiro-tools.ts 10줄 한도는 마흔여덟 개다. 11줄 바이트 한도는 구만육천이다. 175줄 convertKiroToolContext 는 받은 순서대로 넣는다. 201줄 주석은 순위를 매기지 말고 뒤만 자르라고 적혀 있다. 204줄에서 개수나 바이트가 넘으면 그 자리에서 끊는다. 검색으로 돌아온 정의는 파서가 목록 뒤에 붙인다. 한도를 넘기면 그 정의가 잘린다. 모델은 방금 찾은 도구를 쓸 수 없다고 말한다. 2407 이 그 구멍이다. 이미 리뷰가 있다. 다시 리뷰하지 않는다. 이 PR 이 착지하기 전에는 2407 을 닫지 말 것.

이 PR 은 한도를 넘길 때만 순서를 바꾼다. 우선은 검색으로 되살린 정의, 그 다음 검색 관문, 그 다음 일반 도구다. 같은 칸 안에서는 호출 순서를 지킨다. 한도 안이면 원래 순서를 그대로 둔다. 마흔여덟과 구만육천은 안 바꾼다. 시험은 일반 도구 예순여덟, 검색 관문 하나, 되살린 정의 하나를 넣는다. 나가는 마흔여덟의 앞이 되살린 정의와 관문인지 본다. 생략 알림에 일반 도구 이름만 남는지 본다. 구멍의 모양과 같다. 바이트만 넘기는 길은 이 시험이 안 잠근다. 작성자는 모아 둔 시험이 다른 파일에서 멈췄다고 적었다. 깃허브 점검 네 칸은 비어 있다. 초안이다. 지금 머지하지 말 것.

커서 쪽 src/adapters/cursor/request-builder.ts 50줄은 이미 검색 되살림을 순위에 넣는다. 다만 실행 길이와 wait 와 apply_patch 보다 뒤다. 키로에는 그 고정 칸이 없다. 이 PR 이 되살린 정의를 맨 앞에 두는 것은 2407 에 맞다. 한도를 안 넘기면 순위를 안 매기는 것도 맞다. 201줄 뒤만 자르기 정책은 이 구멍 때문에 바뀐다. 그건 이 이슈의 목적이다. types.ts/config.ts 가르기와 상관없다. 닫고 다시 밑지 말 것. 프리뷰 배포가 아니다.

사용자 길이로는 키로 큰 카탈로그에서 검색으로 찾은 도구가 바로 다음 턴에 사라져서 64. 초안이라 지금 착지하면 안 된다. 2407 은 이 PR 이 착지한 뒤에 닫는다. 2474 는 스크립트 작업 폴더 고침이고 이 구멍과 다르다. 2473 은 웹소켓 초안이다. 2472 는 연다. 2463 과 2464 과 2465 는 구현 파일이 없어서 닫지 않는다. 2468 은 메인에 넣지 말 것. 2451 과 2467 과 2471 은 다시 열지 말 것. 카탈로그 팁은 Ox Alpha x-preview-f-free + deepseek-v4-flash-vision-exp. Cursor 정적 카탈로그는 opus-4-8-fast / opus-5-fast. 2334 CursorCredentialRouter 는 여전히 src/providers/cursor-pool.ts 모듈+테스트만 있고 어댑터에 연결되지 않았다. 2332 H2 는 discovery 전용. 2320 overflow + 2342 는 이미 dev. 2188 사이드카는 이미 dev. 2361 __omit__ 는 이미 HEAD 다. src/lib/bounded-body.ts 128줄은 이미 죽은 신호에서 취소 없이 던진다. 2453 이 착지해서 wait 허용 목록은 이제 밑줄 yield_time_ms 와 max_tokens 이다. types.ts/config.ts 가르기는 이 구멍을 바꾸지 않는다. 닫고 다시 밑지 말 것.

src/adapters/kiro-tools.ts 라인 201 - HEAD 주석은 순위를 매기지 말고 뒤만 자르라고 한다. 이 PR 이 한도 초과일 때만 순위를 켠다
src/adapters/kiro-tools.ts 라인 204 - HEAD 는 개수나 바이트가 넘으면 그 자리에서 끊는다. 뒤에 붙은 검색 정의가 잘린다
src/responses/parser.ts 라인 743 - 검색으로 되살린 정의에 loadedFromToolSearch 를 이미 붙인다. 구멍은 여기가 아니다
src/adapters/kiro-tools.ts 라인 10 - 한도는 마흔여덟 개다. 이 PR 이 숫자를 바꾸지 않는다
src/adapters/kiro-tools.ts 라인 11 - 바이트 한도는 구만육천이다. 이 PR 이 숫자를 바꾸지 않는다
tests/kiro-adapter.test.ts - HEAD 는 이 순위 시험이 없다. 이 PR 이 예순여덟 일반 도구와 관문과 되살린 정의로 잠근다
이슈 2407 - 이 PR 이 고친다고 적었다. 착지 전에는 이슈를 닫지 않는다
점검 네 칸 - 모두 비어 있다. 초안이다. 지금 머지하지 말 것

메인테이너의 판단이 필요한 지점

  • 한도를 넘길 때만 순위를 켤지. 켜는 편이 맞다. 한도 안 순서는 그대로 둔다
  • 되살린 정의를 키로에서 맨 앞에 둘지. 두는 편이 맞다. 2407 이 그 구멍이다
  • 바이트만 넘는 길을 이 기차 시험에 넣을지. 넣지 않아도 된다. 다음이어도 된다
  • 점검 네 칸을 채우기 전에 머지할지. 지금은 머지하지 말 것
  • 착지 전에 2407 을 닫을지. 닫지 말 것
  • types.ts/config.ts 가르기로 이 PR 을 닫을지. 해당 없다. 키로 변환 한 파일이다

너의 추천
초안으로 둔다. 지금 머지하지 말 것. 구멍은 맞다. 점검 네 칸과 깃허브 초록이 채워진 뒤에 본다. 착지하면 2407 을 이 PR 로 닫는다. 한도 숫자와 바이트는 그대로 둔다. 한도 안이면 순위를 켜지 말 것. 라벨은 그대로 둔다. 2426 과 2472 와 2463 과 2464 과 2465 는 닫지 않는다. 2473 은 초안이니 머지하지 말 것. 2474 는 리눅스 CI 가 새 시험을 돌리기 전에는 머지하지 말 것. 2468 은 메인에 넣지 말 것. 호출 길을 넓히지 말 것. 프리뷰 배포가 아니다.

이 댓글은 grok-bot이 작성했습니다

@mchlkim
mchlkim force-pushed the codex/fix-kiro-tool-search-priority branch from 6b2c28f to 404151f Compare August 25, 2026 01:16
@mchlkim

mchlkim commented Aug 25, 2026

Copy link
Copy Markdown
Contributor Author

Updated after the draft-gate feedback:

  • Rebasing is complete: this branch now includes current dev HEAD bb89eaf.
  • Focused adapter test: 56 passed, 0 failed.
  • Kiro completion E2E: 4 passed, 0 failed in an unrestricted loopback environment.
  • Typecheck, privacy scan, and diff check passed.
  • The official readiness checklist is now accurate. I am intentionally keeping this PR as Draft until an unrestricted full-suite run is green.

@lidge-jun
lidge-jun marked this pull request as ready for review August 25, 2026 03:24
@lidge-jun
lidge-jun merged commit 0906201 into lidge-jun:dev Aug 25, 2026
7 of 8 checks passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 404151fb51

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

|| serializedToolCatalogBytes([...convertedTools, entry.converted]) > MAX_KIRO_TOOL_CATALOG_BYTES
) {
omittedAt = index;
break;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Continue after a priority tool exceeds the remaining byte budget

When multiple restored tools have large schemas, one may fit while the next exceeds the remaining 96 KiB budget; this break then drops every later candidate, including the small tool_search gateway and ordinary tools that would still fit. An individually oversized restored tool similarly reduces the advertised catalog to zero. Continue scanning after a byte-budget miss and track omitted entries individually rather than treating the first miss as the start of an omitted suffix.

AGENTS.md reference: src/AGENTS.md:L19-L19

Useful? React with 👍 / 👎.


function boundedCatalogPriority(tool: OcxTool): number {
if (tool.loadedFromToolSearch) return 0;
if (tool.toolSearch) return 1;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reserve a catalog slot for the search gateway

When a continuation contains at least 48 definitions restored from prior tool_search_output items, every restored definition sorts ahead of the gateway and fills the count limit, so tool_search is omitted and the session can no longer discover additional deferred tools. Since src/responses/parser.ts accumulates restored definitions across the input history, this state can persist on subsequent turns; pin the gateway ahead of restored definitions or reserve one of the 48 slots for it.

AGENTS.md reference: src/AGENTS.md:L19-L19

Useful? React with 👍 / 👎.

lidge-jun pushed a commit that referenced this pull request Aug 27, 2026
Under Codex code mode, shell, file edits, apply_patch and every MCP helper are
reachable only as nested tools.<name>(...) calls inside one freeform exec tool.
Kiro's catalog budget ranked that tool as ordinary filler, so a crowded session
could drop it and emit a catalog in which every admitted tool is uncallable.
Cursor pins its execution path for the same reason (request-builder.ts, #399).

Priority alone does not fix it. loadedFromToolSearch tools outrank exec and
arrive unbounded -- the Responses parser pushes every tool_search_output spec --
so a session holding MAX_KIRO_TOOL_COUNT loaded tools exhausts the count budget
before reaching the exec tier and drops the one tool that makes the other 48
callable.

Reserve instead of evict. The fill loop projects exec into every count and byte
check, so it admits one fewer tool rather than taking one back. That distinction
is what keeps #2475's loaded-result guarantee intact: Cursor's
evictNonExecutionPath exempts only the execution path and could evict a loaded
tool. Byte checks measure the projected final array because the budget is
computed over the serialized array, where separators shift.

Omissions are now derived by set difference. The previous
candidates.slice(omittedAt) assumed every candidate after the first rejection was
omitted, which stops being true once one of them is reserved and admitted -- the
notice would have named exec unavailable while it was on the wire. Emitted order
is rebuilt from the sorted candidates so the wire order stays
loaded -> exec -> gateway -> filler.

The wp1 test asserting an omitted exec is never named now uses a structured exec,
which stays ordinary filler and stays droppable, so both invariants keep coverage.

Plan and audit: devlog/_plan/260827_kiro_subagent_delegation_unblock/040, /041
tarunravi pushed a commit to tarunravi/opencodex that referenced this pull request Sep 14, 2026
tarunravi pushed a commit to tarunravi/opencodex that referenced this pull request Sep 14, 2026
Under Codex code mode, shell, file edits, apply_patch and every MCP helper are
reachable only as nested tools.<name>(...) calls inside one freeform exec tool.
Kiro's catalog budget ranked that tool as ordinary filler, so a crowded session
could drop it and emit a catalog in which every admitted tool is uncallable.
Cursor pins its execution path for the same reason (request-builder.ts, lidge-jun#399).

Priority alone does not fix it. loadedFromToolSearch tools outrank exec and
arrive unbounded -- the Responses parser pushes every tool_search_output spec --
so a session holding MAX_KIRO_TOOL_COUNT loaded tools exhausts the count budget
before reaching the exec tier and drops the one tool that makes the other 48
callable.

Reserve instead of evict. The fill loop projects exec into every count and byte
check, so it admits one fewer tool rather than taking one back. That distinction
is what keeps lidge-jun#2475's loaded-result guarantee intact: Cursor's
evictNonExecutionPath exempts only the execution path and could evict a loaded
tool. Byte checks measure the projected final array because the budget is
computed over the serialized array, where separators shift.

Omissions are now derived by set difference. The previous
candidates.slice(omittedAt) assumed every candidate after the first rejection was
omitted, which stops being true once one of them is reserved and admitted -- the
notice would have named exec unavailable while it was on the wire. Emitted order
is rebuilt from the sorted candidates so the wire order stays
loaded -> exec -> gateway -> filler.

The wp1 test asserting an omitted exec is never named now uses a structured exec,
which stays ordinary filler and stays droppable, so both invariants keep coverage.

Plan and audit: devlog/_plan/260827_kiro_subagent_delegation_unblock/040, /041
agentHits pushed a commit to agentHits/opencodex that referenced this pull request Sep 17, 2026
agentHits pushed a commit to agentHits/opencodex that referenced this pull request Sep 17, 2026
Under Codex code mode, shell, file edits, apply_patch and every MCP helper are
reachable only as nested tools.<name>(...) calls inside one freeform exec tool.
Kiro's catalog budget ranked that tool as ordinary filler, so a crowded session
could drop it and emit a catalog in which every admitted tool is uncallable.
Cursor pins its execution path for the same reason (request-builder.ts, lidge-jun#399).

Priority alone does not fix it. loadedFromToolSearch tools outrank exec and
arrive unbounded -- the Responses parser pushes every tool_search_output spec --
so a session holding MAX_KIRO_TOOL_COUNT loaded tools exhausts the count budget
before reaching the exec tier and drops the one tool that makes the other 48
callable.

Reserve instead of evict. The fill loop projects exec into every count and byte
check, so it admits one fewer tool rather than taking one back. That distinction
is what keeps lidge-jun#2475's loaded-result guarantee intact: Cursor's
evictNonExecutionPath exempts only the execution path and could evict a loaded
tool. Byte checks measure the projected final array because the budget is
computed over the serialized array, where separators shift.

Omissions are now derived by set difference. The previous
candidates.slice(omittedAt) assumed every candidate after the first rejection was
omitted, which stops being true once one of them is reserved and admitted -- the
notice would have named exec unavailable while it was on the wire. Emitted order
is rebuilt from the sorted candidates so the wire order stays
loaded -> exec -> gateway -> filler.

The wp1 test asserting an omitted exec is never named now uses a structured exec,
which stays ordinary filler and stays droppable, so both invariants keep coverage.

Plan and audit: devlog/_plan/260827_kiro_subagent_delegation_unblock/040, /041
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants