Repository navigation
Conversation
47c290c to
f316211
Compare
|
Hey We are cleaning up the codebase and improving the project structure for better onboarding. As part of this effort, PR #826 reorganizes No logic changes — only file moves and import path updates. What you need to doRebase your branch on git fetch origin && git rebase origin/developGit detects renames automatically. If you get import conflicts, update the paths: use crate::git; // now: use crate::cmds::git::git;
use crate::tracking; // now: use crate::core::tracking;
use crate::config; // now: use crate::core::config;
use crate::init; // now: use crate::hooks::init;
use crate::gain; // now: use crate::analytics::gain;Need help rebasing? Tag @aeppling |
RTK's json_cmd strips values from JSON arrays >5 items, showing only the first item. Multiple P0/P1 issues document user pain from silent data loss (#621, #827, #690, #488). TOON (Token-Oriented Object Notation) encodes JSON 22-59% smaller while preserving ALL values. LLM benchmarks show 76.4% accuracy vs JSON's 75.0%. For a 50-row dataset: Raw JSON: 4,214 bytes (all data) TOON: 1,963 bytes (all data, 53% smaller) RTK compact (old): 142 bytes (1 item shown, 0% data) Changes: - Add toon-rust crate for JSON-to-TOON encoding - New toon_convert module with budget-enforced conversion - json_cmd: try TOON before compact_json in data mode - curl_cmd: try TOON before schema extraction - 16KB byte budget (~4,000 tokens) with line-boundary truncation - RTK_NO_TOON=1 env var to disable - Schema mode (--schema) unchanged
f316211 to
2959d53
Compare
|
Rebased and moved |
|
This pull request has been automatically marked as stale due to 90 days of inactivity. If this pull request is still relevant, please leave any comment (for example, "bump"), and we'll keep it open. Your contribution is very much appreciated — we're sorry we haven't been able to review it yet. |
|
Closing this pull request after a prolonged period of inactivity. If this is still relevant, please ask for it to be reopened. Thank you! |
Summary
json_cmdandcurl_cmdnow try TOON encoding before compact/schema extractionRTK_NO_TOON=1disables, falling back to existing behaviortoon_convertmodule: 1 public function, 9 testsBefore (6-item array):
After:
142 bytes / 0% data → 120 bytes / 100% data.
Addresses #621 (JSON filter strips all values), #827 (silent truncation trust loss), #690 (over-compression causing 3x agent cost), #488 (no middle ground between aggressive and off).
Test plan
cargo fmt --all && cargo clippy --all-targets && cargo test— 1123 passed, 0 warningsecho '[6 items]' | rtk json -shows all items as TOONRTK_NO_TOON=1restores old compact behavior--schemamode unchanged