Skip to content

feat: TOON re-encoder for JSON-array outputs (gh, kubectl, aws, gcloud, az) #128

Closed

Description

@jhonatanjunio

Problem

Many cloud-CLI invocations return a top-level JSON array of uniform flat objects: gh pr list --json …, kubectl get pods -o json, aws ec2 describe-instances --output json, gcloud compute instances list --format=json. JSON repeats every field name on every row. Today CloudHandler only strips noise; the heavy redundant field repetition stays in context.

Token-Oriented Object Notation (TOON, https://github.com/toon-format/toon) is a schema-aware JSON replacement that declares the schema once in a header and lists rows CSV-style. Independent benchmarks report ~40% fewer tokens on this exact shape — typically more.

Concrete example, raw gh pr list --json number,title,state:

[
    {"number": 1, "title": "Add feature", "state": "OPEN"},
    {"number": 2, "title": "Fix bug", "state": "MERGED"}
]

TOON:

items[2]{number,title,state}:
  1,Add feature,OPEN
  2,Fix bug,MERGED

Proposal

New src/strategies/toon.rs with:

  • pub fn try_to_toon(text: &str) -> Option<String> — zero-dep hand-rolled JSON sub-parser + TOON emitter.
  • Strictly lossless-or-reject: returns None for any non-uniform / nested / out-of-order / unsmaller input.
  • CloudHandler calls it when the command requests JSON (--json, -o json, -ojson, --output json, --output=json, --format json, --format=json, -f json); otherwise pipes through to the existing pipeline unchanged.

Tradeoff

  • Slightly more code on the cloud handler path (~340 LoC including tests).
  • Zero new runtime deps.
  • Never makes a scenario worse: if the TOON output isn't smaller than the input, we return None and fall back.

Out of scope

  • NDJSON streams from docker ps --format '{{json .}}' — follow-up.
  • cargo metadata --format-version 1 — nested by design, not worth TOON.

Area

  • Handler (src/commands/*)
  • Compression pipeline (src/strategies/)

Confirmations

  • I've searched existing issues and this isn't a duplicate.
  • This respects the zero-runtime-dependency rule (only libc allowed in Cargo.toml).

Activity

  1. added a commit that references this issue on May 18, 2026
    5202d61
  2. added a commit that references this issue on May 18, 2026
    7aac0fe
  3. added a commit that references this issue on May 18, 2026
    37b006f
  4. added a commit that references this issue on Jun 18, 2026
    76dae3b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestneeds-triageAwaiting maintainer triage

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions