Skip to content

fix(diff): byte-accurate identity and LCS alignment (adopt upstream #3268) - #115

Merged
kylehgc merged 3 commits into
developfrom
fix/diff-byte-identity
Aug 17, 2026
Merged

kylehgc merged 3 commits into
developfrom
fix/diff-byte-identity

Conversation

@kylehgc

@kylehgc kylehgc commented Aug 17, 2026 •

Copy link
Copy Markdown
Owner

Adopts upstream rtk-ai/rtk#3268 by @iliaal. Closes #100.

What this fixes

Two defects in src/cmds/git/diff_cmd.rs, both reachable from a plain rtk diff a b (and from bare diff, which the hook rewrites into this path):

  1. False [ok] Files are identical with exit 0 for files whose only difference is CRLF-vs-LF or a missing final newline — identity was decided from str::lines(), which normalizes both away. A false identity silently passes any verification gate built on diff.
  2. Positional line alignment: one inserted line desynchronized every later comparison, the whole tail rendered as changed, and never_worse then dumped both files concatenated (403 lines for a 1-line insertion).

Commits

Commit Author Kind
ce07aff fix(diff): compare bytes before claiming two files are identical Ilia Alshanetsky cherry-pick of upstream c1d47d6
54cb3a1 fix(diff): align by LCS instead of comparing line N to line N Ilia Alshanetsky cherry-pick of upstream d4411cc
c7d29b0 test(diff): cover the LCS_CELL_CAP wholesale-replacement branch kylehgc fork-authored amendment (review round)

At the cherry-pick tip (54cb3a1), git diff pr-3268 HEAD -- src/cmds/git/diff_cmd.rs was empty — the adopted code is byte-identical to the upstream PR branch. The amendment adds one test on top (28 insertions, test module only) and does not touch production code.

Repro (real binaries, Windows)

Fixtures: base.txt = 200 lines; base_crlf.txt = same text, lines 50–74 CRLF; inserted.txt = base with one line inserted at the top.

$ cmp base.txt base_crlf.txt
base.txt base_crlf.txt differ: char 1041, line 50   (exit 1)

# BEFORE (installed rtk 0.42.4, pre-fix develop)
$ rtk diff base.txt base_crlf.txt
[ok] Files are identical
exit: 0

$ rtk diff base.txt inserted.txt | wc -l
403        # never_worse dumped both files concatenated

# AFTER (target/x86_64-pc-windows-msvc/release/rtk.exe from this branch)
$ rtk diff base.txt base_crlf.txt
base.txt → base_crlf.txt
   differs, text matches (line endings: 0 CRLF vs 25 CRLF)
exit: 1

$ rtk diff base.txt inserted.txt
base.txt → inserted.txt
   +1 added, -0 removed, ~0 modified

+   1 INSERTED
exit: 1

$ rtk diff lf.txt lf.txt          # truly identical files unchanged
[ok] Files are identical
exit: 0

Note: for very small differing files the never_worse guard prints the raw concatenated content instead of the "differs, text matches" message (condensed is longer than raw at that size). The exit code — the part scripts rely on — is correct in every case. Same caveat upstream rtk-ai#3471 documents.

Quality gate

cargo fmt --all (no changes) && cargo clippy --all-targets (zero warnings) && cargo test --all (2691 passed in the main suite, 0 failed across all suites), via the x64 host toolchain (scripts/win-dev-env.ps1).

No pre-existing fork tests were modified; the two cherry-picks add 11 new unit tests and the amendment adds 1 more (LCS_CELL_CAP branch).

Deliberately not adopted

iliaal and others added 3 commits August 17, 2026 18:35
render_file_diff decided identity from `content.lines()`, which strips a
trailing \r and treats the final newline as optional. So a CRLF-vs-LF or
missing-trailing-newline pair collapsed to identical line vectors and rtk
printed "[ok] Files are identical" and exited 0 for files that differ.
Reproduced with 200 lines where 24 carry CRLF: cmp reports a difference at
byte 1061 and exits 1; rtk called them identical.

That is the worst shape for a filter, because `diff` is used as a gate: a
false identity silently passes a broken check. rtk also never runs diff here
(it reimplements the comparison and synthesizes the exit code), so there was
no child status to fall back on.

Gate identity on byte equality, before lines() can normalize anything. When
the bytes differ but the text matches, name the cause (line endings, trailing
newline) rather than rendering a change list of visually identical strings.
That message deliberately avoids the word "identical": it is the signal for
true identity and must not appear for files that differ.
compute_diff walked `for i in 0..max_len` comparing lines1[i] to lines2[i], so
one inserted line desynchronized everything after it: every later pair compared
unrelated lines, the whole tail rendered as changed, and the output grew large
enough that the never_worse guard threw it away and dumped both files
concatenated instead.

Measured on a 200-line file with one line inserted at the top: 403 lines of
output before (both files, concatenated), 4 after ("+1 added, -0 removed" and
the inserted line).

Trims the common prefix and suffix first, which is what keeps the LCS table
affordable on real diffs, then runs an O(n*m) table over the middle. Past
LCS_CELL_CAP cells the middles share too little for an alignment to mean
anything, so it emits a wholesale replacement rather than allocate gigabytes.

Runs of deletions pair with the insertions that follow them, so a rewritten
line still reports as one Modified rather than a Removed plus an unrelated
Added. Line numbers now come from each line's own file instead of a shared
positional index.
Review round on fork PR #115 found the cap fallback untested: the
existing wholesale-replacement test uses 2x2 = 4 cells, so it exercises
lcs_ops, never the cap branch. Add a 2001x2001 fixture (> 4M cells) that
forces the cap path and asserts every line reports as removed + added
with line numbers from each line's own file.

Fork-authored amendment on top of the upstream rtk-ai#3268 cherry-picks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@kylehgc

kylehgc commented Aug 17, 2026

Copy link
Copy Markdown
Owner Author

Review round (pre-merge, two-axis /code-review vs develop)

Standards axis — no hard violations. Two judgement calls, both deliberately left as-is for upstream byte-parity with rtk-ai#3268:

  • Per-changed-line String allocation in the Op edit script (rust-patterns anti-pattern "clone of large strings"); a borrowed Op<'a> would avoid it. Cold path (only allocates for changed lines), not worth deviating from upstream.
  • No count_tokens ≥60% savings test in this module (cli-testing checklist). Pre-existing gap for the whole file-diff module; the savings baseline for a two-file compare is ill-defined. Left for upstream to define.

Spec axis — algorithm verified end to end (LCS backtracking hand-traced, prefix/suffix trim guards checked for overlap, CRLF counting matches the spec message, exit-1 propagation traced to std::process::exit). Two findings:

Quality gate re-run after the amendment: fmt clean, clippy zero warnings, cargo test --all green (2692 passing in main suite).

@kylehgc
kylehgc merged commit a7f2463 into develop Aug 17, 2026
5 checks passed
@kylehgc
kylehgc deleted the fix/diff-byte-identity branch August 17, 2026 22:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

adopt diff: byte-accurate identity, stop false 'identical' verdicts (upstream #3268)

2 participants