Skip to content

feat(windows): add native commands and PowerShell routing - #3301

Open
Luca-it15 wants to merge 14 commits into
rtk-ai:developfrom
Luca-it15:feat/windows-powershell-native-compat
Open

Luca-it15 wants to merge 14 commits into
rtk-ai:developfrom
Luca-it15:feat/windows-powershell-native-compat

Conversation

@Luca-it15

Copy link
Copy Markdown

Summary

  • refresh and carry the native Windows work from feat(system): native Windows ls/grep/wc/tree + PowerShell cmdlet rewrites #2547 onto the current develop branch
  • add pure-Rust fallbacks for ls, tree, wc, and file-based grep/rg when Unix binaries are unavailable
  • add powershell and pwsh wrappers plus safe, case-insensitive routing for Get-Content, Get-ChildItem, and Select-String (including aliases)
  • preserve unsupported scripts, pipelines, interpolation, -File, and -EncodedCommand by passing them through unchanged
  • support non-MSVC Windows linkers instead of always emitting the MSVC-only /STACK flag
  • honor -v / --invert-match in the native search fallback so Select-String -NotMatch remains correct

Context and attribution

This branch deliberately preserves the original commits/authorship from @tienthegainz's #2547 and @sresam89's #2875, then adds the PowerShell/pwsh routing and current-develop integration work on top.

Addresses #1248.
Supersedes #2547.
Includes the fallback work from #2875.

Safety model

Only simple, statically tokenizable cmdlet invocations are routed through RTK. Compound PowerShell syntax and dynamic forms are left to the requested PowerShell executable, avoiding semantic changes to pipelines or scripts.

Validation

  • cargo fmt --all --check
  • cargo clippy --all-targets -- -D warnings
  • 77 focused Windows/PowerShell tests passed (powershell, native ls, tree, wc, and search fallback)
  • manual Windows smoke tests for powershell -Command, native listing/count/tree, and search with rg removed from PATH
  • full local gnullvm run: 2505 passed; the remaining 30 failures are outside the modified modules and depend on Unix helper commands or sandboxed user-profile writes (the upstream base does not link with gnullvm before the linker fix in this PR)

GitHub's existing Windows MSVC job provides the authoritative full-suite check for this draft.

AI disclosure

The integration, tests, and PowerShell routing were developed with Codex assistance and manually validated on Windows.

tienthegainz and others added 7 commits July 29, 2026 18:32
…ng on PATH

Plain Windows PowerShell (no Git Bash/WSL, no ripgrep installed) has neither
grep nor rg on PATH, so `rtk grep`/`rtk rg` hard-failed instead of running.
engine_capture now checks tool_exists(engine.bin()) and, only when the
requested binary is absent, walks paths with the already-vendored `ignore`
crate and matches lines with the already-vendored `regex` crate. This never
substitutes one engine for the other — grep still only falls back when grep
itself is missing, same for rg. Output is emitted in the exact
file\0line:content shape parse_match_line already expects, so grouping/
truncation/display code is untouched. Context lines (-A/-B/-C) are not
supported in fallback mode; only -i/--ignore-case is honored from extra_args.

Forward-ports unreleased fork-local work (originally against the pre-refactor
grep_cmd.rs) onto develop's search.rs/Engine design. Not tied to a GitHub
issue; staged for maintainer discussion before submission.
@CLAassistant

CLAassistant commented Jul 29, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

…shell-native-compat

Port the native fallbacks onto develop: compact_ls now returns
(entries, parsed, truncated, filtered) and plan() no longer exposes
the positional paths, so run_native takes them from child_args and
appends the standard hidden-entries hint.
… nested ifs

The develop guard no_tracked_label_is_a_space_join rejects labels built
with join(" "), and clippy on edition 2024 asks for let-chains.
On stock Windows the native listing stands in for Get-ChildItem (ls and
dir are its aliases), which hides Hidden/System entries and lists
dotfiles. Hiding by a leading dot dropped .gitignore and showed files
marked hidden. Also print no "(empty)" placeholder after the errors when
every target fails.
…Select-String

-TotalCount (and its -Head/-First aliases) is an exact head window, but
--max-lines is a smart truncation: Get-Content -TotalCount 2 printed one
line and a "[3 more lines]" marker. Map it to --head-lines.

Select-String always prints path:line:text; pass -n so the line number
survives, including for a single file.
gci -Recurse rewrites to rtk tree, so the native tree must hide by the
Windows Hidden/System attributes too, not by a leading dot.
Parsing the full clap command graph in an unoptimized Windows build
needs more than the 2 MiB a spawned test thread gets by default:
develop sits just under it, and adding the powershell/pwsh subcommands
pushed the Cli parse tests over, aborting the whole unit-test binary
with STATUS_STACK_OVERFLOW. Measured on Windows MSVC: test_bun_build
overflows at 2 MiB and passes at 4 MiB. Match the /STACK size build.rs
sets for the main thread.
@pszymkowiak
pszymkowiak marked this pull request as ready for review October 8, 2026 04:04
…-spawning rtk

rtk powershell -Command <cmdlet> re-executed current_exe() with the
rewritten argv, which semgrep's dynamic-command-execution rule blocks
and which paid a second process start. Split the command dispatch out
of run_cli and run the rewritten argv through it directly; anything
that does not rewrite still goes to the real shell unchanged.

@KuSh KuSh left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 1 — CHANGES REQUESTED

Claim: rtk ls|tree|wc|grep|rg keep working on a host without the Unix tools through native Rust fallbacks, rtk powershell|pwsh plus bare-cmdlet rewrites route PowerShell file cmdlets through rtk, and the Windows build links with non-MSVC linkers.
Scope: accept if narrowed to "native ls/tree/wc/grep/rg fallbacks that fail loudly on anything they do not implement". Two parts leave the PR. The build.rs linker hunk is the one #3613 already carries (it also keys off the target, and #3613 is the open survivor of #3239), so it lands there. The PowerShell wrappers, cmdlet rewrites and their rules become a follow-up (#1248): they need a shell-aware rewrite, not a mapping table, and the evidence below is for that follow-up. Related open work: #2875 (its commits are carried here; it closes once this lands), #2355 and #2728 (same area); #4401 (flag grammars as data, which is what the fallbacks need to reject what they do not implement) and #4103 (child argument encoding) should land first.
Ran: merge of 3e97f69 onto develop 311ebcb (d5e72aa7c), debug binaries built from separate target dirs (md5 differ). Real tools: GNU ls/wc/tree/grep, ripgrep, PowerShell 7.4.2 (mcr.microsoft.com/powershell). Native paths run with an empty PATH and diffed against the real-tool path: ls 27 shapes, wc 25, tree 26, grep/rg 37 plus 11 confirmation rows; rewrite corpus of 40 commands on both binaries and through rtk hook claude; 14 cmdlet rows against real pwsh. Invariants: non-default targets (symlink, dangling link, dir with space, missing path), over-cap listing with the recall hint, stdin pipe for wc, under the hook, every decision of the new rewrite arm. Savings: ls 5987→775 B, tree 240→85 B against the 20% floor. Merged gate: fmt, clippy, cargo test --all (4091 unit tests) all exit 0; wincheck (windows-gnu) clean; CI 11/11 green on the PR head.
Pins: 143 mutations, 44 killed, 98 survived (11 of them equivalent or unreachable; 87 observable from the CLI).
Matrix: executed this round; pin check: executed this round
CI @3e97f6927: all green

Blocks merge (3, frozen)

  1. src/cmds/system/{ls,wc_cmd,tree,search}.rs — native fallback answers where develop failed loudly (4 sites; all must change). Each fallback drops what it does not implement and prints a plausible result with exit 0 (or 1):
    • ls (ls.rs:309): entries ordered by the synthesized line, so by size text (a_one 1B, c_hundred, b_twenty, d_three); with the display cap the 0 B files show and Cargo.toml, README.md, big.txt fall behind it. -d, -t, -S, -r, --ignore= are ignored; -I zdir treats zdir as a path (exit 2); ls -- -dash.txt lists the cwd (exit 0).
    • wc (wc_cmd.rs:163): wc -L f prints 2L 3W 19B (real: 11), --bytes prints all three counts, --max-line-length is wrong, --total= ignored, wc -l -- -dash.txt reads stdin and prints 0.
    • tree (tree.rs:71): tree src docs prints only src; -I deep is consumed and not applied; -P '*.txt' src makes *.txt the root; tree nosuch exits 0 (real: exit 2, [error opening dir]); --level=1 and -L 0 are accepted.
    • search (search.rs:429, :603): -w, -F, -S, -A/-B/-C ignored (grep -w TOD matches TODO; grep -F . matches every line); BRE a\|b reaches Rust regex and matches nothing; non-UTF-8 and UTF-16 files are skipped silently; a missing path exits 1 with no message, or 0 beside a good path; .gitignore and hidden files are honoured under the grep engine, so grep -rn ignored . finds nothing where grep finds two lines.
    • dispatch (tree.rs:21): cfg!(windows) || !tool_exists("tree") sends every Windows user to the reduced walker, including those with a real tree on PATH (Git for Windows, scoop), who lose -P, -I, -s and the rest. On Linux and macOS the real binary is used whenever present. The native path should engage only when the tree on PATH is not the Unix one (or none exists), not on the platform alone.
      Evidence: transcripts for each row, native vs real tool. Test: a differential test per fallback over the forms it claims, and for every flag or operand it does not implement an assertion that stdout stays empty and exit is non-zero with a message. The corpus must show 0 differing rows on the supported forms.
  2. src/discover/registry.rs:1707, powershell_cmd.rs:115 — PowerShell routing in the shared rewrite engine (cut by the narrowing; otherwise all must change). Regressions against develop:
    • type git → rtk read git (also echo hi && type git, FOO=1 type git), through rtk rewrite and rtk hook claude: bash's type exits 0 with the path, the rewritten form exits 1, or prints a file that happens to be named git. dir/dir src → rtk ls likewise. Copilot CLI hands its powershell tool to the same engine as Bash, so the engine cannot tell the shells apart.
    • /x/explicit/pwsh -File b.ps1 → rtk pwsh -File b.ps1 runs the pwsh found on PATH, not the binary named.
      Cmdlet mapping unfaithful against pwsh 7.4.2: Select-String -Pattern 'one|bar' matches lines 2 and 3, the rewrite is rtk grep -n -i 'one|bar' (BRE): exit 1, no output; -Pattern "-v" becomes the flag -v: exit 1, no output where PowerShell prints a -v b; single file prints 1:text without the path; no-match exits 1 where PowerShell exits 0; Get-ChildItem -Depth N → tree -L N is one level short (-Depth 1 . lists src/main.rs, tree -L 1 does not; the test pins the wrong mapping).
  3. Unpinned behaviour (both areas). Native fallbacks: 45 of 66 mutations survive, including the four dispatch conditions (!tool_exists("ls"|"wc"|"tree"|engine.bin()) inverted: all tests pass), every exit-code mapping (ls.rs:410 2→1, :420 2→0, wc_cmd.rs:195 1→0), the walker options (git_ignore/git_global/git_exclude off), the default path, tree's multi-root and --all, wc's total line. PowerShell: 53 of 77 survive, including run_powershell never rewriting (main.rs:2166), the verbose and is_operational_command arms, the exit-code propagation of run() (→ 0), the redirect suffix, the Normal-context guard, the case-insensitivity and .exe of both rules, and the gc/type/dir/ls/gci aliases. The test the fix needs: for each, the input and the output that decides it (report: pin-A and pin-B tables); each mutation must turn it red. Held until the blockers above are settled, because the fix to 1 and 2 changes what the tests must assert.

Optional — will not hold merge; threads resolved by me

  • src/cmds/system/README.md names native_grep (the code has fallback_capture), says -TotalCount→--max-lines (code: --head-lines) and omits the always-added -n.
  • search.rs:421 cites #Windows-no-binary, which is not an issue.
  • The fallback covers the grouped search form only: -l, -c, -o, --files, --help and the piped-stdin branch still fail with Failed to execute, which is loud and safe, but the PR text reads wider.
  • Native ls -l prints 644/755 on Unix too, where PermissionsExt has the real mode.

Filed as follow-ups

  • Shell-aware PowerShell routing (wrappers, cmdlet rewrites, rules with measured savings_pct instead of the inherited 60, categorize_command and RUST_HANDLED_COMMANDS entries, and ChildArgExt::child_args instead of .args(args) at powershell_cmd.rs:84 for the passthrough spawn): comment on #1248 with the rows above — outside the narrowed claim. Not filed yet.

Checked and correct — no need to re-verify

  • Merge with develop is clean; fmt, clippy, tests and wincheck pass on the merge.
  • tree src, -a, -L 2 and tree sizes equal the real tool after glyph normalisation; grep/rg -v, -i, -e, --max-count equal; wc default, -l -w -c -m -lw, several files and piped stdin equal; ls of a file, an empty dir and a dotfile equal.
  • Get-Content -TotalCount N/-Tail N, Select-String -NotMatch equal pwsh on the same files; unsupported cmdlet options, pipes, $ and ; all pass through unchanged.
  • The recall hint works on the native ls listing.
  • build.rs target-env check type-checks for windows-gnu (wincheck); the hunk itself is #3613's.

Your questions

None.

Next

Narrow to the native fallbacks and the linker plumbing; make each fallback refuse what it does not implement (exit 2, message, empty stdout) and error on a missing target; move the PowerShell part to its own PR after #4401. Pin each fallback decision with a test that fails when mutated.
Threads: 0 open, 0 resolved this round.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants