Skip to content

fix(glab): tokenize glab args instead of walking them by hand - #4005

Merged
KuSh merged 6 commits into
rtk-ai:developfrom
KuSh:feat/glab-arg-tokenizer
Oct 5, 2026
Merged

KuSh merged 6 commits into
rtk-ai:developfrom
KuSh:feat/glab-arg-tokenizer

Conversation

@KuSh

@KuSh KuSh commented Sep 12, 2026 •

Copy link
Copy Markdown
Collaborator

The bug

glab_cmd.rs's extract_identifier_and_extra_args picked the MR/issue identifier with a hand-rolled walker: a hardcoded eight-entry list of value-taking flags (-R, --repo, -g, --group, -F, --output, -m, --message), a skip_next bool, and arg.starts_with('-'). Any value-taking flag missing from that list had its value hoisted out and re-emitted as the identifier. The list also mixed flags from unrelated subcommands — -m/--message belongs to mr merge/mr note, not to mr view, while mr view's own -p, -P and --jq were absent.

run() additionally had no -- awareness, and rtk's own -R/-g were appended after the user's arguments.

Reproducers

Verified against a glab stub that records its argv (tests/glab_argv_test.rs), flags transcribed from glab 1.36.0 and 1.117.0 --help, and every expectation checked against real glab 1.117.0 (registry.gitlab.com/gitlab-org/cli:latest, LC_ALL=C):

typed glab received (develop) glab receives (this PR)
rtk glab mr view --page 2 mr view 2 -F json --page mr view -F json --page 2
rtk glab issue view -P 50 issue view 50 -F json -P issue view -F json -P 50
rtk glab mr view --jq .iid mr view .iid -F json --jq mr view --jq .iid
rtk glab mr view --page 2 42 mr view 2 -F json --page 42 mr view 42 -F json --page 2
rtk glab -R o/r mr diff -- 42 mr diff -- 42 -R o/r mr diff -R o/r -- 42
rtk glab mr view -- --web 42 mr view 42 -- --web mr view -- --web 42
rtk glab mr view -- --web mr view -- --web mr view -- --web

What changed

  • The walker and its flags_with_value array are gone. split_identifier runs arg_tokenizer::tokenize_grammar and takes the first free positional (Token::is_free_positional).
  • One takes_value table per glab subcommand, each transcribed from that subcommand's own --help and taken as the union over 1.36.0 and 1.117.0. mr merge, mr approve, mr note and mr update do not share one: -s is the boolean --squash on merge but the value-taking --sha on approve, -d is --remove-source-branch on merge but --description on update, and -m is --message on merge but --milestone on update. mr view and issue view do share a table — their value-taking flag sets are identical on both versions.
  • Value-taking flags use .claiming_dash_dash(): measured on 1.117.0, glab mr view --page -- 5 reports Invalid argument "--" for "-p, --page" flag, so pflag reads -- as the flag's value, not as the boundary.
  • run() restores the -- clap's trailing_var_arg consumes, over the reassembled [subcommand] + args region rather than args alone; over args alone it lands one token off and duplicates the subcommand. trailing_var_arg only ever lets clap strip one --, the one at the head of that region — before the subcommand, or right after it, since -R/-g are parsed before the subcommand positional either way. That one is rtk's own option terminator and glab never saw it: glab -- mr view 42 prints root help on 1.117.0, glab mr -- view 42 prints the mr help instead of dispatching to view, and glab api -- projects/1 --paginate is Accepts 1 arg(s), received 2. So that one -- is dropped and every later one reaches glab verbatim.
  • glab keeps reading positionals past -- (glab mr view -- 42 views MR 42), so the identifier search reaches past the boundary — but only where unescaping is a no-op: one token, still read as a positional without the --. Re-emitting the identifier in front of the boundary unescapes it along with whatever trailed it, and rtk glab mr view -- --web or -- --web 42 would then run glab with --web live, opening a browser the user had asked glab to treat as a positional. Anything else glab rejects on arity whatever rtk sends (Accepts at most 1 arg(s), received 2), so it goes to glab untouched.
  • rtk's -R/-g now go in at arg_tokenizer::injection_point instead of being appended. Position is unchanged when there is no --.
  • has_output_flag now counts --jq. With the value correctly attached, glab returns the user's own projection, and reformatting it as an MR printed a summary of fields the projection does not have (? MR !0: ???).

Tests

tests/glab_argv_test.rs asserts on the child argv, and on rtk's stdout for the --jq case; each of the 13 tests was confirmed failing before the corresponding change and passing after. The run is isolated with its own HOME and RTK_DB_PATH, which it otherwise read and wrote for real. Sixteen unit tests in glab_cmd.rs cover the per-subcommand tables, including the -s merge/approve split.

Not in scope

has_output_flag and should_passthrough_view still scan with a == "--output", so attached spellings (--jq=.iid) and short clusters miss. That is the separate args.iter().any(...) bug class, not the identifier walker; left for a follow-up.

@rtk-wshm-sync-bot rtk-wshm-sync-bot Bot added bug Something isn't working rust glab labels Sep 12, 2026
@rtk-wshm-sync-bot

Copy link
Copy Markdown

wshm · Automated triage by AI

📊 Automated PR Analysis

🐛 Type bug-fix
🟡 Risk medium

Summary

Replaces the hand-rolled arg walker in glab_cmd.rs's identifier extraction with a proper tokenizer-based approach, using per-subcommand value-taking flag tables and correct -- boundary handling. This fixes bugs where value-taking flags missing from the old hardcoded list would have their values mistakenly hoisted out as the MR/issue identifier.

Review Checklist

  • Tests present
  • Breaking change
  • Docs updated

Analyzed automatically by wshm · This is an automated analysis, not a human review.

KuSh and others added 6 commits September 18, 2026 00:37
`extract_identifier_and_extra_args` picked the MR/issue identifier with a
hand-rolled walker: a hardcoded eight-entry list of value-taking flags, a
`skip_next` bool, and `arg.starts_with('-')`. Anything the list missed had its
value read as the identifier, and the list itself was a mix of flags from
different subcommands.

Verified against glab 1.36.0 and 1.117.0:

    rtk glab mr view --page 2      ->  glab mr view 2 -F json --page
    rtk glab issue view -P 50      ->  glab issue view 50 -F json -P
    rtk glab mr view --jq .iid     ->  glab mr view .iid -F json --jq

Replace it with `arg_tokenizer::tokenize_grammar` plus one `takes_value`
predicate per glab subcommand, transcribed from each subcommand's own `--help`,
and take the first free positional. `mr merge`, `mr approve`, `mr note` and
`mr update` each get their own table: `-s` is the boolean `--squash` on merge
but the value-taking `--sha` on approve, and `-d` is `--remove-source-branch`
on merge but `--description` on update.

`run()` also restores the `--` that clap's `trailing_var_arg` consumes, over
the reassembled `[subcommand] + args` region rather than `args` alone, and
moves rtk's own `-R`/`-g` from the end of the args to the tokenizer's
injection point — glab reads everything past `--` as a positional, so an
appended `-R` arrived as two extra arguments instead of the repo flag.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two fallouts of classifying `--jq` and restoring `--`.

`--jq` now reaches glab with its value intact, so glab returns the user's
projection — and `run_glab_json` reformatted it as an MR summary of fields the
projection does not have (`? MR !0: ???`). Previously the value was hoisted out
as the MR number and glab failed loudly, so this turned a visible error into a
silent wrong answer. `has_output_flag` now counts `--jq`.

A `--` at the head of the region ended rtk's own `-R`/`-g` parsing, not glab's.
Forwarding it makes glab stop looking for a subcommand: `glab -- mr view 42`
prints root help on 1.117.0. Drop it instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`trailing_var_arg` only ever lets clap strip one `--`, the one at the head of the
region it hands to glab — before the subcommand, or right after it, since `-R`/`-g`
are parsed before the subcommand positional either way. Only checking index 0 let
the index-1 case through: `glab mr -- view 42` prints the `mr` help instead of
viewing the MR, and `glab api -- projects/1 --paginate` is `Accepts 1 arg(s),
received 2` on glab 1.117. Every later `--` clap passes through untouched and still
reaches glab verbatim.

`split_identifier` reads positionals past `--` because glab does too
(`glab mr view -- 42` views MR 42), but the callers re-emit the identifier ahead of
the boundary. When the escaped region held more than the identifier, that promoted a
flag the user had escaped back into flag position: `rtk glab mr view -- --web 42`
ran glab with `--web` live, opening a browser the user had asked glab to treat as a
positional. glab takes at most one positional for these subcommands, so an escaped
region holding more than one token is something glab rejects whatever rtk sends.

Also isolate HOME and RTK_DB_PATH in the argv test, which otherwise read the
developer's own config and wrote to their real tracking database.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Refusing to look past `--` only when two or more tokens trailed it still unescaped
the single-token case, which is the one the user is most likely to type:
`rtk glab mr view -- --web` hoisted `--web` into identifier position and ran glab
with the browser flag live, where develop passed the command through untouched.

What makes hoisting safe is not how much the boundary escapes but whether
unescaping it is a no-op: reach past `--` only for one token that is still read as a
positional without it, so `glab mr view -- 42` keeps gaining `-F json`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`escapes_a_bare_positional` landed between the header and the function it describes,
so rustdoc rendered the helper under `split_identifier`'s summary and left
`split_identifier` undocumented.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@KuSh
KuSh force-pushed the feat/glab-arg-tokenizer branch from 997f192 to fd84246 Compare September 17, 2026 22:38

@aeppling aeppling left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@KuSh
KuSh merged commit df39e33 into rtk-ai:develop Oct 5, 2026
11 checks passed
@KuSh
KuSh deleted the feat/glab-arg-tokenizer branch October 5, 2026 20:33
@rtk-release-bot rtk-release-bot Bot mentioned this pull request Oct 5, 2026
KuSh added a commit that referenced this pull request Oct 6, 2026
fix(tests): repair the tests broken by the #4001 and #4005 merges
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working glab rust

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants