Repository navigation
fix(cli): 2D-parallelism error tells the user to pass --tensor-parallel-size, which does not exist #1112
Description
Summary
The 2D (pipeline × tensor) parallelism validation in src/commands/generate.rs:367-371 fails with a message telling the user to pass --tensor-parallel-size > 1. That flag does not exist; the real flag is --tp-size. A user who follows the instruction gets error: unexpected argument '--tensor-parallel-size' found. A comment three lines above points at two docs/en/distributed/*.md files that also do not exist.
Background
src/commands/generate.rs:367-371:
ensure!(
pp.pp_size >= 2 || pp.pp_layers.is_some(),
"2D parallelism requires --pp-size >= 2 (or an explicit --pp-layers spec) \
alongside --tensor-parallel-size > 1"
);grep -rn "tensor-parallel-size" src/ returns exactly one hit: this message string. It is never a clap flag. The tensor-parallel rank count is --tp-size, defined at src/main.rs:728, src/main.rs:1378, and src/bin/mlx_server.rs:763. The message is otherwise accurate (--pp-size and --pp-layers are both real), which makes the one wrong name more misleading than a vague message, because the other two names check out.
The comment at src/commands/generate.rs:362-364 points at docs/en/distributed/pipeline-parallelism.md and docs/en/distributed/tensor-parallelism.md. Neither exists; docs/en/ has never been part of this repository, and these are the only two docs/en/ references in src/. The real operator document is docs/distributed.md.
Proposed Solution
- Change
--tensor-parallel-sizeto--tp-sizein theensure!message. - Repoint the two comment references at
docs/distributed.md, or drop them if that document does not cover 2D composition.
Implementation Notes
- Message and comment text only; the validation logic is unchanged.
- Worth a quick sweep of the neighbouring
ensure!messages in the same function while there:--pp-micro-batch-size(src/commands/generate.rs:356),--pp-size, and--pp-layersall resolve, but the file is the kind of place where one wrong name suggests checking the rest.
Acceptance Criteria
- Every flag named in the message is one the binary accepts.
- No
docs/en/reference remains insrc/. -
grep -rn -- "--tensor-parallel-size" src/returns nothing. - Message text only; the validation logic is unchanged.
Original Suggestion
Title: fix(cli): a 2D-parallelism error tells the user to pass --tensor-parallel-size, which does not exist
Problem / Background
src/commands/generate.rs:367-371 validates a 2D (pipeline × tensor) parallel topology and
fails with:
ensure!(
pp.pp_size >= 2 || pp.pp_layers.is_some(),
"2D parallelism requires --pp-size >= 2 (or an explicit --pp-layers spec) \
alongside --tensor-parallel-size > 1"
);There is no --tensor-parallel-size flag. The tensor-parallel rank count is --tp-size,
defined at src/main.rs:728, src/main.rs:1378, and src/bin/mlx_server.rs:763, with no
--tensor-parallel-size alias anywhere in the tree.
A user who hits this error and follows its instruction gets
error: unexpected argument '--tensor-parallel-size' found. The message is otherwise
accurate and actionable — --pp-size and --pp-layers are both real — which makes the one
wrong name more misleading than a vague message would be, because the other two names in the
same sentence check out.
Second defect in the same block
The comment three lines above, src/commands/generate.rs:362-364, points at the operator
guide:
// 2D (PP x TP) composition is now supported. See
// `docs/en/distributed/pipeline-parallelism.md` and
// `docs/en/distributed/tensor-parallelism.md` for the operator guide.Neither file exists. docs/en/ has never been part of this repository
(git log --all -- docs/en is empty). These are the only two docs/en/ references in
src/, so fixing them here clears the pattern from the code entirely. The real
operator-facing document is docs/distributed.md.
Proposed solution
- Change
--tensor-parallel-sizeto--tp-sizein theensure!message. - Repoint the two comment references at
docs/distributed.md, or drop them if that
document does not cover 2D composition.
Acceptance criteria
- Every flag named in the message is one the binary accepts.
- No
docs/en/reference remains insrc/. -
grep -rn -- "--tensor-parallel-size" src/returns nothing. - Message text only; the validation logic is unchanged.
Technical considerations
Worth a quick sweep of the neighbouring ensure! messages in the same function while you
are there — --pp-micro-batch-size (src/commands/generate.rs:356), --pp-size and
--pp-layers all resolve, but the file is the kind of place where one wrong name suggests
checking the rest.
Activity
[-]fix(cli): a 2D-parallelism error tells the user to pass `--tensor-parallel-size`, which does not exist[/-][+]fix(cli): 2D-parallelism error tells the user to pass `--tensor-parallel-size`, which does not exist[/+]fix: address review findings on the 2D-parallelism message fix
fix(cli): name the real --tp-size flag in the 2D-parallelism error (l…
test(cli): assert every flag named in src/commands exists in some --h…
test: assert every flag named in `src/commands` exists in some `--hel…
test: assert every flag named in `src/commands` exists in some `--hel…
perf(cuda): convert bf16 weights to f16 at load on pre-Ampere GPUs (#…
perf(cuda/quant): add a Volta tensor-core arm to qmm_naive (#1678)
fix(cli): name the real --tp-size flag in the 2D-parallelism error (l…