Skip to content

fix(cli): 2D-parallelism error tells the user to pass --tensor-parallel-size, which does not exist #1112

Closed

Description

@rapsealk

Summary

The 2D (pipeline × tensor) parallelism validation in src/commands/generate.rs:367-371 fails with a message telling the user to pass --tensor-parallel-size > 1. That flag does not exist; the real flag is --tp-size. A user who follows the instruction gets error: unexpected argument '--tensor-parallel-size' found. A comment three lines above points at two docs/en/distributed/*.md files that also do not exist.

Background

src/commands/generate.rs:367-371:

ensure!(
    pp.pp_size >= 2 || pp.pp_layers.is_some(),
    "2D parallelism requires --pp-size >= 2 (or an explicit --pp-layers spec) \
     alongside --tensor-parallel-size > 1"
);

grep -rn "tensor-parallel-size" src/ returns exactly one hit: this message string. It is never a clap flag. The tensor-parallel rank count is --tp-size, defined at src/main.rs:728, src/main.rs:1378, and src/bin/mlx_server.rs:763. The message is otherwise accurate (--pp-size and --pp-layers are both real), which makes the one wrong name more misleading than a vague message, because the other two names check out.

The comment at src/commands/generate.rs:362-364 points at docs/en/distributed/pipeline-parallelism.md and docs/en/distributed/tensor-parallelism.md. Neither exists; docs/en/ has never been part of this repository, and these are the only two docs/en/ references in src/. The real operator document is docs/distributed.md.

Proposed Solution

  1. Change --tensor-parallel-size to --tp-size in the ensure! message.
  2. Repoint the two comment references at docs/distributed.md, or drop them if that document does not cover 2D composition.

Implementation Notes

  • Message and comment text only; the validation logic is unchanged.
  • Worth a quick sweep of the neighbouring ensure! messages in the same function while there: --pp-micro-batch-size (src/commands/generate.rs:356), --pp-size, and --pp-layers all resolve, but the file is the kind of place where one wrong name suggests checking the rest.

Acceptance Criteria

  • Every flag named in the message is one the binary accepts.
  • No docs/en/ reference remains in src/.
  • grep -rn -- "--tensor-parallel-size" src/ returns nothing.
  • Message text only; the validation logic is unchanged.

Original Suggestion

Title: fix(cli): a 2D-parallelism error tells the user to pass --tensor-parallel-size, which does not exist

Problem / Background

src/commands/generate.rs:367-371 validates a 2D (pipeline × tensor) parallel topology and
fails with:

ensure!(
    pp.pp_size >= 2 || pp.pp_layers.is_some(),
    "2D parallelism requires --pp-size >= 2 (or an explicit --pp-layers spec) \
     alongside --tensor-parallel-size > 1"
);

There is no --tensor-parallel-size flag. The tensor-parallel rank count is --tp-size,
defined at src/main.rs:728, src/main.rs:1378, and src/bin/mlx_server.rs:763, with no
--tensor-parallel-size alias anywhere in the tree.

A user who hits this error and follows its instruction gets
error: unexpected argument '--tensor-parallel-size' found. The message is otherwise
accurate and actionable — --pp-size and --pp-layers are both real — which makes the one
wrong name more misleading than a vague message would be, because the other two names in the
same sentence check out.

Second defect in the same block

The comment three lines above, src/commands/generate.rs:362-364, points at the operator
guide:

// 2D (PP x TP) composition is now supported. See
// `docs/en/distributed/pipeline-parallelism.md` and
// `docs/en/distributed/tensor-parallelism.md` for the operator guide.

Neither file exists. docs/en/ has never been part of this repository
(git log --all -- docs/en is empty). These are the only two docs/en/ references in
src/, so fixing them here clears the pattern from the code entirely. The real
operator-facing document is docs/distributed.md.

Proposed solution

  1. Change --tensor-parallel-size to --tp-size in the ensure! message.
  2. Repoint the two comment references at docs/distributed.md, or drop them if that
    document does not cover 2D composition.

Acceptance criteria

  • Every flag named in the message is one the binary accepts.
  • No docs/en/ reference remains in src/.
  • grep -rn -- "--tensor-parallel-size" src/ returns nothing.
  • Message text only; the validation logic is unchanged.

Technical considerations

Worth a quick sweep of the neighbouring ensure! messages in the same function while you
are there — --pp-micro-batch-size (src/commands/generate.rs:356), --pp-size and
--pp-layers all resolve, but the file is the kind of place where one wrong name suggests
checking the rest.

Activity

  1. changed the title [-]fix(cli): a 2D-parallelism error tells the user to pass `--tensor-parallel-size`, which does not exist[/-] [+]fix(cli): 2D-parallelism error tells the user to pass `--tensor-parallel-size`, which does not exist[/+] on Aug 12, 2026
  2. added
    type:bugBug fixes, error corrections, or issue resolutions
    and removed on Aug 12, 2026
  3. added a commit that references this issue on Aug 20, 2026
    33ffc7f
  4. added a commit that references this issue on Oct 27, 2026
    5f9697c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority:lowLow prioritystatus:doneCompletedtype:bugBug fixes, error corrections, or issue resolutions

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions