🐛 Make second column of Rich help output reflect the type consistently, even when using metavar

[svlandeg]

Ok this is a bit of a tricky PR, so I'll dump train of thought here in detail. [disclaimer: 0% vibe-coded, comment nor code]

Fixes #1156 .

Current master behaviour

First let's look at test_tutorial/test_arguments/test_help/test_tutorial006.py, corresponding to the section in the docs about metavar, python source here.

Without rich markdown on master, this looks like:

$ python docs_src/arguments/help/tutorial006.py --help
Usage: tutorial006.py [OPTIONS] ✨username✨

Arguments:
  ✨username✨  [default: World]
...

With rich on master:

$ python docs_src/arguments/help/tutorial006.py --help

 Usage: tutorial006.py [OPTIONS] ✨username✨

┌─ Arguments ──────────────────────────────────────────────────────────────┐
│   name      ✨username✨  [default: World]                               │
└──────────────────────────────────────────────────────────────────────────┘
...

What we see here, is that the original "name" still shows up in the very first column of the output formatting, with the metavar name only in the second column, which is currently on master construed as the "metavar" column.

When we look at another non-metavar example, e.g. tutorial005 we see something similarly weird on master:

$ python docs_src/arguments/help/tutorial005.py --help

 Usage: tutorial005.py [OPTIONS] [NAME]

┌─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│   name      [NAME]  Who to greet [default: (Deadpoolio the amazing's name)]                                             │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

...

i.e. the var's proper formatting [NAME] only shows up in the second metavar column.

Let's look at a much more extensive example, taken from @Noxitu in #438 and in adapted form added to this PR as test_rich_help_metavar in test_rich_utils.py. It gives as output:

$ python issue_438.py --help

 Usage: issue_438.py [OPTIONS] ARG1 ARG3 [ARG4] meta7 ARG8 arg9

┌─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ *    arg1      INTEGER  [required]                                                                                          │
│ *    arg3      INTEGER  [required]                                                                                          │
│      arg4      [ARG4]   [default: 42]                                                                                       │
│      arg7      meta7    [default: 42]                                                                                       │
│ *    arg8      INTEGER  [required]                                                                                          │
│ *    arg9      arg9     [required]                                                                                          │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
┌─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│    --arg2                      INTEGER  [default: 42]                                                                       │
│ *  --arg5                      INTEGER  [required]                                                                          │
│    --arg6                      INTEGER  [default: 42]                                                                       │
│    --install-completion                 Install completion for the current shell.                                           │
│    --show-completion                    Show completion for the current shell, to copy it or customize the installation.    │
│    --help                               Show this message and exit.                                                         │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

What we see here is that types and metavar information is freely mixed in that "metavar" column when Rich is used to print the help.

Compare this to the non-Rich output on master:

$ python issue_438.py --help
Usage: issue_438.py [OPTIONS] ARG1 ARG3 [ARG4] meta7 ARG8 arg9

Arguments:
  ARG1    [required]
  ARG3    [required]
  [ARG4]  [default: 42]
  meta7   [default: 42]
  ARG8    [required]
  arg9    [required]

Options:
  --arg2 INTEGER        [default: 42]
  --arg5 INTEGER        [required]
  --arg6 INTEGER        [default: 42]
  --install-completion  Install completion for the current shell.
  --show-completion     Show completion for the current shell, to copy it or customize the installation.
  --help                Show this message and exit.

This PR

So, what this PR proposes to do is:

• Consistently use the first column of the Rich help format to show the name or the metavar name if set, as the non-Rich formatting does
• Consistently use the second column of the Rich help format to display the type

This required some edits to the tests:

• I've added two tests test_tutorial006_rich.py and test_tutorial006_an_rich.py that double check the same behaviour with or without Rich formatting - both tests fail on master.
• I've added the more extensive test test_rich_help_metavar in test_rich_utils.py.

Results with this PR:

$ python docs_src/arguments/help/tutorial005_py310.py --help

 Usage: tutorial005_py310.py [OPTIONS] [NAME]

┌─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│   [name]      TEXT  Who to greet [default: (Deadpoolio the amazing's name)]                                                 │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
....

This now shows [name] properly, and TEXT as type in the second column.

$ python docs_src/arguments/help/tutorial006_py310.py --help

 Usage: tutorial006_py310.py [OPTIONS] [✨username✨]

┌─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│   [✨username✨]      TEXT  [default: World]                                                                                      │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
┌─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ --help          Show this message and exit.                                                                                 │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

No more "NAME" visible in the output.

And finally for the test_rich_help_metavar test:

 Usage: issue_438.py [OPTIONS] ARG1 ARG3 [ARG4] [meta7] ARG8 arg9

┌─ Arguments ─────────────────────────────────────────────────────────────────┐
│ *    arg1         INTEGER  [required]                                       │
│ *    arg3         INTEGER  [required]                                       │
│      [arg4]       INTEGER  [default: 42]                                    │
│      [meta7]      INTEGER  [default: 42]                                    │
│ *    ARG8         INTEGER  [required]                                       │
│ *    arg9         INTEGER  [required]                                       │
└─────────────────────────────────────────────────────────────────────────────┘
┌─ Options ───────────────────────────────────────────────────────────────────┐
│    --arg2                      INTEGER  [default: 42]                       │
│ *  --arg5                      INTEGER  [required]                          │
│    --arg6                      INTEGER  [default: 42]                       │
│    --install-completion                 Install completion for the current  │
│                                         shell.                              │
│    --show-completion                    Show completion for the current     │
│                                         shell, to copy it or customize the  │
│                                         installation.                       │
│    --help                               Show this message and exit.         │
└─────────────────────────────────────────────────────────────────────────────┘

Which feels a lot more consistent and nice.

Details of the actual fix

Which brings us to the actual fix in rich_utils.py.

The old L375 had code specifically designed for a "metavar column", but as we see in the examples, this column was filled with different types of data. I've found out that in fact, Argument's store an alternative name in their metavar data, while Option objects may store alternative represenations of their types. This function is not directly used in Typer, but becomes apparent for Enum cases where we want to display something like [simple|conv|lstm] instead of Choice. For this reason, the new code in rich_utils.py puts the metavar_str data in a different column, depending on the object instance check. As before, we don't explicitely display BOOLEAN values.

Breaking behaviour

While I consider this to be a bug fix, it's also breaking behaviour and may impact users.

For now, we decided to keep the capitalization of argument names in Rich help formatting the same as before (i.e. lowercased) even though this is inconsistent with non-Rich formatting (where they are upper-cased, which is also what the docs state). This should hopefully minimize impact on users.

[github-actions]

📝 Docs preview

Last commit a6326b9 at: https://0e78222a.typertiangolo.pages.dev

Modified Pages

• https://0e78222a.typertiangolo.pages.dev/tutorial/arguments/help/ - (before)

[svlandeg]

Note that this part is crucial to ensure the current Enum tests won't fail. This is the part that will replace Choice with something like [simple|conv|lstm]

[svlandeg]

I'm not too sure yet about these secondary_opts, probably requires another test (instead of having pragma cover).

[svlandeg]

I think we'll revisit this soon in the Click vendoring / Pydantic typing PRs we have planned, so I personally wouldn't worry about this too much 😇

[svlandeg]

TODO: after internal conversation, we've decided to keep the capitalization of argument names in Rich help formatting the same as before (i.e. lowercased), and refactor these type of help displays more properly in follow-up work.

[svlandeg]

FYI - this also succeeds on master and shows that the capitalization of argument names has remained the same.

[svlandeg]

I've simplified the PR to not touch the argument name capitalization, and to not change the ✨username✨ example from the tutorial.

We'll revisit help display in a follow-up PR, this is considered more of a bug fix and we want to release it separately as such.

[tiangolo]

Nice, thank you! 🚀