Skip to content

feat(test): a test harness for an adopter's own suite #1181

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jdx merged 3 commits into from
Aug 21, 2026
Merged

feat(test): a test harness for an adopter's own suite #1181

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5f09403
feat(test): a test harness for an adopter's own suite
jdx Aug 21, 2026
aabd2ab
chore: check usage-test at the floor it declares
jdx Aug 21, 2026
557f655
fix(test): render a failure plain, not by the environment
jdx Aug 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ jobs:
matrix:
include:
- version: "1.91"
crates: usage-argv usage-derive usage-config usage-validation usage-rs
crates: usage-argv usage-derive usage-config usage-validation usage-rs usage-test
- version: "1.95"
crates: usage-lib usage-config-build clap_usage usage-cli
steps:
Expand Down
8 changes: 8 additions & 0 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ members = [
"config",
"config-build",
"derive",
"test",
"usage-rs",
"validation",
"clap_usage",
Expand Down Expand Up @@ -47,6 +48,7 @@ usage-config = { path = "./config", version = "5.1.0" }
usage-derive = { path = "./derive", version = "5.1.0" }
usage-lib = { path = "./lib", version = "5.1.0", features = ["clap", "validation"] }
usage-rs = { path = "./usage-rs", version = "5.1.0" }
usage-test = { path = "./test", version = "5.1.0" }
usage-validation = { path = "./validation", version = "5.1.0" }

[workspace.metadata.release]
Expand Down
21 changes: 21 additions & 0 deletions PLAN.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -333,6 +333,27 @@ tasks --usage"`, so task names are meant to come from running that. usage-argv d
already failed, and says what clap would have said, colour included, down to
suggesting what was probably meant. `parse()` renders and exits the way a
program does; `parse_from` hands the error back.
- [x] **A test harness for an adopter's own suite** — `usage-test`, reached as
`usage::test` behind a dev-dependency feature. A CLI's observable surface is
three things, and all three were reachable but unpleasant: `parse_from`
wants a `&[&OsStr]` a test cannot write as a literal and hands back a
compact code rather than the message a user reads; a help page needed a
route rebuilt by hand; and a completion answer needed `split` and
`candidates` assembled per test. Now `outcome` returns what `parse()` would
have _done_ — a struct, a page, a version, or a failure, each with the
stream and status it would have used — `help`/`help_tree` render one page or
the whole tree by the path a user types, and `candidates`/`completion`
answer a half-typed line. **Nothing in it formats a page**, which is the
rule that makes it worth having: every page comes from
`usage_argv::help::page` and every failure from `render_failure`, the same
functions the process calls, so a passing test is a statement about what
users see rather than about a second renderer that happens to agree today.
That function is the other half of the change: which page a help request
becomes — short, long, recursive, by route or by address, view or not — was
~150 lines emitted into every derive three times over, and is now decided
once in usage-argv and called from both places. A facade test asserts the two
halves agree: the page `help(spec, &["build"], Page::Long)` renders is
byte-for-byte the one `ex build --help` produces.

### What clap can say that we cannot

Expand Down
74 changes: 74 additions & 0 deletions argv/src/help.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2533,6 +2533,80 @@ pub fn render_all_view_at_styled(
Some(recursive_help(&viewed, path, chain, style, true))
}

/// Which page a help request asks for.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Page {
/// `-h`: the short page for one command.
Short,
/// `--help`: the long page for one command.
Long,
/// [`ArgAction::HelpAll`](crate::ArgAction::HelpAll): the long page for the command and
/// every visible descendant.
All,
}

/// The page a help request becomes, by the route the words took.
///
/// The parser reports the command a request arrived at, but a page is about the route that
/// reached it: one `Subcommands` type mounted under two parents is one address, and a page
/// found by searching for that address carries the first mount's path and globals. Falls back
/// to rendering by address where the route cannot be rebuilt, which only a command from
/// another CLI's tables can reach.
///
/// One function rather than a shape each caller reassembles. `parse()` renders a request this
/// way, and so does anything that wants to know what a command line would have printed
/// without running the program — a test harness, most of all, since a page it renders
/// differently from the process is a page that proves nothing.
pub fn page(
spec: &Spec<'_>,
root: &Command<'_>,
argv: &[&std::ffi::OsStr],
cmd: &Command<'_>,
page: Page,
style: Style,
) -> Option<String> {
match route_to(root, argv, cmd) {
Some(route) => match page {
Page::Short => render_at_styled(spec, &route, false, style),
Page::Long => render_at_styled(spec, &route, true, style),
Page::All => render_all_at_styled(spec, &route, style),
},
None => match page {
Page::Short => render_styled(spec, cmd, false, style),
Page::Long => render_styled(spec, cmd, true, style),
Page::All => render_all_styled(spec, cmd, style),
},
}
}

/// The page a help request becomes when a declared executable view is what the user invoked.
///
/// `argv` includes argv0 here, as [`route_to_view`] requires: the view's own name is what
/// selected it. The fallback is the canonical page, which is better than nothing where the
/// route cannot be rebuilt.
pub fn page_view(
spec: &Spec<'_>,
root: &Command<'_>,
argv: &[&std::ffi::OsStr],
cmd: &Command<'_>,
view: &ViewMeta<'_>,
page: Page,
style: Style,
) -> Option<String> {
match route_to_view(root, argv, cmd, view) {
Some(route) => match page {
Page::Short => render_view_at_styled(spec, &route, view, false, style),
Page::Long => render_view_at_styled(spec, &route, view, true, style),
Page::All => render_all_view_at_styled(spec, &route, view, style),
},
None => match page {
Page::Short => render_styled(spec, cmd, false, style),
Page::Long => render_styled(spec, cmd, true, style),
Page::All => render_all_styled(spec, cmd, style),
},
}
}

pub(crate) fn view_root_flags<'a>(
spec: &'a Spec<'a>,
promoted: &CommandMeta<'a>,
Expand Down
25 changes: 25 additions & 0 deletions argv/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -979,6 +979,21 @@ pub fn render_failure(spec: &spec::Spec<'_>, argv: &[&OsStr], error: &Error<'_,
diagnostic::render(spec, argv, error, diagnostic::Style::auto())
}

/// A parse failure, never coloured.
///
/// [`render_failure`] asks the environment whether to colour, which is right for a process and
/// wrong for anything that keeps the string: a test that asserts on a message, or a snapshot of
/// one, would pass or fail by whether stderr happened to be a terminal. The renderer is the
/// same; only the answer to that question is fixed.
#[cfg(feature = "diagnostics")]
pub fn render_failure_plain(
spec: &spec::Spec<'_>,
argv: &[&OsStr],
error: &Error<'_, '_>,
) -> String {
diagnostic::render(spec, argv, error, diagnostic::Style::PLAIN)
}

/// Render a failure through a spec-declared executable view.
///
/// `argv` is the original full argv, including the view executable as argv0.
Expand All @@ -1002,6 +1017,16 @@ pub fn render_failure(spec: &spec::Spec<'_>, argv: &[&OsStr], error: &Error<'_,
::std::format!("error: {error:?}\n")
}

/// A parse failure without the renderer, which is plain either way.
#[cfg(all(feature = "spec", not(feature = "diagnostics")))]
pub fn render_failure_plain(
spec: &spec::Spec<'_>,
argv: &[&OsStr],
error: &Error<'_, '_>,
) -> String {
render_failure(spec, argv, error)
}

/// Render a failure through a declared view without the optional diagnostics renderer.
#[cfg(all(feature = "spec", not(feature = "diagnostics")))]
pub fn render_failure_view(
Expand Down
159 changes: 39 additions & 120 deletions derive/src/codegen.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -477,6 +477,36 @@ pub fn emit(cli: &Cli) -> TokenStream {
let __usage_spec = Self::spec();
}
};

// One renderer for every help request. Which page a request becomes — and whether it is the
// route the words took or a fallback by address — is decided once, in usage-argv, rather
// than three times here in code nobody reads until it is wrong. It is also what lets a test
// harness render the page this program would have printed rather than one of its own.
let page_of = |style: TokenStream| {
quote! {
let __usage_page = match __usage_selected_view {
::std::option::Option::Some(view) => usage_argv::help::page_view(
__usage_spec,
Self::command(),
&__usage_all_refs,
cmd,
view,
__usage_want,
#style,
),
::std::option::Option::None => usage_argv::help::page(
__usage_spec,
Self::command(),
&__usage_argv,
cmd,
__usage_want,
#style,
),
};
}
};
let render_page = page_of(quote!(usage_argv::help::Style::auto()));
let render_page_stderr = page_of(quote!(usage_argv::help::Style::auto_stderr()));
let runtime_program = cli
.runtime_bin
.as_ref()
Expand Down Expand Up @@ -987,50 +1017,12 @@ pub fn emit(cli: &Cli) -> TokenStream {
}
::std::result::Result::Err(usage_argv::Error::Help { cmd, long }) => {
#effective_spec
// By the route the words took, not by the command's address: one
// `Subcommands` type mounted under two parents is one address, and a
// page found by searching for it carries the first mount's path and
// globals. Falls back where the route cannot be rebuilt.
let __usage_route = match __usage_selected_view {
::std::option::Option::Some(view) =>
usage_argv::help::route_to_view(
Self::command(), &__usage_all_refs, cmd, view,
),
::std::option::Option::None => usage_argv::help::route_to(
Self::command(), &__usage_argv, cmd,
),
};
let __usage_page = match __usage_route {
::std::option::Option::Some(route) => {
match __usage_selected_view {
::std::option::Option::Some(view) => {
usage_argv::help::render_view_at_styled(
__usage_spec,
&route,
view,
long,
usage_argv::help::Style::auto(),
)
}
::std::option::Option::None => {
usage_argv::help::render_at_styled(
__usage_spec,
&route,
long,
usage_argv::help::Style::auto(),
)
}
}
}
::std::option::Option::None => {
usage_argv::help::render_styled(
__usage_spec,
cmd,
long,
usage_argv::help::Style::auto(),
)
}
let __usage_want = if long {
usage_argv::help::Page::Long
} else {
usage_argv::help::Page::Short
};
#render_page
match __usage_page {
::std::option::Option::Some(page) => {
::std::print!("{page}");
Expand All @@ -1042,46 +1034,8 @@ pub fn emit(cli: &Cli) -> TokenStream {
}
::std::result::Result::Err(usage_argv::Error::MissingArgsHelp { cmd }) => {
#effective_spec
let __usage_route = match __usage_selected_view {
::std::option::Option::Some(view) =>
usage_argv::help::route_to_view(
Self::command(), &__usage_all_refs, cmd, view,
),
::std::option::Option::None => usage_argv::help::route_to(
Self::command(), &__usage_argv, cmd,
),
};
let __usage_page = match __usage_route {
::std::option::Option::Some(route) => {
match __usage_selected_view {
::std::option::Option::Some(view) => {
usage_argv::help::render_view_at_styled(
__usage_spec,
&route,
view,
false,
usage_argv::help::Style::auto_stderr(),
)
}
::std::option::Option::None => {
usage_argv::help::render_at_styled(
__usage_spec,
&route,
false,
usage_argv::help::Style::auto_stderr(),
)
}
}
}
::std::option::Option::None => {
usage_argv::help::render_styled(
__usage_spec,
cmd,
false,
usage_argv::help::Style::auto_stderr(),
)
}
};
let __usage_want = usage_argv::help::Page::Short;
#render_page_stderr
match __usage_page {
::std::option::Option::Some(page) => {
::std::eprint!("{page}");
Expand All @@ -1092,43 +1046,8 @@ pub fn emit(cli: &Cli) -> TokenStream {
}
::std::result::Result::Err(usage_argv::Error::HelpAll { cmd }) => {
#effective_spec
let __usage_route = match __usage_selected_view {
::std::option::Option::Some(view) =>
usage_argv::help::route_to_view(
Self::command(), &__usage_all_refs, cmd, view,
),
::std::option::Option::None => usage_argv::help::route_to(
Self::command(), &__usage_argv, cmd,
),
};
let __usage_page = match __usage_route {
::std::option::Option::Some(route) => {
match __usage_selected_view {
::std::option::Option::Some(view) => {
usage_argv::help::render_all_view_at_styled(
__usage_spec,
&route,
view,
usage_argv::help::Style::auto(),
)
}
::std::option::Option::None => {
usage_argv::help::render_all_at_styled(
__usage_spec,
&route,
usage_argv::help::Style::auto(),
)
}
}
}
::std::option::Option::None => {
usage_argv::help::render_all_styled(
__usage_spec,
cmd,
usage_argv::help::Style::auto(),
)
}
};
let __usage_want = usage_argv::help::Page::All;
#render_page
match __usage_page {
::std::option::Option::Some(page) => {
::std::print!("{page}");
Expand Down
1 change: 1 addition & 0 deletions docs/.vitepress/config.mts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ export default defineConfig({
{ text: "Help and Errors", link: "/rust/help" },
{ text: "Performance", link: "/rust/performance" },
{ text: "Completions", link: "/rust/completions" },
{ text: "Testing", link: "/rust/testing" },
{ text: "Spec Output", link: "/rust/spec" }
]
},
Expand Down
Loading