bootstrap: doc: document std crates one by one

[Fabian-Gruenbichler]

Since switching over from individual, per crate cargo rustdoc invocations to a single cargo doc invocation for all crates, the doc output for Std became racy. Dependencies between library/ workspace crates were handled in a flaky fashion, sometimes being emitted as relative references, sometimes as references to the html root.

This is probably only a workaround, and it should actually be fixed in cargo or rustdoc, but I haven't managed to reproduce this issue outside of the Std doc build via bootstrap.

Closes: #156567

Fixes: 58e18dd

[rustbot]

r? @Mark-Simulacrum

rustbot has assigned @Mark-Simulacrum.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

• Owners of files modified in this PR: bootstrap
• bootstrap expanded to 6 candidates
• Random selection from Mark-Simulacrum, clubby789

[Fabian-Gruenbichler]

force-pushed with clippy fix

[Mark-Simulacrum]

I'm not sure how I feel about the tradeoff here...

cc @rust-lang/bootstrap in case others have ideas

View changes since this review

[Mark-Simulacrum]

It's preexisting, but I think we should be able to just pass target_dir here? Cargo::arg should take Path/OsStr.

[Mark-Simulacrum]

I think this is a regression in terms of needing STD_PUBLIC_CRATES again... right?

cc @jyn514 since you wrote #111955

[Fabian-Gruenbichler]

yes, but it seems it is still required to document things in the right order. I am happy to fix this another way, but I haven't yet found a simple reproducer that allows me to figure out what takes a wrong turn where. if somebody has a clue, I am happy to take a closer look there :)

[jieyouxu]

I'm a bit hesistant on papering over this in bootstrap 🤔 Is there any particular combo that makes it more likely to trigger this?

EDIT: ah, the discussion on the issue also points to rust-lang/cargo#8487.

Also not super sure on the tradeoffs. Do we know if using this approach can bring back other problems?

[Fabian-Gruenbichler]

my reproducer was simply running x.py doc --stage 2 a few times and comparing the generated html files. it sometimes takes up to 10 tries on my machine until it does it wrong, but usually the first "rebuild" already has differences.

[Kobzol]

Is there a way to override the order without documenting the crates one by one? Sure would be nice to fix this in Cargo than to add yet another hack to bootstrap :)

[Fabian-Gruenbichler]

AFAIK/AFAICT cargo already tries to pick the right order, but somehow still sometimes takes a wrong turn. I tried manually reproducing it with a workspace and crates referencing eachother, but failed to do so. I think the way bootstrap calls cargo doc here is a bit special (always passing --manifest-path pointing at sysroot, which in turn references other crates, but also specifying -p .. for all the documented crates), but even with trying to mimic those I failed to reproduce it with a minimized set of crates.