Skip to content

Resolve: more preperation work for parallelizing the import resolution loop - #159440

Merged
rust-bors[bot] merged 1 commit into
rust-lang:mainfrom
LorrensP-2158466:inplace-import-res
Jul 21, 2026
Merged

Resolve: more preperation work for parallelizing the import resolution loop#159440
rust-bors[bot] merged 1 commit into
rust-lang:mainfrom
LorrensP-2158466:inplace-import-res

Conversation

@LorrensP-2158466

@LorrensP-2158466 LorrensP-2158466 commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

View all comments

This is basically #158845 but we do not:

  • actually use par_slice because:
  • we do not migrate CmRefCell to use RwLocks (yet) because of perf reasons.

The resolution loop is now in place instead of recollecting indeterminate imports.

r? @petrochenkov

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Jul 17, 2026
@rust-log-analyzer

This comment has been minimized.

Comment thread compiler/rustc_resolve/src/lib.rs Outdated
@petrochenkov petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 17, 2026
@LorrensP-2158466

Copy link
Copy Markdown
Contributor Author

I double-checked with git diff f3d5c561f4c compiler/rustc_resolve and I only see comments or the unsafe impls as + diff.

@rustbot ready

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jul 17, 2026
@rust-log-analyzer

This comment has been minimized.

Comment thread compiler/rustc_resolve/src/build_reduced_graph.rs Outdated
Comment thread compiler/rustc_resolve/src/ident.rs
Comment thread compiler/rustc_resolve/src/imports.rs Outdated
Comment thread compiler/rustc_resolve/src/imports.rs Outdated
self.write_import_resolutions(&to_resolve_imports);

self.indeterminate_imports = to_resolve_imports
.extract_if(.., |(_, _, count)| {

@petrochenkov petrochenkov Jul 17, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nicer to extract_if into determined_imports instead and then do self.indeterminate_imports = imports_to_resolve;.
But it may be slower due to majority of the elements in the vector being determinate.
I'll benchmark it separately later after this PR lands.

View changes since the review

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would expect it to be faster, now that I thought about more clearly.

Currently we extract all indeterminate imports, which will leave small holes in the vector and thus requiring a lot of chunks to be moved. And then we still have to iterate over the vector such that we only get the import itself.

Extracting into determined imports is the same kind of map as we have now, with the occasional skip of an undetermined import. And then we just have a simple assignment. This looks like less work.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in #159681.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in #159681.

It's a minor regression.

Comment thread compiler/rustc_resolve/src/imports.rs Outdated
Comment thread compiler/rustc_resolve/src/lib.rs Outdated
@petrochenkov petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 17, 2026
…par_slice`.

Currently it is all still single threaded, but almost everything needed to make the loop use multithreaded code is done in this commit.

For performance reasons we do the resolution now in place instead of recollecting the indeterminate imports ([see benchmark pr](rust-lang#159387) and pr of this commit).

Populating external resolution tables now use `Once` to ensure only 1 thread builds the table and all others wait if they need it, this has no impact on single threaded behaviour.

`Cache(Ref)Cells` are not tackled yet because they are not needed here (and require extra work).
@LorrensP-2158466

Copy link
Copy Markdown
Contributor Author

Force pushed to keep it at 1 commit, @rustbot ready.

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jul 18, 2026
@petrochenkov

Copy link
Copy Markdown
Contributor

Perf-sensitive change, benchmarked in #159387 (comment).
@bors r+ rollup=never

@rust-bors

rust-bors Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

📌 Commit 2f3022a has been approved by petrochenkov

It is now in the queue for this repository.

@rust-bors rust-bors Bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 20, 2026
@rust-bors

This comment has been minimized.

@rust-bors rust-bors Bot added merged-by-bors This PR was explicitly merged by bors. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jul 21, 2026
@rust-bors

rust-bors Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

☀️ Test successful - CI
Approved by: petrochenkov
Duration: 3h 31m 11s
Pushing f65b272 to main...

@rust-bors
rust-bors Bot merged commit f65b272 into rust-lang:main Jul 21, 2026
14 checks passed
@github-actions

Copy link
Copy Markdown
Contributor
What is this? This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing cde3f8a (parent) -> f65b272 (this PR)

Test differences

Show 14 test diffs

14 doctest diffs were found. These are ignored, as they are noisy.

Test dashboard

Run

cargo run --manifest-path src/ci/citool/Cargo.toml -- \
    test-dashboard f65b272fc92b1e7527dea4a224430a249ab25c2d --output-dir test-dashboard

And then open test-dashboard/index.html in your browser to see an overview of all executed tests.

Job duration changes

  1. x86_64-gnu-debug: 1h 36m -> 2h 31m (+56.8%)
  2. x86_64-gnu-llvm-21-2: 1h 41m -> 1h 4m (-36.8%)
  3. dist-armv7-linux: 1h 32m -> 58m 33s (-36.8%)
  4. dist-various-1: 56m 14s -> 1h 16m (+35.4%)
  5. optional-x86_64-gnu-parallel-frontend: 2h 11m -> 1h 25m (-35.3%)
  6. x86_64-gnu-nopt: 1h 53m -> 2h 32m (+34.4%)
  7. i686-gnu-nopt-2: 1h 45m -> 2h 20m (+33.5%)
  8. x86_64-gnu-llvm-21-3: 1h 23m -> 1h 50m (+32.5%)
  9. armhf-gnu: 1h 10m -> 1h 33m (+32.3%)
  10. dist-x86_64-llvm-mingw: 1h 35m -> 2h (+26.7%)
How to interpret the job duration changes?

Job durations can vary a lot, based on the actual runner instance
that executed the job, system noise, invalidated caches, etc. The table above is provided
mostly for t-infra members, for simpler debugging of potential CI slow-downs.

@rust-timer

Copy link
Copy Markdown
Collaborator

Finished benchmarking commit (f65b272): comparison URL.

Overall result: ✅ improvements - no action needed

@rustbot label: -perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-0.2% [-0.3%, -0.2%] 18
Improvements ✅
(secondary)
-0.6% [-1.5%, -0.1%] 29
All ❌✅ (primary) -0.2% [-0.3%, -0.2%] 18

Max RSS (memory usage)

Results (primary 1.6%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)
1.6% [1.6%, 1.6%] 1
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) 1.6% [1.6%, 1.6%] 1

Cycles

This perf run didn't have relevant results for this metric.

Binary size

Results (primary -0.0%, secondary -0.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-0.0% [-0.0%, -0.0%] 4
Improvements ✅
(secondary)
-0.0% [-0.1%, -0.0%] 15
All ❌✅ (primary) -0.0% [-0.0%, -0.0%] 4

Bootstrap: 486.164s -> 487.415s (0.26%)
Artifact size: 387.87 MiB -> 387.80 MiB (-0.02%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merged-by-bors This PR was explicitly merged by bors. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants