Adressing compile time regression from loop hint attributes #161217

#156816 added new loop hint attributes (unroll and various friends). This caused a small compile time regression. At the time of merge, the regression was believed to be caused by adding attributes field to Terminator, which increased Terminator (and BasicBlockData ) size by 8 bytes.

However, this doesn't seem to be the case (at least anymore). I tried to mess around with the Terminator in various ways to decrese its size, but I wasn't able to get any significant changes. Here's a few pieces of evidence:

• [wip] terminator perf experiments #159928 shrinks Terminator by 8 bytes, no significant changes.
• wip: implement encodable for terminator manually to save a byte #160857 implements Encodable manually to remove the metadata size overhead. No significant changes. This just confirms that metadata size increase is not the problem, which was already noted in the original PR.
• Change SwitchTarget (in TerminatorKind::SwitchInt) to use ScalarInt #124127 adds 8 bytes to Terminator, no significant changes
• In [experiment] remove attributes from Terminator #161167, I finally figure out I could estimate this impact better by just removing the array copletely and didn't get any significant changes. This means the cost is truly not in the Terminator::attributes.

It's possible that the original analysis was correct, but something changed since then:

• The bottleneck shifted somewhere else in the meantime. For example, BasicBlockData size was 160 bytes at the time of writing, but it was reduced to 144 bytes since.

• The regression was fixed by something else (e.g. LLVM PR which optimized the releated rustc code better).

Either way, the original justification for accepting the regression doesn't seem to apply anymore, which means it might be worth addressing.

Some TODO items to try to narrow it down:

• try to run perf on revert of the original PR to test whether the regression still exists (and where it is at the moment). This is a bit annoying because the PR doesn't revert cleanly anymore, but it's doable.
• check br_with_attrs cost, maybe outline the cold attribute list processing (there might be some overhead in processing the empty list that LLVM is not able to optimize). Doesn't seem to be likely to me.

That said, based on eyeballing the detailed results in the original PR, I think the cost is somewhere before MIR, so that's probably the place to investigate.

The regression is small, so it might not be worth the opportunity cost, but I already sank some time into it so I figured I might as well just write this down so it's not forgotten.