LLVM's pass ordering chokes on zero-cost abstractions.

[eddyb]

View all comments

The latest example of this I've seen was posted by @Veedrac to Hacker News (try on playground):

static table: [i32; 4] = [0; 4];

pub fn exists_in_table(v: i32) -> bool {
    for &x in table.iter() {
        if x == v {
            return true;
        }
    }

    false
}

After -C opt-level=3 the ~600 lines of unoptimized IR turn into this (note br i1 false):

; playground::exists_in_table
; Function Attrs: nounwind uwtable
define zeroext i1 @_ZN10playground15exists_in_table17h0a39bf7934b9be37E(i32) unnamed_addr #0 {
start:
  br label %bb3

bb3:                                              ; preds = %start
  br i1 false, label %bb9, label %bb6

bb6:                                              ; preds = %bb3
  %1 = icmp eq i32 %0, 0
  br i1 %1, label %bb9, label %bb3.1

bb9:                                              ; preds = %bb6.4, %bb3.4, %bb6.3, %bb3.3, %bb6.2, %bb3.2, %bb6.1, %bb3.1, %bb3, %bb6
  %_0.0 = phi i1 [ true, %bb6 ], [ false, %bb3 ], [ false, %bb3.1 ], [ true, %bb6.1 ], [ false, %bb3.2 ], [ true, %bb6.2 ], [ false, %bb3.3 ], [ true, %bb6.3 ], [ false, %bb3.4 ], [ true, %bb6.4 ]
  ret i1 %_0.0

bb3.1:                                            ; preds = %bb6
  br i1 false, label %bb9, label %bb6.1

bb6.1:                                            ; preds = %bb3.1
  %2 = icmp eq i32 %0, 0
  br i1 %2, label %bb9, label %bb3.2

bb3.2:                                            ; preds = %bb6.1
  br i1 false, label %bb9, label %bb6.2

bb6.2:                                            ; preds = %bb3.2
  %3 = icmp eq i32 %0, 0
  br i1 %3, label %bb9, label %bb3.3

bb3.3:                                            ; preds = %bb6.2
  br i1 false, label %bb9, label %bb6.3

bb6.3:                                            ; preds = %bb3.3
  %4 = icmp eq i32 %0, 0
  br i1 %4, label %bb9, label %bb3.4

bb3.4:                                            ; preds = %bb6.3
  br i1 true, label %bb9, label %bb6.4

bb6.4:                                            ; preds = %bb3.4
  br label %bb9
}

[nagisa]

cc #33299

[andjo403]

I'm very new to llvm and compilers but I was trying to see what the problem was with this so I was running:
rustc -Cllvm-args='-print-after-all' -O --emit llvm-ir badopt.rs |& rustfilt and get a part of the log that looks like this:

*** IR Dump After Loop-Closed SSA Form Pass ***
; Function Attrs: nounwind uwtable
define zeroext i1 @badopt::exists_in_table(i32) unnamed_addr #0 {
start:
  br label %bb3

bb3:                                              ; preds = %bb6, %start
  %iter.sroa.0.0 = phi i32* [ getelementptr inbounds ([4 x i32], [4 x i32]* @badopt::table, i64 0, i64 0), %start ], [ %2, %bb6 ]
  %1 = icmp eq i32* %iter.sroa.0.0, getelementptr inbounds ([4 x i32], [4 x i32]* @badopt::table, i64 1, i64 0)
  br i1 %1, label %bb9, label %bb6

bb6:                                              ; preds = %bb3
  %2 = getelementptr inbounds i32, i32* %iter.sroa.0.0, i64 1
  %3 = load i32, i32* %iter.sroa.0.0, align 4
  %4 = icmp eq i32 %3, %0
  br i1 %4, label %bb9, label %bb3

bb9:                                              ; preds = %bb3, %bb6
  %_0.0 = phi i1 [ true, %bb6 ], [ false, %bb3 ]
  ret i1 %_0.0
}
*** IR Dump After Combine redundant instructions ***
; Function Attrs: nounwind uwtable
define zeroext i1 @badopt::exists_in_table(i32) unnamed_addr #0 {
start:
  br label %bb3

bb3:                                              ; preds = %start
  br i1 false, label %bb9, label %bb6

bb6:                                              ; preds = %bb3
  %1 = icmp eq i32 %0, 0
  br i1 %1, label %bb9, label %bb3.1

bb9:                                              ; preds = %bb6.4, %bb3.4, %bb6.3, %bb3.3, %bb6.2, %bb3.2, %bb6.1, %bb3.1, %bb3, %bb6
  %_0.0 = phi i1 [ true, %bb6 ], [ false, %bb3 ], [ false, %bb3.1 ], [ true, %bb6.1 ], [ false, %bb3.2 ], [ true, %bb6.2 ], [ false, %bb3.3 ], [ true, %bb6.3 ], [ false, %bb3.4 ], [ true, %bb6.4 ]
  ret i1 %_0.0

bb3.1:                                            ; preds = %bb6
  br i1 false, label %bb9, label %bb6.1

bb6.1:                                            ; preds = %bb3.1
  %2 = icmp eq i32 %0, 0
  br i1 %2, label %bb9, label %bb3.2

bb3.2:                                            ; preds = %bb6.1
  br i1 false, label %bb9, label %bb6.2

bb6.2:                                            ; preds = %bb3.2
  %3 = icmp eq i32 %0, 0
  br i1 %3, label %bb9, label %bb3.3

bb3.3:                                            ; preds = %bb6.2
  br i1 false, label %bb9, label %bb6.3

bb6.3:                                            ; preds = %bb3.3
  %4 = icmp eq i32 %0, 0
  br i1 %4, label %bb9, label %bb3.4

bb3.4:                                            ; preds = %bb6.3
  br i1 true, label %bb9, label %bb6.4

bb6.4:                                            ; preds = %bb3.4
  br label %bb9
}

after the Combine redundant instructions (instcombine) pass there is no more Simplify the CFG (simplifycfg). To me it looks like instcombine change the CFG. But the instcombine pass description states that "This pass does not modify the CFG." is it possible that this is the problem?

[eddyb]

@andjo403 Oh wow, that is very useful, thanks! I thought I was wrong in assuming that a very complex interaction between passes causes this result. Those are consecutive passes, right?

I don't understand how there's more basic blocks after instcombine. Looking more at it, the .1, .2, .3 and .4 suffixes suggest loop unrolling, but instcombine shouldn't do that AFAIK.

[andjo403]

I added -print-before-all also and then there is an Unroll loops between the passes from the log before so instcombine is not the problem. new log:

*** IR Dump Before Unroll loops ***
bb3:                                              ; preds = %bb6, %start
  %iter.sroa.0.0 = phi i32* [ getelementptr inbounds ([4 x i32], [4 x i32]* @badopt::table, i64 0, i64 0), %start ], [ %2, %bb6 ]
  %1 = icmp eq i32* %iter.sroa.0.0, getelementptr inbounds ([4 x i32], [4 x i32]* @badopt::table, i64 1, i64 0)
  br i1 %1, label %bb9, label %bb6

bb6:                                              ; preds = %bb3
  %2 = getelementptr inbounds i32, i32* %iter.sroa.0.0, i64 1
  %3 = load i32, i32* %iter.sroa.0.0, align 4
  %4 = icmp eq i32 %3, %0
  br i1 %4, label %bb9, label %bb3
*** IR Dump Before Combine redundant instructions ***
; Function Attrs: nounwind uwtable
define zeroext i1 @badopt::exists_in_table(i32) unnamed_addr #0 {
start:
  br label %bb3

bb3:                                              ; preds = %start
  br i1 false, label %bb9, label %bb6

bb6:                                              ; preds = %bb3
  %1 = icmp eq i32 0, %0
  br i1 %1, label %bb9, label %bb3.1

bb9:                                              ; preds = %bb6.4, %bb3.4, %bb6.3, %bb3.3, %bb6.2, %bb3.2, %bb6.1, %bb3.1, %bb3, %bb6
  %_0.0 = phi i1 [ true, %bb6 ], [ false, %bb3 ], [ false, %bb3.1 ], [ true, %bb6.1 ], [ false, %bb3.2 ], [ true, %bb6.2 ], [ false, %bb3.3 ], [ true, %bb6.3 ], [ false, %bb3.4 ], [ true, %bb6.4 ]
  ret i1 %_0.0

bb3.1:                                            ; preds = %bb6
  br i1 false, label %bb9, label %bb6.1

bb6.1:                                            ; preds = %bb3.1
  %2 = icmp eq i32 0, %0
  br i1 %2, label %bb9, label %bb3.2

bb3.2:                                            ; preds = %bb6.1
  br i1 false, label %bb9, label %bb6.2

bb6.2:                                            ; preds = %bb3.2
  %3 = icmp eq i32 0, %0
  br i1 %3, label %bb9, label %bb3.3

bb3.3:                                            ; preds = %bb6.2
  br i1 false, label %bb9, label %bb6.3

bb6.3:                                            ; preds = %bb3.3
  %4 = icmp eq i32 0, %0
  br i1 %4, label %bb9, label %bb3.4

bb3.4:                                            ; preds = %bb6.3
  br i1 true, label %bb9, label %bb6.4

bb6.4:                                            ; preds = %bb3.4
  br label %bb9
}

apparently Unroll loops do not make a IR dump after the pass strange.

[eddyb]

Okay, so the question is why would loop unrolling not have a CFG simplification pass after it?

[eddyb]

This seems like the right combination of passes: https://github.com/rust-lang/llvm/blob/d9e7d2696e41983b6b5a0b4fac604b4e548a84d3/lib/Transforms/IPO/PassManagerBuilder.cpp#L782-L790

Also good here: https://github.com/rust-lang/llvm/blob/d9e7d2696e41983b6b5a0b4fac604b4e548a84d3/lib/Transforms/IPO/PassManagerBuilder.cpp#L620-L625

This on the other hand looks bad: https://github.com/rust-lang/llvm/blob/d9e7d2696e41983b6b5a0b4fac604b4e548a84d3/lib/Transforms/IPO/PassManagerBuilder.cpp#L374

And this is probably the one that unrolls the loop here: https://github.com/rust-lang/llvm/blob/d9e7d2696e41983b6b5a0b4fac604b4e548a84d3/lib/Transforms/IPO/PassManagerBuilder.cpp#L629-L632

[nossralf]

So @andjo403 and I spoke about this at lunch the other day, and I took some time today to just add a CFG simplification pass at the places you outlined above to see what happened.

Turns out, adding it on line 633 after the unroll/instruction simplification fixes the issue. The resulting IR for the function reduces to:

; Function Attrs: nounwind uwtable
define zeroext i1 @_ZN6static15exists_in_table17hbdcb2c4e00b87deaE(i32) unnamed_addr #0 {
start:
  %1 = icmp eq i32 %0, 0
  %. = select i1 %1, i1 true, i1 false
  ret i1 %.
}

which is what we want, I believe.

I suspect that the unroll on line 374 may in fact not be a problem thanks to the CFG simplification pass that follows it on line 382. (Unless the LoadCombine/DCE passes that run in between cause changes that negatively affect SimplifyCFG. I know nothing about LLVM internals, but my hunch is that it's unlikely.)

[hanna-kruppe]

That's great!

I can't help but notice, though, that the IR you showed could be simplified further by instcombine. Maybe it would make sense to add the simplifyCFG before the instcombine, or add another instcombine afterwards?

[nossralf]

I'll run a couple of builds trying that out.

[nossralf]

Nice, simply moving SimplifyCFG up before InstCombine gives this:

; Function Attrs: nounwind uwtable
define zeroext i1 @_ZN6static15exists_in_table17hbdcb2c4e00b87deaE(i32) unnamed_addr #0 {
start:
  %1 = icmp eq i32 %0, 0
  ret i1 %1
}

Adding another InstCombine after the SimplifyCFG from my previous attempt gives the same output.

Not sure if there are situations where the longer incantation of Unroll>InstCombine>SimplifyCFG>InstCombine would be preferable to just Unroll>SimplifyCFG>InstCombine.

[hanna-kruppe]

Yeah I'm unsure as well. Running more passes for no good reason should be avoided, but InstCombine can also unlock opportunities for CFG simplification (by turning branch conditions into constants), so in general there no single right order. In fact, some of the PMB pieces @eddyb linked above run InstCombine>SimplifyCFG and others run SimplifyCFG>InstCombine (though none run IC>SimplifyCFG>IC, IIRC). Maybe LLVM developers have an idea? (Has someone already filed an LLVM issue for this?)