Skip to content

Miscompilation with FFI bool return type on x86_64 #163911

Closed

Description

@wezm

I'm observing erroneous behaviour around bool and FFI. A Rust function returning bool is producing values other than 0 or 1. I have a minimal reproduction here, which reproduces the issue on x86_64 Linux:

https://github.com/yeslogic/rust-bool-ffi

gcc and clang both reproduce.

For reference this is the C and Rust code:

#include <stdio.h>
#include <stdbool.h>

extern bool rust_fn_returning_bool(unsigned char);

int main(int argc, char **argv) {
  bool ret = rust_fn_returning_bool(0x10);

  printf("ret = %d\n", (int)ret);

  return 0;
}
#[unsafe(no_mangle)]
pub unsafe extern "C" fn rust_fn_returning_bool(flags: u8) -> bool {
    (flags & 0x1) == 0x1
}

When ./ffi is run I get the output: 16 instead of the expected 0 or 1.

rust_fn_returning_bool compiles to the following on 1.99.0:

0000000000000000 <rust_fn_returning_bool>:
       0: 89 f8                         movl    %edi, %eax
       2: c3                            retq

And this on 1.98.0:

0000000000000000 <rust_fn_returning_bool>:
       0: 89 f8                         movl    %edi, %eax
       2: 24 01                         andb    $0x1, %al
       4: c3                            retq

That missing and appears to be the source of the issue.

Meta

rustc --version --verbose:

rustc 1.99.0 (b940084d7 2026-09-28)
binary: rustc
commit-hash: b940084d7eb6a299eb4bfeb8e34901bc051e7ac4
commit-date: 2026-09-28
host: x86_64-unknown-linux-gnu
release: 1.99.0
LLVM version: 23.1.1

The issue also occurs on nightly:

rustc 1.101.0-nightly (ea137335b 2026-10-05)
binary: rustc
commit-hash: ea137335b78829b4514bf1b4c16302f74fab8581
commit-date: 2026-10-05
host: x86_64-unknown-linux-gnu
release: 1.101.0-nightly
LLVM version: 23.1.3

I've only reproduced on x86_64. On aarch64 and loongarch64 the value is masked before returning.

Possibility related to:

Activity

  1. added
    needs-triageThis issue may need triage. Remove when done. See docs forge.rust-lang.org/release/issue-triaging
    on Oct 7, 2026
  2. asquared31415 commented on Oct 7, 2026

    @asquared31415
    Contributor

    The LLVM IR emitted by rustc changed from

    define noundef zeroext i1 @rust_fn_returning_bool
    

    to

    define noundef i1 @rust_fn_returning_bool
    

    Which then causes LLVM to believe that it can return the higher bits unmodified. The test that was added to ensure that our bool handling was correct is not testing x86_64-unknown-linux-gnu, it's testing x86_64-unknown-uefi, which for some reason has a different (correct) bool handling???

    @rustbot label -needs-triage +T-compiler +I-miscompile +A-llvm +A-ABI +I-prioritize

  3. added
    A-ABIArea: Concerning the application binary interface (ABI)
    A-LLVMArea: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues.
    I-miscompileIssue: Correct Rust code lowers to incorrect machine code
    I-prioritizeIssue needs a team member to assess the impact. Will be replaced by P-{low,medium,high,critical}
    T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.
    and removed
    needs-triageThis issue may need triage. Remove when done. See docs forge.rust-lang.org/release/issue-triaging
    on Oct 7, 2026
  4. ds84182 commented on Oct 7, 2026

    @ds84182

    llvm/llvm-project#12579 (comment)

    The x86-64 psabi commit linked in the LLVM issue above has always said that bits 1-7 must be 0, but 8-N is undefined. So a bool return type must be zero extended to at least 8 bits.

  5. asquared31415 commented on Oct 7, 2026

    @asquared31415
    Contributor

    The issue is #142389 removed zero extension of all scalar return types for x86_64, but #159317 uncovered the bug, since before then, there was code unconditionally zero extending it.

  6. self-assigned this
    on Oct 7, 2026
  7. removed
    I-prioritizeIssue needs a team member to assess the impact. Will be replaced by P-{low,medium,high,critical}
    on Oct 7, 2026
  8. rustbot commented on Oct 7, 2026

    @rustbot
    Collaborator

    Assigning P-critical (discussion on Zulip).

  9. 25 remaining items

  10. added a commit that references this issue on Oct 21, 2026
  11. added a commit that references this issue on Oct 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

A-ABIArea: Concerning the application binary interface (ABI)A-LLVMArea: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues.A-codegenArea: Code generationC-bugCategory: This is a bug.I-miscompileIssue: Correct Rust code lowers to incorrect machine codeP-criticalCritical priorityT-compilerRelevant to the compiler team, which will review and decide on the PR/issue.regression-from-stable-to-stablePerformance or correctness regression from one stable version to another.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions