global_allocator does not work with -C prefer-dynamic

[nicbn]

Could not find anything related to this online. Using RUSTFLAGS="-C prefer-dynamic" cargo run on this does not panic.

use std::alloc::{GlobalAlloc, Layout};

#[global_allocator]
static ALLOC: Alloc = Alloc;

pub struct Alloc;

unsafe impl GlobalAlloc for Alloc {
    unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
        panic!()
    }
    
    unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 {
        panic!()
    }
    
    unsafe fn realloc(&self, ptr: *mut u8, layout: Layout, new_size: usize) -> *mut u8 {
        panic!()
    }
    
    unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
        panic!()
    }
}

fn main() {
    let _ = Box::new(1);
}

Meta

rustc --version --verbose:

rustc 1.65.0-nightly (29e4a9ee0 2022-08-10)
binary: rustc
commit-hash: 29e4a9ee0253cd39e552a77f51f11f9a5f1c41e6
commit-date: 2022-08-10
host: x86_64-unknown-linux-gnu
release: 1.65.0-nightly
LLVM version: 14.0.6

[Noratrieb]

Does this also happen with -Cpanic=abort? Unwinding from GlobalAlloc is UB, so it would be useful to see a non-UB example.

[Nemo157]

Non-UB (I believe) example that fails for me (SIGSEGV normally, completes successfully with -C prefer-dynamic):

use std::alloc::{GlobalAlloc, Layout};

#[global_allocator]
static ALLOC: Alloc = Alloc;

pub struct Alloc;

#[allow(unconditional_recursion)]
fn abort() -> ! {
    abort()
}

unsafe impl GlobalAlloc for Alloc {
    unsafe fn alloc(&self, _layout: Layout) -> *mut u8 {
        abort()
    }

    unsafe fn alloc_zeroed(&self, _layout: Layout) -> *mut u8 {
        abort()
    }

    unsafe fn realloc(&self, _ptr: *mut u8, _layout: Layout, _new_size: usize) -> *mut u8 {
        abort()
    }

    unsafe fn dealloc(&self, _ptr: *mut u8, _layout: Layout) {
        abort()
    }
}

fn main() {
    let _ = Box::new(1);
}

[nicbn]

You can't use panic=abort with -C prefer-dynamic, because the stdlib you are linking against unwinds (or at least I never managed to get these flags together to compile). But you can do anything inside the function, loop {}, or just return ptr::null_mut(). It just won't do it.

[Noratrieb]

That's a very interesting case. Now, calls to GlobalAlloc are allowed to be optimized out, so this behavior isn't technically illegal, but it's certainly very odd and deserves some deeper investigation.

[nicbn]

Optimization is not the cause of what is going on. Also the allocator will be called before the Box::new by the runtime. You can try any usage for the allocator really, even calling alloc::alloc() directly.

[bjorn3]

It seems that #112794 accidentally fixed this. I was under the impression that rustc would error out when using #[global_allocator] when an upstream dylib already provided the allocator shim. I don't know if this will result in any inconsistencies where libstd will use std::alloc::System as global allocator and the main executable will use #[global_allocator] though on some systems. For ELF based systems it should be fine, but I'm worried that Mach-O based systems (apple) with two level namespaces cause problems and I don't know if PE (windows) supports this kind of interposition from the main executable to dll's.

[ChrisDenton]

I don't know if PE (windows) supports this kind of interposition from the main executable to dll's.

Not really, iirc. You could shim something by having the dll manually get an exported symbol from the exe when the dll is loaded. I'm not sure how I feel about that though.

[bjorn3]

Would making usage of #[global_allocator] when linking a dylib which contains an allocator shim a hard error be acceptable? It didn't work before anyway.

[ChrisDenton]

That sounds reasonable to me. And it doesn't rule out doing something differently in the future.