Transmuting struct that contains itself causes compiler stack overflow

[theemathas]

Code

This uses a similar construction to #148653 and #153205, with a struct containing itself directly as a field.

See also #155423 for issues with recursion in transmutes.

The stack overflow occurs in SizeSkeleton::compute, which is the code for checking whether two potentially-generic types necessarily have the same size for the purpose of transmutes.

cc @RalfJung @lcnr

use std::mem::transmute;

trait Trait {
    type Assoc;
}
impl<T> Trait for T {
    type Assoc = T;
}
// Effectively:
// type Alias<T> = T;
type Alias<T> = <T as Trait>::Assoc;

pub struct Thing<T: ?Sized>(*const T, Alias<Thing<T>>);

pub fn weird<T: ?Sized>(value: Thing<T>) {
    let _: i32 = unsafe { transmute(value) };
}

Meta

Reproducible on the playground with version 1.97.0-nightly (2026-05-03 ad3a598ca4bc7c68bcbb)

Error output

error: rustc interrupted by SIGSEGV, printing backtrace

Backtrace

/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-6b2bc02a44a4715e.so(+0x3b0feec)[0x75852df0feec]
/lib/x86_64-linux-gnu/libc.so.6(+0x45330)[0x75852a233330]
/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-6b2bc02a44a4715e.so(+0x523a53e)[0x75852f63a53e]
/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-6b2bc02a44a4715e.so(+0x523a91b)[0x75852f63a91b]
/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-6b2bc02a44a4715e.so(_RNvMsD_NtCs4DdX3hukfYu_12rustc_middle2tyNtB5_8FieldDef2ty+0x152)[0x75852f638092]
/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-6b2bc02a44a4715e.so(_RNvMs3_NtNtCs4DdX3hukfYu_12rustc_middle2ty6layoutNtB5_12SizeSkeleton7compute+0x3a5)[0x7585309dae25]

### cycle encountered after 6 frames with period 8
/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-6b2bc02a44a4715e.so(_RNvMs3_NtNtCs4DdX3hukfYu_12rustc_middle2ty6layoutNtB5_12SizeSkeleton7compute+0x418)[0x7585309dae98]
/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-6b2bc02a44a4715e.so(_RNvMs3_NtNtCs4DdX3hukfYu_12rustc_middle2ty6layoutNtB5_12SizeSkeleton7compute+0x3b8)[0x7585309dae38]
/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-6b2bc02a44a4715e.so(_RNvMs3_NtNtCs4DdX3hukfYu_12rustc_middle2ty6layoutNtB5_12SizeSkeleton7compute+0x418)[0x7585309dae98]
/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-6b2bc02a44a4715e.so(_RNvMs3_NtNtCs4DdX3hukfYu_12rustc_middle2ty6layoutNtB5_12SizeSkeleton7compute+0x3b8)[0x7585309dae38]
/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-6b2bc02a44a4715e.so(_RNvMs3_NtNtCs4DdX3hukfYu_12rustc_middle2ty6layoutNtB5_12SizeSkeleton7compute+0x418)[0x7585309dae98]
/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-6b2bc02a44a4715e.so(_RNvMs3_NtNtCs4DdX3hukfYu_12rustc_middle2ty6layoutNtB5_12SizeSkeleton7compute+0x3b8)[0x7585309dae38]
/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-6b2bc02a44a4715e.so(_RNvMs3_NtNtCs4DdX3hukfYu_12rustc_middle2ty6layoutNtB5_12SizeSkeleton7compute+0x418)[0x7585309dae98]
/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-6b2bc02a44a4715e.so(_RNvMs3_NtNtCs4DdX3hukfYu_12rustc_middle2ty6layoutNtB5_12SizeSkeleton7compute+0x3b8)[0x7585309dae38]
### recursed 31 times

/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-6b2bc02a44a4715e.so(_RNvMs3_NtNtCs4DdX3hukfYu_12rustc_middle2ty6layoutNtB5_12SizeSkeleton7compute+0x418)[0x7585309dae98]
/playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-6b2bc02a44a4715e.so(_RNvMs3_NtNtCs4DdX3hukfYu_12rustc_middle2ty6layoutNtB5_12SizeSkeleton7compute+0x3b8)[0x7585309dae38]

note: rustc unexpectedly overflowed its stack! this is a bug
note: maximum backtrace depth reached, frames may have been lost
note: we would appreciate a report at https://github.com/rust-lang/rust
help: you can increase rustc's stack size by setting RUST_MIN_STACK=16777216
note: backtrace dumped due to SIGSEGV! resuming signal
error: could not compile `playground` (lib)

[RalfJung]

Cc @oli-obk -- do we just need to call this stack growth gadget or how is this usually handled?

[oli-obk]

That won't help here (it will turn the stack overflow into a hang/oom)

We need to teach skeleton computation about cycles or add a recursion limit to them. Cycle detection + stack growing probably suffices if the check is for self-recursion just based on DefId. That will regress Struct<Struct<()>> tho. So maybe just a recursion limit plus stack growing

[rajgandhi1]

can this be solved by implementing cycle detection and tracking the types being currently visited?

[oli-obk]

probably not, because you get infinitly growing types like Thing<Thing<Thing<...>>>

[rajgandhi1]

what if an active-path guard is implemented (e.g., a FxHashSet threaded through the recursion) to track types in the current branch. By checking the normalised form against this path before descending, we can detect cycles and return Errors gracefully.

[oli-obk]

If we run normalization, we probably hit the regular recursion limit errors anyway. But we can't always run it. I think just adding a recursion limit to catch such infinite recursions seems best. There should then be follow-up errors that are nicer

[rajgandhi1]

ah ya makes sense. I can help implement it. Should I write it and create a PR?