Memory use grows over time

[saethlin]

There are a few places in both the interpreter and the additional Miri runtime where we accumulate state and never release it.

For example, this program uses more memory over time:

fn main() {
    for _ in 0..1_000_000 {}
}

(I like this program because it seems comically simple but of course ranges are quite complicated without any optimization)

There are a few sources of this memory growth. I'm going to try to fix or reduce as many as possible. In most cases the growth is manageable but I have a real world case someone showed me where we hit ~10 GB in just a few minutes due to these issues.

• Stacked Borrows Allocation history and in particular the invalidations is the biggest offender. I think we can fix this with the tag GC. I'll send a PR tomorrow.
• The spans we save for all allocations/locals are probably next. Not really sure what to do here, maybe also use the tag GC + the set of exposed allocs.
• Intptrcast state cannot just be cleaned up on deallocation. I tried doing that and I didn't get the test failures I was hoping to see. Perhaps the tag GC can help us again here, or maybe we can only clean up one side of the mapping.
• The dead allocs map in the interpreter. Based on the comments, I don't think we need this in Miri?
• Return a finite number of AllocIds per ConstAllocation in Miri rust#118336

The trickiest thing about these is that so far it looks like fixing these problems does not make Miri any faster or slower, and we don't have peak memory benchmarks. And hyperfine doesn't support them.

[RalfJung]

Intptrcast state cannot just be cleaned up on deallocation. I tried doing that and I didn't get the test failures I was hoping to see. Perhaps the tag GC can help us again here, or maybe we can only clean up one side of the mapping.

This would need an AllocId GC instead of tag GC, and even then I am not sure whether it is sufficient. We'd have to be very sure that future int2ptr calls behave the same before after after cleanup.

The dead allocs map in the interpreter. Based on the comments, I don't think we need this in Miri?

It will be used at least with -Zmiri-symbolic-alignment.

[saethlin]

We'd have to be very sure that future int2ptr calls behave the same before after after cleanup.

This is what I was getting at with the "+ the set of exposed allocs". I think expose_addr is simply asking for a leak, and so it shall be in the interpreter.

[saethlin]

Since rust-lang/rust#118336 landed, I think this is done. I don't observe memory growth anymore.