Find a solution to prevent processes being OOM killed due to inaccurate memory accounting #25650

Is your feature request related to a problem or challenge?

DataFusion's memory accounting isn't accurate. When a process runs in a container (e.g. on k8s) and allocates more memory than is available, it gets OOM killed. This hits downstream projects like Comet: https://datafusion.apache.org/comet/contributor-guide/memory_management.html

Let's use this issue to discuss approaches.

Describe the solution you'd like

One idea is to hook into the actual allocator (e.g. jemalloc) so we can see real allocated bytes, and then:

• log actual usage next to the memory pool's reported usage, so we can at least see where they drift
• add a memory pool that rejects allocations once real allocated bytes go over a limit

We could also work on improving the estimated memory usage in various operators and expressions.

Additional context

• Prior work: Track allocator-level memory vs MemoryPool during SLTs to prevent OOMs #22626 added allocator-level tracking vs MemoryPool in SLTs, but it was reverted in Revert custom allocator auditing of MemoryPool tracking in SLTs #22860
• Brent Gardner's OomGuard talk at the Denver DataFusion meetup: recording, slides
• Related: [EPIC] A collection of Memory Accounting Limitations and Improvements #22758