Repository navigation
ComfyUI fails to cache model to RAM, reloads from disk every run. #14076
Description
Custom Node Testing
- I have tried disabling custom nodes and the issue persists (see how to disable custom nodes if you need help)
Expected Behavior
Cache models to RAM as needed.
Actual Behavior
Reads models from disk once they are evicted from VRAM, meaning every single time you run a workflow with multiple models that don't fit entirely into VRAM.
Steps to Reproduce
Run this workflow on 24GB VRAM. Marvel at endless disk loading every time you re-run the workflow.
Debug Logs
Downloads/ComfyUINew$ uv run main.py
setup plugin alembic.autogenerate.schemas
setup plugin alembic.autogenerate.tables
setup plugin alembic.autogenerate.types
setup plugin alembic.autogenerate.constraints
setup plugin alembic.autogenerate.defaults
setup plugin alembic.autogenerate.comments
Found comfy_kitchen backend triton: {'available': True, 'disabled': True, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8']}
Found comfy_kitchen backend cuda: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'scaled_mm_nvfp4']}
Found comfy_kitchen backend eager: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': ['apply_rope', 'apply_rope1', 'dequantize_mxfp8', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'scaled_mm_mxfp8', 'scaled_mm_nvfp4']}
Checkpoint files will always be loaded safely.
Total VRAM 24105 MB, total RAM 63989 MB
pytorch version: 2.12.0+cu130
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 3090 Ti : cudaMallocAsync
Using async weight offloading with 2 streams
Enabled pinned memory 57590.0
Using pytorch attention
aimdo: /project/src-posix/cuda-funchooks.c:52:DEBUG:aimdo_setup_hooks: hooks successfully installed
aimdo: /project/src/control.c:236:INFO:comfy-aimdo inited for GPU: NVIDIA GeForce RTX 3090 Ti (VRAM: 24105 MB)
DynamicVRAM support detected and enabled
Python version: 3.12.3 (main, Jan 8 2026, 11:30:50) [GCC 13.3.0]
ComfyUI version: 0.22.0
comfy-aimdo version: 0.4.3
comfy-kitchen version: 0.2.8
****** User settings have been changed to be stored on the server instead of browser storage. ******
****** For multi-user setups add the --multi-user CLI argument to enable multiple user profiles. ******
comfyui-frontend-package version: 1.43.18
comfyui-workflow-templates version: 0.9.82
comfyui-embedded-docs version: 0.5.0
comfy-kitchen version: 0.2.8
comfy-aimdo version: 0.4.3
[Prompt Server] web root: /home/il/Downloads/ComfyUINew/.venv/lib/python3.12/site-packages/comfyui_frontend_package/static
Asset seeder disabled
Import times for custom nodes:
0.0 seconds: /home/il/Downloads/ComfyUINew/custom_nodes/websocket_image_save.py
Context impl SQLiteImpl.
Will assume non-transactional DDL.
Context impl SQLiteImpl.
Will assume non-transactional DDL.
Running upgrade -> 0001_assets, Initial assets schema
Revision ID: 0001_assets
Revises: None
Create Date: 2025-12-10 00:00:00
Running upgrade 0001_assets -> 0002_merge_to_asset_references, Merge AssetInfo and AssetCacheState into unified asset_references table.
Running upgrade 0002_merge_to_asset_references -> 0003_add_metadata_job_id, Add system_metadata and job_id columns to asset_references.
Change preview_id FK from assets.id to asset_references.id.
Database upgraded from None to 0003_add_metadata_job_id
Using RAM pressure cache.
Starting server
To see the GUI go to: http://127.0.0.1:8188
got prompt
Using pytorch attention in VAE
Using pytorch attention in VAE
VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
Found quantization metadata version 1
Using MixedPrecisionOps for text encoder
CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
Requested to load WanTEModel
Model WanTEModel prepared for dynamic VRAM loading. 6419MB Staged. 0 patches attached. Force pre-loaded 73 weights: 488 KB.
Model WanTEModel prepared for dynamic VRAM loading. 6419MB Staged. 0 patches attached. Force pre-loaded 73 weights: 488 KB.
Requested to load WanVAE
Model WanVAE prepared for dynamic VRAM loading. 241MB Staged. 0 patches attached. Force pre-loaded 60 weights: 61 KB.
model weight dtype torch.float16, manual cast: None
model_type FLOW
Requested to load WAN21
Model WAN21 prepared for dynamic VRAM loading. 27251MB Staged. 0 patches attached. Force pre-loaded 160 weights: 1603 KB.
100%|█████████████████████████████████████████████| 2/2 [00:02<00:00, 1.32s/it]
model weight dtype torch.float16, manual cast: None
model_type FLOW
Requested to load WAN21
Model WAN21 prepared for dynamic VRAM loading. 27251MB Staged. 0 patches attached. Force pre-loaded 160 weights: 1603 KB.
100%|█████████████████████████████████████████████| 2/2 [00:02<00:00, 1.34s/it]
0 models unloaded.
Model WanVAE prepared for dynamic VRAM loading. 241MB Staged. 0 patches attached. Force pre-loaded 60 weights: 61 KB.
Prompt executed in 226.60 seconds
got prompt
Model WAN21 prepared for dynamic VRAM loading. 27251MB Staged. 0 patches attached. Force pre-loaded 160 weights: 1603 KB.
100%|█████████████████████████████████████████████| 2/2 [00:06<00:00, 3.03s/it]
Model WAN21 prepared for dynamic VRAM loading. 27251MB Staged. 0 patches attached. Force pre-loaded 160 weights: 1603 KB.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:07<00:00, 3.60s/it]
0 models unloaded.
Model WanVAE prepared for dynamic VRAM loading. 241MB Staged. 0 patches attached. Force pre-loaded 60 weights: 61 KB.
Prompt executed in 174.92 seconds
got prompt
Model WAN21 prepared for dynamic VRAM loading. 27251MB Staged. 0 patches attached. Force pre-loaded 160 weights: 1603 KB.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:02<00:00, 1.22s/it]
0 models unloaded.
Model WAN21 prepared for dynamic VRAM loading. 27251MB Staged. 0 patches attached. Force pre-loaded 160 weights: 1603 KB.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:02<00:00, 1.43s/it]
0 models unloaded.
Model WanVAE prepared for dynamic VRAM loading. 241MB Staged. 0 patches attached. Force pre-loaded 60 weights: 61 KB.
Prompt executed in 167.58 secondsOther
Might be related to #13978 but this is without GGUF or any other custom nodes, brand new install, etc etc.
PS: Also happens with FP8 model for both, though host mem usage according to nvtop now is only 2.5GB, down from about 16GB in bf16. Which also serves again as a highlight for the failure to cache to RAM.
Model WAN21 prepared for dynamic VRAM loading. 13627MB Staged. 0 patches attached. Force pre-loaded 160 weights: 1603 KB.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:04<00:00, 2.32s/it]
model weight dtype torch.float8_e4m3fn, manual cast: torch.float16
model_type FLOW
Requested to load WAN21
Model WAN21 prepared for dynamic VRAM loading. 13627MB Staged. 0 patches attached. Force pre-loaded 160 weights: 1603 KB.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:06<00:00, 3.18s/it]
0 models unloaded.
Model WanVAE prepared for dynamic VRAM loading. 241MB Staged. 0 patches attached. Force pre-loaded 60 weights: 61 KB.
Prompt executed in 256.09 seconds
got prompt
Model WAN21 prepared for dynamic VRAM loading. 13627MB Staged. 0 patches attached. Force pre-loaded 160 weights: 1603 KB.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:07<00:00, 3.64s/it]
Model WAN21 prepared for dynamic VRAM loading. 13627MB Staged. 0 patches attached. Force pre-loaded 160 weights: 1603 KB.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:04<00:00, 2.09s/it]
0 models unloaded.
Model WanVAE prepared for dynamic VRAM loading. 241MB Staged. 0 patches attached. Force pre-loaded 60 weights: 61 KB.
Prompt executed in 146.36 seconds
Activity
coderabbitai commentedon May 23, 2026
🧪 Issue enrichment is currently in open beta.
You can configure auto-planning by selecting labels in the issue_enrichment configuration.
To disable automatic issue enrichment, add the following to your
.coderabbit.yaml:💬 Have feedback or questions? Drop into our discord!
souravroy-dev commentedon May 23, 2026
having the same problem i am even seeing much less speed in ltx 2.3 .when i did the new update .22.0 it was working the moment i tried to update further like after 12 or 14 hours some more updates came in comfyui it broke something serious now ltx model takes much longer and also eating my ram like crazy and its continuosly moving model weight from my nvme .
BobJohnson24 commentedon May 23, 2026
Further testing has revealed to me that this issue only appears on my linux install (Linux Mint 22.2). It properly reserves and uses RAM under Windows.
BobJohnson24 commentedon May 23, 2026
Additional testing on linux:
It partially works in comfyui 0.21.1, just like reports on the GGUF issue proclaim. Though it does appear to reload the first wan model on the second run. By the third run both models are properly cached. Host Mem in nvtop reports 55GB used by Python, which is a lot more than the 3GB it uses in 0.22.0. Top claims 53,8g 53,2g for python resident and shared memory respectively.
However, considering 6419MB FP8 Text Encoder+13627MB Wan1 FP8 +13627 Wan2 FP8 +242 VAE = 33915MB, it is clear this might not be an optimal solution either.
Currently, at least windows is a lot closer on this same workflow with ~37.5GB of RAM reserved according to task manager.
Here is just a quick log from linux on comfy 0.21.1, showing it properly load from RAM instead of disk.
rattus128 commentedon May 24, 2026
OK so you are right on the edge of "it fits" with this setup.
I put an 8GB VRAM burner app in background to cut my 5090 to 24GB, BIOSed my NVME down to Gen1 (still quite a bit faster than yours) and booted my kernel to 64GB ram so its reasonably close to your system. This is Linux.
Im using your WF unchanged. I get that fast-3rd-time pattern - where it takes two runs to fully find RAM.
The long story:
So in this case the FP16 diffusion models do cause some pinning to happen.
Using the VRAM visualizer (https://github.com/kijai/ComfyUI-MemoryVisualization) we can see about 12GB will be pinned across the two models due to unsufficient VRAM to load the full full 26GB. Its able to load 20GB which is fair for this workflow.
What happens, is the pins themselves count as RAM consumption, but the touch of the disk to load the pins themselves causes a duplicate in the linux disk cache. Linux isn't smart enough to know to drop the pins in favor of the unpinned so the whole thing gets touched in disk cache first run.
12GiB pins + 6GiB misc RAM usage + 60GiB models, is much more than the 64GB (60GiB) RAM so you get that LRU disk cache eviction and the second run pulls at least the first model from disk again. On the second run, it doesnt have to pin, so the disk cache gets smarter, and only hits the bits it needs to. That (-12GiB) and we are not doing the TE second time around (-6GiB), so it fits now. So after the second load, the disk cache is exactly where it needs to be.
The good news is linux has an API (posix_fadvise) to de-prioritize that double copy that we dont need. When implemented, I don't get that second run slowness (-12GiB on the above math).
#14088
here is the end state RAM usage (where its able to just rerun 7.5s WFs:
Regarding FP8, the low RAM usage is fully expected and it should still be fast as the disk cache should cover everything. Nothing will pin.
Here is my FP8 performance:
Here is the RAM graph as it finishes up the first run. Notice that RAM is the low (5GB) just like you observe, but CHE is 33.6GB. CHE is where the models are and how it avoids disk the second time.
Thanks for the detailed report. Hope that change helps.
P.S. If that's a spinning HDD or a old SATA, you should consider even a small NVME as an in-between cache for your frequently-used models. Even a 512GB gen3 drive you put your favorites in is probably a huge QoL boost for you. Please also ensure swap is completely disabled with your disk being so slow. Any sort of pre-emptive swapping (which linux tends to do) will cripple your performance.
mjforgame commentedon May 24, 2026
I don't think it a good idea that comfyui does not cache any models to RAM.
I have 16G VRAM + 64GB RAM, in Win10.
I use wan2.2 fp8 models. The offical Wan 2.2 workflow uses much less RAM than the offical ltx 2.3 workflow. In previous version such as comfyui 0.21.0, the offical Wan 2.2 workflow uses about 45GB RAM when cache model to RAM. And in the lastest comfyui, the windows system can still cache the models to RAM even comfyui does not cache the models. So there is no need to load from disk again on second run in my computer.
However, the offical ltx 2.3 workflow uses much RAM. I have reported the behavior in #13978. In the lastest comfyui, it uses about 30GB RAM when it does not cache any models. But the gemma 3 textcoder + ltx 2.3 fp8 models ≈ 10GB + 27GB = 37GB, and sometimes I also need some loras. And 30GB + 37GB + (loras size) > 64GB, so the windows system can't cache the models to RAM. It would read the models from disk again and again on each run.
But in previous version such as comfyui 0.21.0, everything works fine. It uses about from 50GB to 60GB RAM to run the offical ltx2.3 workflow, and no need to read from disk again on second run. Even I add a lot of loras, it still works well, the RAM usage does not change, it only read from disk once time.
BobJohnson24 commentedon May 24, 2026
I see, so the main issue with the FP16 models loaded to FP8 is that this still creates an FP16 disk cache, which will get destroyed due to RAM limits. With pre-converted FP8 models it does seem better.
I would love to have a fast NVME but Linux ntfs support says "no" to fast and I am not going to waste an nvme on linux file systems on a dual boot machine.
FrozenAshRfg commentedon May 24, 2026
Are you using ntfs-3g to mount your NTFS disk? If so, have you considered trying the ntfs3 kernel driver instead? In my experience, switching significantly improves performance in ComfyUI.
For reference, here are some comparative logs from ComfyUI running the same workflow:
1. baseline: comfyui v0.22.0(before #13802 merged) aimdo 0.4.2
first load:
second load:
2. comfyui origin/prs/aimdo-045-cold-linux-pins aimdo v0.4.5 with ntfs-3g
first load:
second load:
3. comfyui origin/prs/aimdo-045-cold-linux-pins aimdo v0.4.5 with ntfs3
first load:
second load:
One minor note though: under origin/prs/aimdo-045-cold-linux-pins (aimdo v0.4.5), the per-step timing in the progress bar seems to be reporting incorrectly — the displayed it/s doesn't always match the actual elapsed time.
rattus128 commentedon May 24, 2026
Yep. Type changes are done on the GPU. VRAM consumption matches the model type while RAM matches the files
Yeah I get 6 GB/s loads with ntfs3 when I have my disk at full speed. The stats I posted here today are ntfs3 (I dual boot everything too as you could imagine). I would investigate ntfs3 as it is a total game changer for dual booters.
rattus128 commentedon May 24, 2026
Interesting numbers @TxLysy. Can I get you system specs? It's close to worth a startup warning if it's that bad as ntfs dual boot is going to be common.
FrozenAshRfg commentedon May 24, 2026
Thanks for your interest and the prompt reply, @rattus128.
Here are my system specs for reference. That said, more testing may be needed to draw a solid conclusion:
System Info:
Kingston KC3000 2T PCIE 4.0*4NTFS Driver Version:
Other Info:
Hope this information helps.
BobJohnson24 commentedon May 24, 2026
With some more testing, the second run slowness with partial disk read I initially saw seems to be somewhat reliant on shitty memory management on the OS level. Once I have run it a few times, then cleared disk cache by filling up ram with random trash, it seems that it disappears on 0.22.0 with aimdo 0.4.3
I have tried ntfs3 instead of ntfs3g and sadly it is broken with mint, and the mint devs have deemed to blame the driver and black list it instead of trying to fox it. (It was working fine on arch btw...)
BobJohnson24 commentedon May 24, 2026
@rattus128 Actually, I came across another similar bug in 0.22.0. It seems that non-aimdo weights (for example a dit using torch compile with disable_dynamic_vram enabled via Kijais torch compile advanced or just loading a GGUF) get removed from VRAM immediately after being used, as seen here by the vram fluctuations.
Static weight missing after inference:
BobJohnson24 commentedon May 24, 2026
PS it still happens with aimdo 0.4.5 and + the reference PR, though I know it wasn't made to address this.
Ph0rk0z commentedon May 25, 2026
Indeed.. I assume mr rattus is going to give me a stipend for NVME SSD because my loading speed is 150MB/s. For the whole several terrabytes of models too, right? I am still fine at v.22 if I simply disable dynamic vram.
Post 5aa5ccc dynamic vram cannot really be disabled because it still evicts models from ram, especially GGUF. I don't think there is a combination of flags to return the old behavior.
Meanwhile my 384g of RAM (the "nvme" ssd I should buy) sits fallow.
brendanhoar commentedon May 25, 2026
Yeah, the 512GB of RAM in this 64-core threadripper Windows system w/ a 4090 is now much less used, but model loads are taking forever off of a combination of nvme and spinning disks. I'm not sure why this is better? T2I2U2V (U = upscaler) workflows keep spending a couple minutes at the beginning of the LTX 2.3 portion doing...nothing? This is via SwarmUI with ComfyUI as the backend.
Don't have current logs, but in a simpler workflow I was seeing this on every run, a smaller > 1 minute delay (might have been NVME vs HDD above) each time LTX was utilized in the workflow (not just on the first use in a series of generations, but on each one):
17:03:19.669 [Debug] [ComfyUI-0/STDERR] 0%| | 0/8 [00:00<?, ?it/s]
17:04:40.485 [Debug] [ComfyUI-0/STDERR] 0%| | 0/8 [00:00<?, ?it/s, Model Initializing ... ]
Ph0rk0z commentedon May 25, 2026
I am only using simple workflows too.. Am scared to see what happens with more complex stuff like https://github.com/komikndr/raylight or when multiple models swap. Where FP8 weights need compilation and the copy has to stay in ram. All that good stuff comfy was known for. Now the only official use case is API and consumer PC with 3060/16g of ram.
Its hard to retest everything at every update so that 2 weeks later you don't find your speeds cut in half or new model broke support. The repo is becoming anxiety inducing to git pull this year. PRs merged with no comments.
999purple999 commentedon May 26, 2026
Hi @rattus128 read your #13802 (threaded loader) and #14088 (cold pin marking) first; the perf gains on cold loads are clean, and the third-run-fast pattern you reproduced (76s/49s/8s) lines up with OS page-cache warming.
Want to put a different angle on the table in case it's complementary - tracing from the Python ref-management side rather than IO throughput.
In
comfy/model_management.py,free_memory()pops the LoadedModel out ofcurrent_loaded_models(pop(i)around line ~706), the caller inload_models_gpu()discards the returned list, andmodel_unload()itself doesself.model.detach(...); self.real_model = None. That drops the only Python-side handle to the patcher and weights - CPU-side weights become eligible for GC.If GC fires before the next prompt re-requests the same model, you take the cold-disk path (now fast thanks to your threaded loader, but still cold). If GC delays a cycle, the page cache catches it - 8s instead of 49s. That'd partially explain the non-deterministic third-time-fast pattern from the Python side, not only from OS page cache behaviour. Possibly two effects compounding.
Full write-up with the exact lines and a sketch of where I'd put a ref-retaining cache:
https://github.com/999purple999/ComfyUI/blob/proposal/14076-ram-model-cache/PROPOSAL_ISSUE_14076.md
(The doc predates your
--cache-ramwork so the flag name in there is wrong, ignore that bit.)One question before I send any code: would you rather have a ref-retaining cache fit into the
--cache-ramstory you already shipped (extend that flag to retain Python handles), or as a separate composing flag? I don't want to add a knob you'd then need to deprecate. Happy to send a PR within 2-3 days of alignment, against eithermasteror yourdev/threaded-loaderbranch, whichever shape works.If the ref-drop diagnosis is actually already covered by something on the way (or I'm reading the lifetime wrong), say the word and I'll revise.
ioritree commentedon May 27, 2026
Psylenceo commentedon May 30, 2026
I've had this issue as well, except I'm on a 4090 and the only solution I've found for myself personally, was to revert my driver from 595.xx.yy to 590.48.01.
with the 590 driver I get <200sec gens, with the 595 driver 30minutes later and the workflow still has not made it to the sampler.
In case it matters, System Info:
Other Info: