feat: Support MiniMax-H3 (CORE-375) - #15224
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds MiniMax H3 support for joint audio-video generation. The changes add video and audio latent formats and VAEs, a packed audio-video diffusion transformer, Qwen3-VL-32B conditioning, checkpoint detection, model registration, quantized linear scaling, and workflow nodes for empty latents, keyframes, references, and sigma shifts. AV latent node descriptions include MiniMax H3 compatibility. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 7
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@comfy_extras/nodes_minimax_h3.py`:
- Around line 240-246: Update the frame-count handling around the loop in the
reference-video processing path to explicitly reject clips shorter than 5 frames
with a concise error, unless the model contract confirms off-grid lengths are
supported. Ensure validation occurs before deriving the packed latent dimensions
from frames.shape[0], while preserving the existing 17k + 5 adjustment for valid
clips.
In `@comfy/ldm/minimax/audio_vae.py`:
- Around line 234-243: Initialize the registered buffer zero_k_bias with
torch.zeros instead of uninitialized memory, including the constructor and any
checkpoint-conversion path that creates it; alternatively mark it non-persistent
while still zero-initializing it. Ensure the QKV bias concatenation in forward
always receives a zero K bias when checkpoints omit pre_block.attn.zero_k_bias.
- Around line 426-442: The MiniMax H3 audio VAE has inconsistent stereo waveform
layouts. Update MiniMaxH3AudioVAE.encode and its docstring in
comfy/ldm/minimax/audio_vae.py to consistently accept [B, 2, L], and update the
wrapper call in comfy_extras/nodes_minimax_h3.py (lines 198-206) so
_encode_ref_audio passes that layout instead of [B, L, 1].
In `@comfy/ldm/minimax/model.py`:
- Around line 489-515: Cache the augmented condition rows for each sampling run
instead of regenerating them inside _cond_video_rows and _cond_audio_rows on
every _forward step. Initialize or invalidate the cache when the payload/seed
changes, preserve the existing augmentation and deterministic seeding behavior,
and reuse the device-resident rows during subsequent steps.
- Around line 457-487: Remove the model-owned _layout_cache initialization and
caching in _layout; make PackedLayout temporary to each top-level execution
instead. Pass a per-call layout cache through transformer_options, or rebuild
the layout per forward, and update the relevant callers so _layout receives and
reuses that cache without storing tensor-bearing layouts on the model instance.
- Line 651: Update the unpatchify call in the surrounding model forward path to
derive all grid dimensions from self.patch_size: divide latent_t by its temporal
patch dimension and lat_h/lat_w by their corresponding spatial patch dimensions.
Remove the hardcoded factors so unpatchify_video receives dimensions consistent
with the configured patch size.
In `@comfy/ops.py`:
- Around line 946-1000: Update linear_input_act to detect a non-None
linear.Prequant_scale before invoking quant_ops.ck.int8_linear and route that
case through the existing eager activation plus regular linear path. Keep the
fused INT8 path unchanged when Prequant_scale is absent.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: b4007b57-4d1a-4f1c-8058-99f5be786e10
📒 Files selected for processing (16)
comfy/latent_formats.pycomfy/ldm/minimax/audio_vae.pycomfy/ldm/minimax/model.pycomfy/ldm/minimax/vae.pycomfy/model_base.pycomfy/model_detection.pycomfy/ops.pycomfy/quant_ops.pycomfy/sd.pycomfy/supported_models.pycomfy/text_encoders/llama.pycomfy/text_encoders/minimax.pycomfy/text_encoders/qwen3vl.pycomfy_extras/nodes_lt.pycomfy_extras/nodes_minimax_h3.pynodes.py
📜 Review details
🧰 Additional context used
📓 Path-based instructions (7)
**/*
📄 CodeRabbit inference engine (AGENTS.md)
**/*: Keep changes and file scope as small and direct as possible; prefer practical fixes, minimal dependencies, existing patterns, and removal of obsolete code.
Preserve existing APIs, node names, model-loading behavior, file layout, and workflow compatibility unless replacement is intentional.
Do not add core ComfyUI code that makes outbound internet requests, including telemetry, analytics, tracking, reporting, update checks, remote configuration, licensing checks, or background network activity. User-authorized model downloads are permitted only for the requested artifact and without telemetry.
Keep state and capability flags on the object that owns the behavior; prefer explicit parent-owned attributes over probing child objects withgetattrfor parent control flow.
Preserve shared method signatures, argument conventions, return types, side effects, and error behavior unless the shared contract and all affected callers are intentionally updated.
Files:
comfy/quant_ops.pycomfy_extras/nodes_lt.pycomfy/text_encoders/llama.pycomfy/model_detection.pycomfy/supported_models.pycomfy/latent_formats.pynodes.pycomfy/model_base.pycomfy/sd.pycomfy_extras/nodes_minimax_h3.pycomfy/ops.pycomfy/text_encoders/qwen3vl.pycomfy/ldm/minimax/audio_vae.pycomfy/ldm/minimax/vae.pycomfy/text_encoders/minimax.pycomfy/ldm/minimax/model.py
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Do not addtorch.no_grad,torch.inference_mode, inference-mode wrappers, or explicit model freeze/trainability toggles; only disable globally enabled inference mode when a training path needs gradients.
Remove inference-only training behavior such as dropout while preserving checkpoint and state-dict compatibility; usenn.Identitywhen removing a module would alter keys or ordering.
Keep imports at module scope except for established optional-backend probes or import-cycle avoidance; avoid unnecessary exception handling and use specific exception types with useful fallbacks.
Do not add code for unsupported pinned library versions or obsolete PyTorch workarounds; unsupported formats, invalid quantization metadata, and bad states should fail clearly rather than silently degrading output.
Match local Python style, keep comments sparse and useful, and remove comments that merely restate obvious code.
Treat dtype, device placement, VRAM usage, and offloading as correctness concerns; use existing ComfyUI cast, offload, cleanup, quantization, and memory helpers.
Model implementations must use an existing optimized Comfy Kitchen, ComfyUI, quantization, or backend operation when it supports the required math, layout, dtype, device, memory, and interface contracts; inspect available operations before writing local kernels.
Retain local or differentiable fallbacks only when no optimized operation satisfies the required math or patch/autograd contract; adapt inputs to shared operation layouts while preserving exact model behavior.
Treat optimized attention and similar backend-selected callables as opaque; callers must rely on documented interfaces and result contracts rather than function identity, names, modules, or implementation details.
Do not duplicate existing inference operations with custom float32-upcasting implementations, such as custom RMSNorm variants; use generic ComfyUI or native torch operations.
If a model constructor has anoperationsparame...
Files:
comfy/quant_ops.pycomfy_extras/nodes_lt.pycomfy/text_encoders/llama.pycomfy/model_detection.pycomfy/supported_models.pycomfy/latent_formats.pynodes.pycomfy/model_base.pycomfy/sd.pycomfy_extras/nodes_minimax_h3.pycomfy/ops.pycomfy/text_encoders/qwen3vl.pycomfy/ldm/minimax/audio_vae.pycomfy/ldm/minimax/vae.pycomfy/text_encoders/minimax.pycomfy/ldm/minimax/model.py
**
⚙️ CodeRabbit configuration file
**: IMPORTANT: Only comment on issues directly introduced by this PR's code changes.
Treat AGENTS.md as mandatory repository policy, not optional style guidance.
Flag PR changes that violate AGENTS.md even when the code is otherwise functional.
In particular, enforce architecture boundaries, dtype/device/memory rules,
interface contracts, import style, no unnecessary try/except blocks, no inline
imports, no outbound internet paths in core ComfyUI, and narrow scoped fixes.
Prefer direct findings over suggestions when a rule is violated. Only ignore
AGENTS.md when it clearly conflicts with a newer explicit maintainer instruction
in the PR.
Do NOT flag pre-existing issues in code that was merely moved, re-indented,
de-indented, or reformatted without logic changes. If code appears in the diff
only due to whitespace or structural reformatting (e.g., removing awith:block),
treat it as unchanged. Contributors should not feel obligated to address
pre-existing issues outside the scope of their contribution.
Files:
comfy/quant_ops.pycomfy_extras/nodes_lt.pycomfy/text_encoders/llama.pycomfy/model_detection.pycomfy/supported_models.pycomfy/latent_formats.pynodes.pycomfy/model_base.pycomfy/sd.pycomfy_extras/nodes_minimax_h3.pycomfy/ops.pycomfy/text_encoders/qwen3vl.pycomfy/ldm/minimax/audio_vae.pycomfy/ldm/minimax/vae.pycomfy/text_encoders/minimax.pycomfy/ldm/minimax/model.py
comfy/**
⚙️ CodeRabbit configuration file
comfy/**: Core ML/diffusion engine. Focus on:
- Backward compatibility (breaking changes affect all custom nodes)
- Memory management and GPU resource handling
- Performance implications in hot paths
- Thread safety for concurrent execution
Files:
comfy/quant_ops.pycomfy/text_encoders/llama.pycomfy/model_detection.pycomfy/supported_models.pycomfy/latent_formats.pycomfy/model_base.pycomfy/sd.pycomfy/ops.pycomfy/text_encoders/qwen3vl.pycomfy/ldm/minimax/audio_vae.pycomfy/ldm/minimax/vae.pycomfy/text_encoders/minimax.pycomfy/ldm/minimax/model.py
**/*node*.py
📄 CodeRabbit inference engine (AGENTS.md)
Keep node changes backward compatible by default, use sensible defaults for new inputs, avoid output-type changes, expose only inputs and outputs the node actually reads or owns, and avoid pass-through or workflow-shaping sockets.
Files:
comfy_extras/nodes_lt.pynodes.pycomfy_extras/nodes_minimax_h3.py
comfy_extras/**
⚙️ CodeRabbit configuration file
comfy_extras/**: Community-contributed extra nodes. Focus on:
- Consistency with node patterns (INPUT_TYPES, RETURN_TYPES, FUNCTION, CATEGORY)
- No breaking changes to existing node interfaces
Files:
comfy_extras/nodes_lt.pycomfy_extras/nodes_minimax_h3.py
nodes.py
⚙️ CodeRabbit configuration file
nodes.py: Core node definitions (2500+ lines). Focus on:
- Backward compatibility of NODE_CLASS_MAPPINGS
- Consistency of INPUT_TYPES return format
Files:
nodes.py
🧠 Learnings (9)
📚 Learning: 2026-02-21T14:01:41.482Z
Learnt from: pythongosssss
Repo: Comfy-Org/ComfyUI PR: 12555
File: comfy_extras/nodes_glsl.py:719-724
Timestamp: 2026-02-21T14:01:41.482Z
Learning: In PyOpenGL, bare Python scalars can be accepted for 1-element array parameters by NumberHandler. This means you can pass an int/float directly to OpenGL texture deletion (e.g., glDeleteTextures(tex)) without wrapping in a list. Verify function-specific expectations and ensure types match what the OpenGL call expects; use explicit lists only when the API requires an array.
Applied to files:
comfy/quant_ops.pycomfy_extras/nodes_lt.pycomfy/text_encoders/llama.pycomfy/model_detection.pycomfy/supported_models.pycomfy/latent_formats.pynodes.pycomfy/model_base.pycomfy/sd.pycomfy_extras/nodes_minimax_h3.pycomfy/ops.pycomfy/text_encoders/qwen3vl.pycomfy/ldm/minimax/audio_vae.pycomfy/ldm/minimax/vae.pycomfy/text_encoders/minimax.pycomfy/ldm/minimax/model.py
📚 Learning: 2026-05-13T12:31:45.069Z
Learnt from: rattus128
Repo: Comfy-Org/ComfyUI PR: 13802
File: comfy/pinned_memory.py:19-30
Timestamp: 2026-05-13T12:31:45.069Z
Learning: When reviewing code that uses comfy/pinned_memory.py’s `HostBuffer.extend(size=..., reallocate=...)`: by default (`reallocate` is not True / False), `extend(size=...)` is a *relative increment* that grows the buffer by `size` bytes—so slicing like `[offset:offset+size]` after `hostbuf.extend(size=size)` is correct and the argument should not be rewritten to `offset + size`. Only in the single-segment reallocation mode (`reallocate=True`, e.g., as used by `resize_pin_buffer()` in `comfy/model_management.py`) should `size` be treated as an *absolute target* and the call/arguments should be checked accordingly.
Applied to files:
comfy/quant_ops.pycomfy/text_encoders/llama.pycomfy/model_detection.pycomfy/supported_models.pycomfy/latent_formats.pycomfy/model_base.pycomfy/sd.pycomfy/ops.pycomfy/text_encoders/qwen3vl.pycomfy/ldm/minimax/audio_vae.pycomfy/ldm/minimax/vae.pycomfy/text_encoders/minimax.pycomfy/ldm/minimax/model.py
📚 Learning: 2026-03-04T14:05:31.426Z
Learnt from: jtydhr88
Repo: Comfy-Org/ComfyUI PR: 12757
File: comfy_extras/nodes_custom_sampler.py:1069-1089
Timestamp: 2026-03-04T14:05:31.426Z
Learning: In the ComfyUI sampling pipeline, treat percent_to_sigma(0.0) as a sentinel value (999999999.9) that means starting from pure noise. This is consistent with BasicScheduler via calculate_sigmas. The SamplingPercentToSigma node’s return_actual_sigma flag differentiates this sentinel from sigma_max. Reviewers should not flag CurveToSigmas or similar nodes that rely on percent_to_sigma as bugs; downstream samplers are expected to handle the sentinel correctly. When reviewing related sampling-related code, assume this sentinel semantics unless there is explicit handling for a real sigma_max.
Applied to files:
comfy_extras/nodes_lt.pycomfy_extras/nodes_minimax_h3.py
📚 Learning: 2026-04-04T13:29:15.653Z
Learnt from: kijai
Repo: Comfy-Org/ComfyUI PR: 13258
File: comfy_extras/nodes_frame_interpolation.py:151-189
Timestamp: 2026-04-04T13:29:15.653Z
Learning: In this ComfyUI codebase, node `execute()` inference is already run under a global `torch.inference_mode()` context established in the execution engine (e.g., `execution.py` around line ~732). During review, avoid recommending changes that wrap node inference loops in `torch.inference_mode()`—it is already applied, so such suggestions are likely redundant.
Applied to files:
comfy_extras/nodes_lt.pycomfy_extras/nodes_minimax_h3.py
📚 Learning: 2026-05-09T18:40:40.199Z
Learnt from: kijai
Repo: Comfy-Org/ComfyUI PR: 13813
File: comfy_extras/nodes_wandancer.py:868-872
Timestamp: 2026-05-09T18:40:40.199Z
Learning: When building video/temporal decoding nodes that call ComfyUI’s VAE.decode (comfy/sd.py), leverage VAE.decode’s existing VRAM-aware chunking along dim 0. Reshape or transpose the latent so the temporal dimension T is folded into dim 0 (e.g., transform a latent of shape [B, T, C, H, W] into [B*T, C, H, W] before calling vae.decode). This lets VAE.decode do chunked decoding without needing an explicit per-frame loop inside the node itself.
Applied to files:
comfy_extras/nodes_lt.pycomfy_extras/nodes_minimax_h3.py
📚 Learning: 2026-05-20T00:10:14.673Z
Learnt from: Pauan
Repo: Comfy-Org/ComfyUI PR: 13997
File: comfy_extras/nodes_string.py:12-25
Timestamp: 2026-05-20T00:10:14.673Z
Learning: In the ComfyUI `comfy_extras/` codebase, some nodes intentionally ship with a default input string that references parameters that may not yet be connected. If the default would raise a `KeyError` (e.g., examples like `MathExpression` default `a + b`, or `StringFormat` default `{a}` with `min=0` and autogrow inputs), treat it as an intentional “hint default” UX pattern, not a bug. During review, do not flag this behavior or recommend changing `min` to `1` or altering the default to an empty string solely to avoid the `KeyError`.
Applied to files:
comfy_extras/nodes_lt.pycomfy_extras/nodes_minimax_h3.py
📚 Learning: 2026-07-26T18:37:44.213Z
Learnt from: jtydhr88
Repo: Comfy-Org/ComfyUI PR: 15090
File: comfy_extras/nodes_video.py:230-235
Timestamp: 2026-07-26T18:37:44.213Z
Learning: In ComfyUI node implementations under `comfy_extras`, do not add regular widget/prompt inputs to `fingerprint_inputs` if they are already included in the node cache signature via `comfy_execution/caching.py:get_immediate_node_signature` (it records every non-link prompt input as `(key, inputs[key])`). Reserve `fingerprint_inputs` only for out-of-band state that can change without changing the prompt inputs (e.g., the selected source file’s modification time). For example, inputs like `LoadVideo.edit` should not be redundantly added to `fingerprint_inputs`; use it only for things not represented in prompt inputs.
Applied to files:
comfy_extras/nodes_lt.pycomfy_extras/nodes_minimax_h3.py
📚 Learning: 2026-04-23T13:22:31.631Z
Learnt from: kijai
Repo: Comfy-Org/ComfyUI PR: 13531
File: comfy_extras/nodes_lt.py:715-722
Timestamp: 2026-04-23T13:22:31.631Z
Learning: In the ComfyUI LTXV audio pipeline, when preparing waveforms for `AudioVAE.encode()`, resample inputs to the AudioVAE public interface sample rate: `vae_sample_rate = getattr(audio_vae, "audio_sample_rate", 44100)`. Do not resample to `first_stage_model.sample_rate` (often ~16000 Hz), since that is the VAE’s internal mel-spectrogram rate and is handled inside the VAE. Ensure the resampling/`VAEEncodeAudio.execute()` path uses `vae_sample_rate` to match `AudioVAE.encode()` expectations.
Applied to files:
comfy_extras/nodes_lt.py
📚 Learning: 2026-05-04T18:30:37.579Z
Learnt from: Talmaj
Repo: Comfy-Org/ComfyUI PR: 13655
File: comfy/model_detection.py:907-917
Timestamp: 2026-05-04T18:30:37.579Z
Learning: In ComfyUI’s internal supported model implementations (comfy/supported_models_base.py and comfy/supported_models/*.py), ensure model classes do not override matches() in their own class bodies. All supported models should use BASE.matches() for backward compatibility; if a future change introduces a matches() override in a subclass, treat it as a backward-compatibility risk and require additional review/testing to confirm behavior remains consistent with BASE.matches().
Applied to files:
comfy/supported_models.py
🪛 ast-grep (0.45.0)
comfy/text_encoders/minimax.py
[warning] 90-98: Do not use an empty list as a default parameter
Context: def forward(self, input_ids, attention_mask=None, embeds=None, num_tokens=None,
intermediate_output=None, final_layer_norm_intermediate=True,
dtype=None, embeds_info=[], **kwargs):
seq = embeds.shape[1] if embeds is not None else input_ids.shape[1]
self.last_token_tags = token_tags_from_embeds_info(seq, embeds_info)
return super().forward(input_ids, attention_mask=attention_mask, embeds=embeds,
num_tokens=num_tokens, intermediate_output=intermediate_output,
final_layer_norm_intermediate=final_layer_norm_intermediate,
dtype=dtype, embeds_info=embeds_info, **kwargs)
Note: [CWE-710] Improper Adherence to Coding Standards (mutable default argument).
(no-empty-list-as-parameter)
[warning] 140-185: Do not use an empty list as a default parameter
Context: def tokenize_with_weights(self, text, return_word_ids=False, images=[],
minimax_ref_items=None, **kwargs):
entries = []
def add_text(s):
entries.extend((tid, 1.0) for tid in self._text_ids(s))
def add_vision(data, video_block=False):
entries.append((VISION_START, 1.0))
entries.append((self._vision_entry(data, video_block), 1.0))
entries.append((VISION_END, 1.0))
if minimax_ref_items:
counters = {"image": 0, "audio": 0, "video": 0}
for item in minimax_ref_items:
kind = item["type"]
counters[kind] += 1
if kind == "image":
add_text("<Picture %d>: " % counters["image"])
add_vision(item["data"])
elif kind == "audio":
add_text("<Audio %d>: " % counters["audio"])
elif kind == "video":
frames = item["data"] # [T, H, W, C], sampled at 2 fps
timestamps = item.get("timestamps")
if timestamps is None:
timestamps = [i / 2.0 for i in range(frames.shape[0])]
if frames.shape[0] % 2 == 1: # repeat-pad to temporal patch of 2
frames = torch.cat([frames, frames[-1:]], dim=0)
timestamps = list(timestamps) + [timestamps[-1]]
add_text("<Video %d>: " % counters["video"])
for i in range(0, frames.shape[0], 2):
block_ts = (timestamps[i] + timestamps[i + 1]) / 2.0
add_text("<%.1f seconds>" % block_ts)
add_vision(frames[i:i + 2], video_block=True)
else:
for i, img in enumerate(images):
add_text("<Picture %d>: " % (i + 1))
add_vision(img)
add_text(text)
if len(entries) == 0:
entries.append((151643, 1.0))
if return_word_ids:
entries = [t + (0,) for t in entries]
return {"qwen3vl_32b": [entries]}
Note: [CWE-710] Improper Adherence to Coding Standards (mutable default argument).
(no-empty-list-as-parameter)
🪛 OpenGrep (1.26.0)
comfy/ldm/minimax/vae.py
[ERROR] 21-21: Possible credit card number (PAN) detected in source code. Credit card numbers should never be hardcoded or stored in source files. Use a secrets manager or tokenization service instead.
(coderabbit.pii.credit-card-number)
[ERROR] 22-22: Possible credit card number (PAN) detected in source code. Credit card numbers should never be hardcoded or stored in source files. Use a secrets manager or tokenization service instead.
(coderabbit.pii.credit-card-number)
[ERROR] 23-23: Possible credit card number (PAN) detected in source code. Credit card numbers should never be hardcoded or stored in source files. Use a secrets manager or tokenization service instead.
(coderabbit.pii.credit-card-number)
[ERROR] 23-23: Possible credit card number (PAN) detected in source code. Credit card numbers should never be hardcoded or stored in source files. Use a secrets manager or tokenization service instead.
(coderabbit.pii.credit-card-number)
[ERROR] 25-25: Possible credit card number (PAN) detected in source code. Credit card numbers should never be hardcoded or stored in source files. Use a secrets manager or tokenization service instead.
(coderabbit.pii.credit-card-number)
[ERROR] 29-29: Possible credit card number (PAN) detected in source code. Credit card numbers should never be hardcoded or stored in source files. Use a secrets manager or tokenization service instead.
(coderabbit.pii.credit-card-number)
🔇 Additional comments (21)
comfy/ops.py (1)
1310-1310: Persistence and application ofpre_quant_scaleare consistent.
state_dictnow persistspre_quant_scaleviaextra_quant_params, andforward()applies it withcomfy.model_management.cast_to_devicebefore shape handling. This matches the existingload_extra_paramsloading path in_load_quantized_module, which registers any parameter listed in the format'sQUANT_ALGOS[...]["parameters"]set. See the related comment on Lines 946-1000 for the one gap where this scaling is bypassed.Also applies to: 1349-1353
comfy/quant_ops.py (1)
230-247: LGTM!comfy/latent_formats.py (1)
570-608: LGTM!comfy/text_encoders/llama.py (1)
267-277: LGTM!comfy/text_encoders/qwen3vl.py (1)
11-22: LGTM!comfy/sd.py (2)
75-77: LGTM!Also applies to: 942-955, 1428-1428, 1485-1485, 1552-1554, 1784-1786
956-970: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winAdd
self.disable_offload = Trueto the MiniMax H3 audio VAE, matching sibling audio VAEs.This block sets
latent_dim = 2but does not setdisable_offload. Every otherlatent_dim == 2audio VAE in this function (LTX Audio, ACE-Step Audio) setsself.disable_offload = True.Without it, an OOM during encode or decode falls back to the generic
decode_tiled_/encode_tiled_path. That path tiles overshape[2]/shape[3]as spatial height/width. For this VAE's[B, 32, 2, T]latent layout, those are the stereo-channel dimension (2) and the time dimension (T), not spatial dimensions. Tiling them this way produces incorrect output instead of a controlled failure, matching a known failure class already reported for other audio VAEs in this codebase when the tiled fallback is hit.🛡️ Proposed fix
self.latent_dim = 2 # [B, 32, stereo 2, T] self.process_output = lambda audio: audio self.process_input = lambda audio: audio self.working_dtypes = [torch.float32] + self.disable_offload = True # encode gets the waveform shape [B, 2, samples], decode the latent shape [B, 32, 2, T]> Likely an incorrect or invalid review comment.comfy/model_detection.py (1)
362-391: LGTM!comfy/supported_models.py (1)
18-18: LGTM!Also applies to: 959-985, 2438-2438
comfy/text_encoders/minimax.py (1)
91-116: 🩺 Stability & AvailabilityNo change needed for
last_token_tags.
SDClipModel.encode_token_weightscallsencode()once perencode_token_weights()invocation, and the MiniMax H3 tokenizer produces one batch item[entries]per prompt, so this side-channel state does not create stale or batch-mismatched tags in the current path.> Likely an incorrect or invalid review comment.comfy/ldm/minimax/vae.py (3)
19-35: Static analysis PII hits are false positives.OpenGrep reports credit-card numbers on the
LATENTS_MEANandLATENTS_STDlines. These are model normalization constants. No action is needed.Source: Linters/SAST tools
168-318: LGTM!Also applies to: 400-518, 654-694
46-55: 🩺 Stability & AvailabilityNo change needed.
comfy.ops.disable_weight_init.Conv3d.forwardacceptsautopadand handlescause_zero.comfy/ldm/minimax/audio_vae.py (1)
24-27: LGTM!Also applies to: 55-118, 152-208, 270-368
comfy/ldm/minimax/model.py (1)
35-118: LGTM!Also applies to: 120-294, 297-412, 517-649
comfy/model_base.py (2)
24-24: LGTM!Also applies to: 2067-2110
2104-2104: 🎯 Functional CorrectnessNo change needed:
seedreaches the MiniMax H3 payload.
ComfyModel.cfg_guider.inner_samplecallsprocess_conds(..., seed=seed)before sampling, andprocess_condsforwards each cond’sseedintoencode_model_conds(...).comfy_extras/nodes_minimax_h3.py (1)
33-76: LGTM!Also applies to: 79-151, 276-330
comfy_extras/nodes_lt.py (1)
749-750: LGTM!Also applies to: 786-788
nodes.py (2)
2439-2439: LGTM!
995-995: 🎯 Functional CorrectnessNo change needed.
CLIPType.MINIMAXis already defined and has a MiniMax text-encoder branch.
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
comfy/sd.py (1)
960-974: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winRoute the audio VAE through 1-D tiling.
At Line 968,
latent_dim = 2describes a stereo/time latent, but this block does not setextra_1d_channel = 2. During OOM fallback, the generic VAE path therefore treats the stereo and time axes as 2-D spatial axes instead of usingencode_tiled_1danddecode_tiled_1d. This can produce invalid shapes or fail. (raw.githubusercontent.com)Set
extra_1d_channel = 2, or provide model-owned 1-D tiling.As per path instructions,
comfy/**code must treat memory management and GPU resource handling as correctness concerns.Proposed fix
self.latent_dim = 2 # [B, 32, stereo 2, T] + self.extra_1d_channel = 2🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@comfy/sd.py` around lines 960 - 974, Set extra_1d_channel to 2 in the MiniMax H3 audio VAE initialization block alongside latent_dim = 2, so generic OOM fallback routes stereo/time latents through encode_tiled_1d and decode_tiled_1d while preserving the existing model configuration.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@comfy/sd.py`:
- Around line 960-974: Set extra_1d_channel to 2 in the MiniMax H3 audio VAE
initialization block alongside latent_dim = 2, so generic OOM fallback routes
stereo/time latents through encode_tiled_1d and decode_tiled_1d while preserving
the existing model configuration.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 86e592fa-bf12-4410-b787-f08eadc2efb9
📒 Files selected for processing (1)
comfy/sd.py
📜 Review details
⏰ Context from checks skipped due to timeout. (8)
- GitHub Check: test (ubuntu-latest)
- GitHub Check: test (windows-2022)
- GitHub Check: Run Pylint
- GitHub Check: test (ubuntu-latest)
- GitHub Check: test (macos-latest)
- GitHub Check: test (macos-latest)
- GitHub Check: test
- GitHub Check: test (windows-latest)
🧰 Additional context used
📓 Path-based instructions (4)
**/*
📄 CodeRabbit inference engine (AGENTS.md)
**/*: Keep changes and file scope as small and direct as possible; prefer practical fixes, minimal dependencies, existing patterns, and removal of obsolete code.
Preserve existing APIs, node names, model-loading behavior, file layout, and workflow compatibility unless replacement is intentional.
Do not add core ComfyUI code that makes outbound internet requests, including telemetry, analytics, tracking, reporting, update checks, remote configuration, licensing checks, or background network activity. User-authorized model downloads are permitted only for the requested artifact and without telemetry.
Keep state and capability flags on the object that owns the behavior; prefer explicit parent-owned attributes over probing child objects withgetattrfor parent control flow.
Preserve shared method signatures, argument conventions, return types, side effects, and error behavior unless the shared contract and all affected callers are intentionally updated.
Files:
comfy/sd.py
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Do not addtorch.no_grad,torch.inference_mode, inference-mode wrappers, or explicit model freeze/trainability toggles; only disable globally enabled inference mode when a training path needs gradients.
Remove inference-only training behavior such as dropout while preserving checkpoint and state-dict compatibility; usenn.Identitywhen removing a module would alter keys or ordering.
Keep imports at module scope except for established optional-backend probes or import-cycle avoidance; avoid unnecessary exception handling and use specific exception types with useful fallbacks.
Do not add code for unsupported pinned library versions or obsolete PyTorch workarounds; unsupported formats, invalid quantization metadata, and bad states should fail clearly rather than silently degrading output.
Match local Python style, keep comments sparse and useful, and remove comments that merely restate obvious code.
Treat dtype, device placement, VRAM usage, and offloading as correctness concerns; use existing ComfyUI cast, offload, cleanup, quantization, and memory helpers.
Model implementations must use an existing optimized Comfy Kitchen, ComfyUI, quantization, or backend operation when it supports the required math, layout, dtype, device, memory, and interface contracts; inspect available operations before writing local kernels.
Retain local or differentiable fallbacks only when no optimized operation satisfies the required math or patch/autograd contract; adapt inputs to shared operation layouts while preserving exact model behavior.
Treat optimized attention and similar backend-selected callables as opaque; callers must rely on documented interfaces and result contracts rather than function identity, names, modules, or implementation details.
Do not duplicate existing inference operations with custom float32-upcasting implementations, such as custom RMSNorm variants; use generic ComfyUI or native torch operations.
If a model constructor has anoperationsparame...
Files:
comfy/sd.py
**
⚙️ CodeRabbit configuration file
**: IMPORTANT: Only comment on issues directly introduced by this PR's code changes.
Treat AGENTS.md as mandatory repository policy, not optional style guidance.
Flag PR changes that violate AGENTS.md even when the code is otherwise functional.
In particular, enforce architecture boundaries, dtype/device/memory rules,
interface contracts, import style, no unnecessary try/except blocks, no inline
imports, no outbound internet paths in core ComfyUI, and narrow scoped fixes.
Prefer direct findings over suggestions when a rule is violated. Only ignore
AGENTS.md when it clearly conflicts with a newer explicit maintainer instruction
in the PR.
Do NOT flag pre-existing issues in code that was merely moved, re-indented,
de-indented, or reformatted without logic changes. If code appears in the diff
only due to whitespace or structural reformatting (e.g., removing awith:block),
treat it as unchanged. Contributors should not feel obligated to address
pre-existing issues outside the scope of their contribution.
Files:
comfy/sd.py
comfy/**
⚙️ CodeRabbit configuration file
comfy/**: Core ML/diffusion engine. Focus on:
- Backward compatibility (breaking changes affect all custom nodes)
- Memory management and GPU resource handling
- Performance implications in hot paths
- Thread safety for concurrent execution
Files:
comfy/sd.py
🧠 Learnings (2)
📚 Learning: 2026-02-21T14:01:41.482Z
Learnt from: pythongosssss
Repo: Comfy-Org/ComfyUI PR: 12555
File: comfy_extras/nodes_glsl.py:719-724
Timestamp: 2026-02-21T14:01:41.482Z
Learning: In PyOpenGL, bare Python scalars can be accepted for 1-element array parameters by NumberHandler. This means you can pass an int/float directly to OpenGL texture deletion (e.g., glDeleteTextures(tex)) without wrapping in a list. Verify function-specific expectations and ensure types match what the OpenGL call expects; use explicit lists only when the API requires an array.
Applied to files:
comfy/sd.py
📚 Learning: 2026-05-13T12:31:45.069Z
Learnt from: rattus128
Repo: Comfy-Org/ComfyUI PR: 13802
File: comfy/pinned_memory.py:19-30
Timestamp: 2026-05-13T12:31:45.069Z
Learning: When reviewing code that uses comfy/pinned_memory.py’s `HostBuffer.extend(size=..., reallocate=...)`: by default (`reallocate` is not True / False), `extend(size=...)` is a *relative increment* that grows the buffer by `size` bytes—so slicing like `[offset:offset+size]` after `hostbuf.extend(size=size)` is correct and the argument should not be rewritten to `offset + size`. Only in the single-segment reallocation mode (`reallocate=True`, e.g., as used by `resize_pin_buffer()` in `comfy/model_management.py`) should `size` be treated as an *absolute target* and the call/arguments should be checked accordingly.
Applied to files:
comfy/sd.py
🔇 Additional comments (2)
comfy/sd.py (2)
75-77: LGTM!Also applies to: 942-959, 1432-1432, 1489-1489, 1788-1790
1556-1558: 🎯 Functional CorrectnessNo detector change needed.
The MiniMax path injects
5120forembedding_size, which matchesQwen3VL_32BConfig.hidden_size; the current key-based discriminator does not route an incompatible 5120-sized Qwen3-VL checkpoint.> Likely an incorrect or invalid review comment.
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
comfy_extras/nodes_minimax_h3.py (1)
245-247: 🚀 Performance & Scalability | 🟠 Major | ⚡ Quick winLimit reference frames before resizing.
The code resizes the complete video tensor and then discards frames after
frame_count. Long references therefore consume unnecessary memory and compute. The slice also retains the full resized backing storage during VAE encoding.Suggested fix
- frames = _resize(video_frames, cw, ch, "disabled") - if frames.shape[0] > frame_count: - frames = frames[:frame_count] + frames = _resize(video_frames[:frame_count], cw, ch, "disabled")As per coding guidelines, copy or limit large tensor slices so long-lived views do not retain large backing storage.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@comfy_extras/nodes_minimax_h3.py` around lines 245 - 247, Update the frame preparation flow around _resize so video_frames is limited to frame_count before resizing, avoiding work on discarded frames. Ensure the retained frame slice is copied or otherwise materialized as an independent tensor before VAE encoding, preventing a long-lived view from retaining the full video backing storage.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@comfy_extras/nodes_minimax_h3.py`:
- Around line 222-228: Update the dimension alignment in the ref_image_size
handling near the scale calculation so rounding cannot increase either dimension
beyond the scaled reference size when scale is 1.0. Use floor-based
CANVAS_MULTIPLE alignment while retaining the required minimum canvas size,
preserving the down-only behavior for both “match” and short-edge modes.
- Around line 178-179: Update the ref_image_size Combo.Input in
MiniMaxH3ReferenceToVideo to default to "max", preserving the historical
reference-image sizing for workflows that omit this option, and revise the
tooltip so its default behavior is accurately described.
---
Outside diff comments:
In `@comfy_extras/nodes_minimax_h3.py`:
- Around line 245-247: Update the frame preparation flow around _resize so
video_frames is limited to frame_count before resizing, avoiding work on
discarded frames. Ensure the retained frame slice is copied or otherwise
materialized as an independent tensor before VAE encoding, preventing a
long-lived view from retaining the full video backing storage.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 8bb1a6fa-1606-4d0b-83b7-ce613d3e2295
📒 Files selected for processing (1)
comfy_extras/nodes_minimax_h3.py
📜 Review details
⏰ Context from checks skipped due to timeout. (16)
- GitHub Check: test
- GitHub Check: test (ubuntu-latest)
- GitHub Check: test (windows-2022)
- GitHub Check: test (macos-latest)
- GitHub Check: Run Pylint
- GitHub Check: test (ubuntu-latest)
- GitHub Check: test (macos-latest)
- GitHub Check: test (windows-latest)
- GitHub Check: test
- GitHub Check: Run Pylint
- GitHub Check: test (windows-2022)
- GitHub Check: test (ubuntu-latest)
- GitHub Check: test (macos-latest)
- GitHub Check: test (windows-latest)
- GitHub Check: test (macos-latest)
- GitHub Check: test (ubuntu-latest)
🧰 Additional context used
📓 Path-based instructions (5)
**/*
📄 CodeRabbit inference engine (AGENTS.md)
**/*: Keep changes and file scope as small and direct as possible; prefer practical fixes, minimal dependencies, existing patterns, and removal of obsolete code.
Preserve existing APIs, node names, model-loading behavior, file layout, and workflow compatibility unless replacement is intentional.
Do not add core ComfyUI code that makes outbound internet requests, including telemetry, analytics, tracking, reporting, update checks, remote configuration, licensing checks, or background network activity. User-authorized model downloads are permitted only for the requested artifact and without telemetry.
Keep state and capability flags on the object that owns the behavior; prefer explicit parent-owned attributes over probing child objects withgetattrfor parent control flow.
Preserve shared method signatures, argument conventions, return types, side effects, and error behavior unless the shared contract and all affected callers are intentionally updated.
Files:
comfy_extras/nodes_minimax_h3.py
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Do not addtorch.no_grad,torch.inference_mode, inference-mode wrappers, or explicit model freeze/trainability toggles; only disable globally enabled inference mode when a training path needs gradients.
Remove inference-only training behavior such as dropout while preserving checkpoint and state-dict compatibility; usenn.Identitywhen removing a module would alter keys or ordering.
Keep imports at module scope except for established optional-backend probes or import-cycle avoidance; avoid unnecessary exception handling and use specific exception types with useful fallbacks.
Do not add code for unsupported pinned library versions or obsolete PyTorch workarounds; unsupported formats, invalid quantization metadata, and bad states should fail clearly rather than silently degrading output.
Match local Python style, keep comments sparse and useful, and remove comments that merely restate obvious code.
Treat dtype, device placement, VRAM usage, and offloading as correctness concerns; use existing ComfyUI cast, offload, cleanup, quantization, and memory helpers.
Model implementations must use an existing optimized Comfy Kitchen, ComfyUI, quantization, or backend operation when it supports the required math, layout, dtype, device, memory, and interface contracts; inspect available operations before writing local kernels.
Retain local or differentiable fallbacks only when no optimized operation satisfies the required math or patch/autograd contract; adapt inputs to shared operation layouts while preserving exact model behavior.
Treat optimized attention and similar backend-selected callables as opaque; callers must rely on documented interfaces and result contracts rather than function identity, names, modules, or implementation details.
Do not duplicate existing inference operations with custom float32-upcasting implementations, such as custom RMSNorm variants; use generic ComfyUI or native torch operations.
If a model constructor has anoperationsparame...
Files:
comfy_extras/nodes_minimax_h3.py
**/*node*.py
📄 CodeRabbit inference engine (AGENTS.md)
Keep node changes backward compatible by default, use sensible defaults for new inputs, avoid output-type changes, expose only inputs and outputs the node actually reads or owns, and avoid pass-through or workflow-shaping sockets.
Files:
comfy_extras/nodes_minimax_h3.py
**
⚙️ CodeRabbit configuration file
**: IMPORTANT: Only comment on issues directly introduced by this PR's code changes.
Treat AGENTS.md as mandatory repository policy, not optional style guidance.
Flag PR changes that violate AGENTS.md even when the code is otherwise functional.
In particular, enforce architecture boundaries, dtype/device/memory rules,
interface contracts, import style, no unnecessary try/except blocks, no inline
imports, no outbound internet paths in core ComfyUI, and narrow scoped fixes.
Prefer direct findings over suggestions when a rule is violated. Only ignore
AGENTS.md when it clearly conflicts with a newer explicit maintainer instruction
in the PR.
Do NOT flag pre-existing issues in code that was merely moved, re-indented,
de-indented, or reformatted without logic changes. If code appears in the diff
only due to whitespace or structural reformatting (e.g., removing awith:block),
treat it as unchanged. Contributors should not feel obligated to address
pre-existing issues outside the scope of their contribution.
Files:
comfy_extras/nodes_minimax_h3.py
comfy_extras/**
⚙️ CodeRabbit configuration file
comfy_extras/**: Community-contributed extra nodes. Focus on:
- Consistency with node patterns (INPUT_TYPES, RETURN_TYPES, FUNCTION, CATEGORY)
- No breaking changes to existing node interfaces
Files:
comfy_extras/nodes_minimax_h3.py
🧠 Learnings (6)
📚 Learning: 2026-02-21T14:01:41.482Z
Learnt from: pythongosssss
Repo: Comfy-Org/ComfyUI PR: 12555
File: comfy_extras/nodes_glsl.py:719-724
Timestamp: 2026-02-21T14:01:41.482Z
Learning: In PyOpenGL, bare Python scalars can be accepted for 1-element array parameters by NumberHandler. This means you can pass an int/float directly to OpenGL texture deletion (e.g., glDeleteTextures(tex)) without wrapping in a list. Verify function-specific expectations and ensure types match what the OpenGL call expects; use explicit lists only when the API requires an array.
Applied to files:
comfy_extras/nodes_minimax_h3.py
📚 Learning: 2026-03-04T14:05:31.426Z
Learnt from: jtydhr88
Repo: Comfy-Org/ComfyUI PR: 12757
File: comfy_extras/nodes_custom_sampler.py:1069-1089
Timestamp: 2026-03-04T14:05:31.426Z
Learning: In the ComfyUI sampling pipeline, treat percent_to_sigma(0.0) as a sentinel value (999999999.9) that means starting from pure noise. This is consistent with BasicScheduler via calculate_sigmas. The SamplingPercentToSigma node’s return_actual_sigma flag differentiates this sentinel from sigma_max. Reviewers should not flag CurveToSigmas or similar nodes that rely on percent_to_sigma as bugs; downstream samplers are expected to handle the sentinel correctly. When reviewing related sampling-related code, assume this sentinel semantics unless there is explicit handling for a real sigma_max.
Applied to files:
comfy_extras/nodes_minimax_h3.py
📚 Learning: 2026-04-04T13:29:15.653Z
Learnt from: kijai
Repo: Comfy-Org/ComfyUI PR: 13258
File: comfy_extras/nodes_frame_interpolation.py:151-189
Timestamp: 2026-04-04T13:29:15.653Z
Learning: In this ComfyUI codebase, node `execute()` inference is already run under a global `torch.inference_mode()` context established in the execution engine (e.g., `execution.py` around line ~732). During review, avoid recommending changes that wrap node inference loops in `torch.inference_mode()`—it is already applied, so such suggestions are likely redundant.
Applied to files:
comfy_extras/nodes_minimax_h3.py
📚 Learning: 2026-05-09T18:40:40.199Z
Learnt from: kijai
Repo: Comfy-Org/ComfyUI PR: 13813
File: comfy_extras/nodes_wandancer.py:868-872
Timestamp: 2026-05-09T18:40:40.199Z
Learning: When building video/temporal decoding nodes that call ComfyUI’s VAE.decode (comfy/sd.py), leverage VAE.decode’s existing VRAM-aware chunking along dim 0. Reshape or transpose the latent so the temporal dimension T is folded into dim 0 (e.g., transform a latent of shape [B, T, C, H, W] into [B*T, C, H, W] before calling vae.decode). This lets VAE.decode do chunked decoding without needing an explicit per-frame loop inside the node itself.
Applied to files:
comfy_extras/nodes_minimax_h3.py
📚 Learning: 2026-05-20T00:10:14.673Z
Learnt from: Pauan
Repo: Comfy-Org/ComfyUI PR: 13997
File: comfy_extras/nodes_string.py:12-25
Timestamp: 2026-05-20T00:10:14.673Z
Learning: In the ComfyUI `comfy_extras/` codebase, some nodes intentionally ship with a default input string that references parameters that may not yet be connected. If the default would raise a `KeyError` (e.g., examples like `MathExpression` default `a + b`, or `StringFormat` default `{a}` with `min=0` and autogrow inputs), treat it as an intentional “hint default” UX pattern, not a bug. During review, do not flag this behavior or recommend changing `min` to `1` or altering the default to an empty string solely to avoid the `KeyError`.
Applied to files:
comfy_extras/nodes_minimax_h3.py
📚 Learning: 2026-07-26T18:37:44.213Z
Learnt from: jtydhr88
Repo: Comfy-Org/ComfyUI PR: 15090
File: comfy_extras/nodes_video.py:230-235
Timestamp: 2026-07-26T18:37:44.213Z
Learning: In ComfyUI node implementations under `comfy_extras`, do not add regular widget/prompt inputs to `fingerprint_inputs` if they are already included in the node cache signature via `comfy_execution/caching.py:get_immediate_node_signature` (it records every non-link prompt input as `(key, inputs[key])`). Reserve `fingerprint_inputs` only for out-of-band state that can change without changing the prompt inputs (e.g., the selected source file’s modification time). For example, inputs like `LoadVideo.edit` should not be redundantly added to `fingerprint_inputs`; use it only for things not represented in prompt inputs.
Applied to files:
comfy_extras/nodes_minimax_h3.py
🔇 Additional comments (2)
comfy_extras/nodes_minimax_h3.py (2)
201-207: 🎯 Functional CorrectnessRequire an explicit audio-VAE sample-rate contract.
getattr(audio_vae, "audio_sample_rate", 32000)treats a missing attribute as valid 32 kHz input. A genericio.Vae.Input("audio_vae")can then resample audio at the wrong rate without a clear failure. Confirm that every supported MiniMax audio VAE exposes the expected rate, or fail when the contract is missing.As per coding guidelines, unsupported formats and bad states should fail clearly rather than silently degrade output.
Source: Coding guidelines
154-177: LGTM!Also applies to: 180-181, 183-198, 208-208, 211-212, 218-221, 234-244, 248-280
| io.Combo.Input("ref_image_size", options=["match", "max"], default="match", | ||
| tooltip="Reference image sizing. 'match' scales each ref (down only, keeping aspect) to the generation's pixel area; 'max' uses the reference pipeline's 2048px short edge for best identity fidelity. Reference tokens ride through every sampling step, so 'max' can be several times slower."), |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== locate file =="
fd -a 'nodes_minimax_h3.py' . || true
echo "== diff/stat if available =="
git diff --stat -- comfy_extras/nodes_minimax_h3.py || true
git diff -- comfy_extras/nodes_minimax_h3.py | sed -n '1,260p' || true
echo "== target file relevant sections =="
if [ -f comfy_extras/nodes_minimax_h3.py ]; then
nl -ba comfy_extras/nodes_minimax_h3.py | sed -n '1,340p'
fi
echo "== search previous ref_image_size and ref_image docs in tracked history =="
git -C . log --oneline --all -- comfy_extras/nodes_minimax_h3.py | sed -n '1,20p'
git -C . show `git -C . rev-list --max-count=1 --pretty=tformat:%H HEAD^ comfy_extras/nodes_minimax_h3.py 2>/dev/null || true`:comfy_extras/nodes_minimax_h3.py 2>/dev/null | rg -n "ref_image|ref_image_size|max_short|2048|INPUT_TYPES|RETURN_TYPES|FUNCTION" -C 3 || trueRepository: Comfy-Org/ComfyUI
Length of output: 331
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== git diff/stat =="
git diff --stat -- comfy_extras/nodes_minimax_h3.py || true
git diff -- comfy_extras/nodes_minimax_h3.py | sed -n '1,240p' || true
echo "== target relevant sections =="
awk '{printf "%s\n", NR":"$0}' comfy_extras/nodes_minimax_h3.py | sed -n '1,360p' | sed -n '150,210p' || true
awk '{printf "%s\n", NR":"$0}' comfy_extras/nodes_minimax_h3.py | sed -n '210,260p' || true
echo "== all ref_image related terms in current file =="
grep -n "ref_image_size\|ref_image\|2048\|match\|max_short\|short edge" comfy_extras/nodes_minimax_h3.py || true
echo "== search for input definition context =="
grep -n -A 8 -B 8 "ref_image_size" comfy_extras/nodes_minimax_h3.py || true
echo "== git diff refs in current file =="
git log --oneline --all -- comfy_extras/nodes_minimax_h3.py | sed -n '1,20p' || true
git -C . show HEAD^:comfy_extras/nodes_minimax_h3.py 2>/dev/null | grep -n -A 8 -B 8 "ref_image_size\|ref_image\|2048\|max_short" || trueRepository: Comfy-Org/ComfyUI
Length of output: 15351
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== parent ref_image contract =="
git show dc7db2ad4:comfy_extras/nodes_minimax_h3.py | sed -n '160,240p'
git show dc7db2ad4:comfy_extras/nodes_minimax_h3.py | grep -n -A 4 -B 4 "ref_image_size\|ref_image\|2048\|short edge" || true
echo "== minimal behavioral probe for current sizing default =="
python3 - <<'PY'
import math
CANVAS_MULTIPLE = 32
REF_IMAGE_SHORT_EDGE = 2048
def resize_dims(w, h, width, height):
scale = min(1.0, math.sqrt((width * height) / (w * h)))
tw = max(CANVAS_MULTIPLE, round(w * scale / CANVAS_MULTIPLE) * CANVAS_MULTIPLE)
th = max(CANVAS_MULTIPLE, round(h * scale / CANVAS_MULTIPLE) * CANVAS_MULTIPLE)
return tw, th
def max_dims(w, h):
scale = min(1.0, REF_IMAGE_SHORT_EDGE / min(w, h))
tw = max(CANVAS_MULTIPLE, round(w * scale / CANVAS_MULTIPLE) * CANVAS_MULTIPLE)
th = max(CANVAS_MULTIPLE, round(h * scale / CANVAS_MULTIPLE) * CANVAS_MULTIPLE)
return tw, th
for w, h in [(64, 64), (1024, 1024), (2048, 2048), (4096, 2304)]:
print((w,h), "match", resize_dims(w, h, 1344, 768), "max", max_dims(w, h), "match_area", resize_dims(w, h, 1344, 768)[0] * resize_dims(w, h, 1344, 768)[1] if w<=1344 and h<=768 else "<larger")
PYRepository: Comfy-Org/ComfyUI
Length of output: 7038
Preserve the previous reference-image default.
MiniMaxH3ReferenceToVideo historically downscaled reference images to the 2048px short-edge canvas, while this new combo default selects "match" instead. Existing workflows that omit ref_image_size will change reference conditioning and VRAM/token cost. Default the combo to "max" and update the tooltip.
Suggested fix
- io.Combo.Input("ref_image_size", options=["match", "max"], default="match",
+ io.Combo.Input("ref_image_size", options=["match", "max"], default="max",📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| io.Combo.Input("ref_image_size", options=["match", "max"], default="match", | |
| tooltip="Reference image sizing. 'match' scales each ref (down only, keeping aspect) to the generation's pixel area; 'max' uses the reference pipeline's 2048px short edge for best identity fidelity. Reference tokens ride through every sampling step, so 'max' can be several times slower."), | |
| io.Combo.Input("ref_image_size", options=["match", "max"], default="max", | |
| tooltip="Reference image sizing. 'match' scales each ref (down only, keeping aspect) to the generation's pixel area; 'max' uses the reference pipeline's 2048px short edge for best identity fidelity. Reference tokens ride through every sampling step, so 'max' can be several times slower."), |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@comfy_extras/nodes_minimax_h3.py` around lines 178 - 179, Update the
ref_image_size Combo.Input in MiniMaxH3ReferenceToVideo to default to "max",
preserving the historical reference-image sizing for workflows that omit this
option, and revise the tooltip so its default behavior is accurately described.
Source: Path instructions
…, keyframe guides Day-0 groundwork for Comfy-Org/ComfyUI#15224 (MiniMax H3). New minimax_h3.py is the one place that knows how H3 differs from LTXAV; everything else consults it. Four divergences that failed silently before: - Audio latent axis. LTXAV audio is [B,C,T,freq] (time on dim 2, same as video); H3 audio is [B,32,stereo,T]. Both 4-D, so only the model family can say which axis is time. The Chain Sampler now carries a per-stream time axis through continuation, blending, tail crops and JoyAI audio memory. - Frame grid. LTXAV 8k+1 -> k+1; H3 17k+5 -> 5k+2. H3's downscale_index_formula is the index map, not the count map, so the old uniform formula under-counted by ~20%. Frame counts now come from vae.downscale_ratio[0], which reproduces LTXAV's answer exactly. - Attention patches. LTX calls optimized_attention through its module; H3 binds it at import, so FunPack's temperature/capture patch was inert on H3. It now rebinds every module holding a reference, and position-indexed work (BachVid K/V, attn importance) sits out H3's pre-reshaped [1,heads,S,D] layout. - Batch size 1. H3's DiT refuses a batched forward and comfy batches cond+uncond for cfg != 1.0. install_batch_split runs them one at a time, keeping cfg live. Guides become keyframe pins: H3 packs condition rows into the sequence instead of appending a masked latent frame, so no tail is appended and none is cropped. The layout only accepts a first/last pin, and a mid-clip request is refused up front instead of raising inside PackedLayout. Studio: reference images route through Qwen3-VL's images= kwarg (the Gemma3 vision probe returns False for H3 and was dropping them); the attn2 K/V direction patch reports that H3 has no cross-attention to hook instead of installing a no-op. Features that depend on LTX transformer structure (bounded attention, Best-FaceID identity transfer, v2a_grad_scale, segmented detailing, second-pass latent ops) are now switched off explicitly on H3 with the reason printed, rather than each one discovering its own missing attribute mid-scene. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
3 participants
Add MiniMax H3 audio-video model support
Support for MiniMax H3, a single-stream packed-token DiT that denoises video (24ch, 16x spatial / 17k+5 frame grid) and stereo audio (32ch, 40 Hz) latents jointly, conditioned on Qwen3-VL-32B hidden states with per-token modality tags.
<Picture i> / <Video k> / <Audio j>labels and 2 fps timestamped video blocks.Nodes:
EmptyMiniMaxH3LatentAV
MiniMaxH3ImageToVideo
MiniMaxH3ReferenceToVideo
MiniMaxH3SigmaShift
Reuses the LTXV AV latent concat/separate nodes (display names generalized).
Not model specific changes:
ModelOpt AWQ-style pre_quant_scale support in quant ops
fused activation+quantize path for INT8 linears (linear_input_act).