Skip to content

Allow text-only batches in MIMO encoder prefetch - #6033

Closed
yashaswikarnati wants to merge 9 commits into
NVIDIA:mainfrom
yashaswikarnati:agent/mimo-prefetch-text-only-batches
Closed

yashaswikarnati wants to merge 9 commits into
NVIDIA:mainfrom
yashaswikarnati:agent/mimo-prefetch-text-only-batches

Conversation

@yashaswikarnati

Copy link
Copy Markdown
Contributor

What changed

  • pass text-only encoder batches through with their CPU input_ids
  • skip frozen-encoder feature production when a batch has no modality inputs
  • only record feature streams and projection timing when prefetched features exist
  • add a focused regression test for text-only batches

Why

Real multimodal datasets can produce text-only microbatches. The encoder-prefetch loader currently rejects those batches before the MIMO model can use its existing empty-encoder-output path.

Validation

  • Python syntax and whitespace checks passed locally
  • OCI-HSG job 5587057: 5 nodes / 20 GPUs, encoder DP 4, LLM DP/EP 16, GBS 32, four dataloader workers, prefetch depth 4
  • completed 5/5 iterations with finite loss, zero skipped iterations, and zero NaN iterations
  • audited all 20 rank stderr logs with no traceback or fatal-signature matches

Dependency

This is a stacked draft on top of #5833 because encoder prefetch is not yet present on main. After #5833 lands, this branch should be rebased onto main, leaving the single fix commit c8cc9aabb5.

Signed-off-by: ykarnati <ykarnati@nvidia.com>
Signed-off-by: ykarnati <ykarnati@nvidia.com>
Signed-off-by: ykarnati <ykarnati@nvidia.com>
Signed-off-by: ykarnati <ykarnati@nvidia.com>
Signed-off-by: ykarnati <ykarnati@nvidia.com>
Signed-off-by: ykarnati <ykarnati@nvidia.com>
Signed-off-by: ykarnati <ykarnati@nvidia.com>
Signed-off-by: ykarnati <ykarnati@nvidia.com>
Signed-off-by: ykarnati <ykarnati@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Jul 24, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant