Skip to content

[Relax][NN] Use int64 for RoPE apply flag#19430

Merged
tlopex merged 1 commit into
apache:mainfrom
xthomaswang:relax/rope-call-tir-int64-flag
Apr 24, 2026
Merged

[Relax][NN] Use int64 for RoPE apply flag#19430
tlopex merged 1 commit into
apache:mainfrom
xthomaswang:relax/rope-call-tir-int64-flag

Conversation

@xthomaswang

Copy link
Copy Markdown
Contributor

This patch aligns the dtype of the apply_rope flag used by
llama_rope_with_position_map with the host-side value passed through
Relax call_tir.

Previously the PrimFunc declared apply_rope as T.int32, while the
caller-side scalar value is represented as an int64 Relax PrimValue /
ShapeExpr value. This caused Relax well-formed analysis to reject the IR
with:

Argument N type mismatch: expected R.Prim("int32"), given R.Prim(value=1)

The mismatch can be reproduced through downstream nn.Module.export_tvm
paths such as MLC-LLM convert_weight / compile.

This change updates:

  • llama_rope_with_position_map: apply_rope: T.int32 -> T.int64
  • PagedKVCache: pass the split-rotary flag as int64_t

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the data type of the apply_rope parameter from int32 to int64 in the Python frontend and changes the corresponding cast to int64_t in the C++ runtime. These changes ensure consistency in integer precision across the codebase. I have no feedback to provide.

@xthomaswang

Copy link
Copy Markdown
Contributor Author

Downstream impact:

This fixes MLC-LLM export paths that instantiate the RoPE PrimFunc through
Relax nn.Module.export_tvm, including mlc_llm convert_weight and
mlc_llm compile. Without this patch, those paths may produce a
not-well-formed Relax IR warning or fail at the frontend well-formedness
assert, depending on whether the downstream tree demotes the assert.

@cchung100m cchung100m left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks to @xthomaswang 😄

@tlopex tlopex merged commit 2c72b90 into apache:main Apr 24, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants