You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Address all 5 review findings plus remove --optimize from examples:
1. Fix --optimize description: clarify it applies mobius rewrite rules
(group_query_attention, packed_attention, skip_norm), not general
constant folding. Remove from basic command examples since it's
optional, not default.
2. Fix EP descriptions: 'default' is portable ONNX (not CPU-specific),
'onnx-standard' inlines custom-domain functions (not DML-specific).
Add note about 'mobius list eps' for all available EPs.
3. Replace Olive direct-API snippets with config-driven olive.run()
pattern matching the repo's examples/olive/ convention. Reference
the ministral example for a complete working setup.
4. Add tokenizer + processor config files to the multi-model
quantization copy step — without these ORT GenAI won't load.
5. Replace fabricated generate_golden_data() with the real
scripts/generate_golden.py entrypoint and compare_golden() from
mobius._testing.parity.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Justin Chu <justinchu@microsoft.com>
|`--optimize [RULES]`| Apply mobius rewrite rules after building (e.g. `group_query_attention`, `packed_attention`, `skip_norm`). Use without value for all rules, or specify comma-separated names. Not needed for basic exports.|
45
44
|`--ep <variant>`| Execution provider variant (see below) |
46
-
|`--runtime ort-genai`| Generate `genai_config.json` for ORT GenAI runtime |
45
+
|`--runtime ort-genai`| Generate `genai_config.json`and copy tokenizer files for ORT GenAI runtime |
47
46
|`--external-data safetensors`| Store weights externally in safetensors format |
48
47
|`--max-shard-size 5GB`| Split external data into shards ≤ 5GB |
49
48
@@ -54,17 +53,20 @@ rewrites and fused ops:
54
53
55
54
| EP | Flag | When to use |
56
55
|----|------|-------------|
57
-
|`default`|`--ep default`|CPU inference. No custom ops — pure standard ONNX. Compatible with all runtimes. |
56
+
|`default`|`--ep default`|Portable ONNX — no vendor-specific fusions. Compatible with all execution providers and runtimes. This is the default if `--ep` is omitted. |
58
57
|`cuda`|`--ep cuda`| NVIDIA GPU inference. Emits `com.microsoft` fused ops (GroupQueryAttention, MoE, etc.) for maximum CUDA performance. |
59
-
|`onnx-standard`|`--ep onnx-standard`| DirectML / cross-platform GPU. Standard ONNX ops only — no contrib ops. Works on AMD, Intel, and NVIDIA via DML. |
58
+
|`onnx-standard`|`--ep onnx-standard`| Strict ONNX-only — inlines all custom-domain functions into standard ONNX ops. Use when targeting runtimes that don't support `com.microsoft` ops. |
59
+
60
+
Other EPs are available (`cpu`, `dml`, `webgpu`, `trt-rtx`). Run
61
+
`mobius list eps` to see all options.
60
62
61
63
**Typical export matrix:** Build each dtype × EP combination:
0 commit comments