[ORT] onnxruntime_perf_test: Add multi-shape profiling support to perftest tool (--data_shape) - #29571
Merged
Edward Chen (edgchen1) merged 3 commits intoJul 9, 2026
Conversation
…ta_shape) (#1132) Usage: .\onnxruntime_perf_test.exe -v -e openvino -m times -r 10 -I --data_shape "data:[1,3,60,40][1,3,60,60]" -i "device_type|NPU" "PSO2_ctx.onnx" or, using testdata- .\onnxruntime_perf_test.exe -v -e openvino -m times -r 10 --data_shape "data:[1,3,60,40][1,3,60,60]" -i "device_type|NPU" "PSO2_ctx.onnx" Addresses the feature request: microsoft#28628
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends the onnxruntime_perf_test tool to support multi-shape profiling via a new --data_shape flag, enabling a single session to run multiple input shape groups (round-robin) and report per-shape latency statistics.
Changes:
- Added
--data_shapeCLI flag parsing and storage inRunConfig. - Implemented multi-shape input generation/selection and round-robin execution, with per-shape latency tracking and reporting.
- Added a small ONNX model generator script for exercising multi-shape behavior.
Reviewed changes
Copilot reviewed 10 out of 11 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| onnxruntime/test/testdata/dynamic_shape_add.py | Adds a small dynamic-shape ONNX model generator for testing --data_shape. |
| onnxruntime/test/perftest/test_session.h | Extends RunTiming to carry the shape group index used by an iteration. |
| onnxruntime/test/perftest/test_configuration.h | Adds RunConfig::data_shape_groups to hold per-input shape groups. |
| onnxruntime/test/perftest/strings_helper.h | Declares ParseDataShapeGroups for parsing the --data_shape argument. |
| onnxruntime/test/perftest/strings_helper.cc | Implements ParseDataShapeGroups string parsing + validation. |
| onnxruntime/test/perftest/performance_runner.h | Adds per-shape timing storage to PerformanceResult and a printing helper. |
| onnxruntime/test/perftest/performance_runner.cc | Implements warmup per shape group, per-shape stats reporting, and multi-shape init paths. |
| onnxruntime/test/perftest/ort_test_session.h | Adds round-robin controls and helper APIs for multi-shape test data selection. |
| onnxruntime/test/perftest/ort_test_session.cc | Implements round-robin selection and multi-shape input generation; tags timing with shape index. |
| onnxruntime/test/perftest/command_args_parser.cc | Adds the --data_shape flag and wires parsing into RunConfig. |
Contributor
|
Thanks Edward Chen (@edgchen1) for the review! Comments are addressed. |
Mayuresh Varerkar (MayureshV1)
approved these changes
Jul 9, 2026
Edward Chen (edgchen1)
approved these changes
Jul 9, 2026
Edward Chen (edgchen1)
enabled auto-merge (squash)
July 9, 2026 01:48
This was referenced Aug 12, 2026
Closed
This was referenced Aug 17, 2026
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adds a
--data_shapeflag toonnxruntime_perf_testapp that enables profiling multiple input shapes within a single session. The model is compiled once and then run with each specified shape group in round-robin order, providing per-shape latency statistics (Avg, Min, Max, P50, P90, P95, P99). This is useful for evaluating dynamic-shape model performance across different input dimensions without needing to recompile or restart the session for each shape.Usage:
#With -I (generate random default inputs of specified shapes):
onnxruntime_perf_test.exe -v -e cpu -m times -r 10 -I --data_shape "data:[1,3,60,10][1,3,60,20]" "model.onnx"#Without -I (select matching test data folders):
onnxruntime_perf_test.exe -v -e cpu -m times -r 10 --data_shape "data:[1,3,60,10][1,3,60,20]" "model.onnx"Motivation and Context
Addresses the feature request: #28628