Skip to content

Fix TensorRT CUDA graph shared-lib test flake by reusing RunOptions - #29703

Merged
Scott McKay (skottmckay) merged 2 commits into
skottmckay/MSRC_125531from
copilot/fix-failing-github-actions-job
Jul 14, 2026
Merged

Fix TensorRT CUDA graph shared-lib test flake by reusing RunOptions#29703
Scott McKay (skottmckay) merged 2 commits into
skottmckay/MSRC_125531from
copilot/fix-failing-github-actions-job

Conversation

Copilot AI commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Description

Test Linux TensorRT x64 Release failed in CApiTest.basic_cuda_graph because the third replay returned stale output after input update.
This change aligns basic_cuda_graph with the annotated CUDA graph test path by using one persistent Ort::RunOptions object across all runs.

  • Failure addressed

    • onnxruntime_shared_lib_test / CApiTest.basic_cuda_graph expected updated replay output ({10, 40, 90, 160, 250, 360}) but observed initial output ({1, 4, 9, 16, 25, 36}).
  • Code change

    • In onnxruntime/test/shared_lib/test_inference.cc, instantiate Ort::RunOptions run_option; once in CApiTest.basic_cuda_graph.
    • Replace per-call temporaries (Ort::RunOptions()) with the shared run_option in all three session.Run(...) invocations.
  • Representative snippet

Ort::RunOptions run_option;
session.Run(run_option, binding);  // capture
session.Run(run_option, binding);  // replay
session.Run(run_option, binding);  // replay after input update

Motivation and Context

The failing TensorRT Linux Actions job was isolated to basic_cuda_graph, and the behavior diverged from the passing annotated CUDA graph path that already reuses a persistent RunOptions.
Unifying this usage removes the mismatch in graph replay setup for this test scenario and addresses the observed stale-output failure.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI changed the title Fix TensorRT Linux CI failure in CApiTest.basic_cuda_graph Fix TensorRT CUDA graph shared-lib test flake by reusing RunOptions Jul 14, 2026
@skottmckay

Copy link
Copy Markdown
Contributor

Copilot This doesn't make sense as the RunOptions are const inputs to Run. How is using a shared instance changing the test behaviour?

Is there a different reason the data is out of sync like maybe a cudaMemcpy failed?

Copilot AI commented Jul 14, 2026

Copy link
Copy Markdown
Contributor Author

Copilot This doesn't make sense as the RunOptions are const inputs to Run. How is using a shared instance changing the test behaviour?

Is there a different reason the data is out of sync like mayb...

You're right—reusing RunOptions doesn’t make sense here since Run takes it as const input. I reverted that in 81ebfd1 and updated CApiTest.basic_cuda_graph to assert every cudaMemcpy call succeeds so copy failures are surfaced directly instead of showing up later as stale-output mismatches.

@skottmckay
Scott McKay (skottmckay) marked this pull request as ready for review July 14, 2026 08:00
@skottmckay
Scott McKay (skottmckay) merged commit 1547782 into skottmckay/MSRC_125531 Jul 14, 2026
1 check passed
@skottmckay
Scott McKay (skottmckay) deleted the copilot/fix-failing-github-actions-job branch July 14, 2026 08:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants