-
Notifications
You must be signed in to change notification settings - Fork 5.5k
Refactor common variables used for the "send runtime tests to helix" steps into a shared template that all of our pipelines that run runtime tests can use. #81926
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
jkoritzinsky
merged 12 commits into
dotnet:main
from
jkoritzinsky:runtime-test-variables-template
Feb 23, 2023
Merged
Changes from all commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
ba2b392
Refactor common variables used for the "send runtime tests to helix" …
jkoritzinsky 22e2c8e
Move parameters around so all "processed/known" args are passed first…
jkoritzinsky b63a81a
Move runtimeFlavor before testBuildArgs so testBuildArgs can end with…
jkoritzinsky 59c4e2a
Pass variables with the extra steps template.
jkoritzinsky 5be8aa3
Fix indentation
jkoritzinsky 857d96e
Revert "Fix indentation"
jkoritzinsky f1ee436
Revert "Pass variables with the extra steps template."
jkoritzinsky 22b774c
Foward common parameters by default to extra variable templates.
jkoritzinsky 658b467
Pass through the live libraries build configuration
jkoritzinsky 97edfc4
Actually pass through the config and set the test group in the right …
jkoritzinsky 9676ec4
Default test group to innerloop as that best represents the expected …
jkoritzinsky a2dcb1b
Merge branch 'main' into runtime-test-variables-template
jkoritzinsky File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
100 changes: 100 additions & 0 deletions
100
eng/pipelines/common/templates/runtimes/test-variables.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,100 @@ | ||
| parameters: | ||
| testGroup: 'innerloop' | ||
| readyToRun: false | ||
| runtimeFlavor: '' | ||
| runtimeVariant: '' | ||
|
|
||
| variables: | ||
|
|
||
| # Set job timeouts | ||
| # | ||
| # "timeoutPerTestCollectionInMinutes" is the time needed for the "biggest" xUnit test collection to complete. | ||
| # In case xUnit test wrappers get refactored this number should also be adjusted. | ||
| # | ||
| # "timeoutPerTestInMinutes" corresponds to individual test running time. This is implemented by setting | ||
| # the __TestTimeout variable, which is later read by the coreclr xunit test wrapper code (the code in the | ||
| # xunit test dlls that invokes the actual tests). | ||
| # | ||
| # Note that "timeoutInMinutes" is an Azure DevOps Pipelines parameter for a "job" that specifies the | ||
| # total time allowed for a job, and is specified lower down. Make sure you set it properly for any new testGroup. | ||
| # | ||
| # Please note that for Crossgen2 R2R runs, the "test running time" also includes the | ||
| # time needed to compile the test into native code with the Crossgen2 compiler. | ||
|
|
||
| - name: timeoutPerTestInMinutes | ||
| value: 10 | ||
| - name: timeoutPerTestCollectionInMinutes | ||
| value: 30 | ||
| - ${{ if in(parameters.testGroup, 'outerloop') }}: | ||
| - name: timeoutPerTestCollectionInMinutes | ||
| value: 120 | ||
| - ${{ if eq(parameters.readyToRun, true) }}: | ||
| - name: timeoutPerTestCollectionInMinutes | ||
| value: 90 | ||
| - name: timeoutPerTestInMinutes | ||
| value: 30 | ||
| - ${{ if in(parameters.testGroup, 'gc-longrunning', 'gc-simulator') }}: | ||
| - name: timeoutPerTestCollectionInMinutes | ||
| value: 360 | ||
| # gc reliability may take up to 2 hours to shutdown. Some scenarios have very long iteration times. | ||
| - name: timeoutPerTestInMinutes | ||
| value: 240 | ||
| - ${{ if in(parameters.testGroup, 'jitstress', 'jitstress-random', 'jitstress-isas-arm', 'jitstress-isas-x86', 'jitstress-isas-avx512', 'jitstressregs-x86', 'jitstressregs', 'jitstress2-jitstressregs', 'jitelthookenabled' ) }}: | ||
| - name: timeoutPerTestCollectionInMinutes | ||
| value: 120 | ||
| - name: timeoutPerTestInMinutes | ||
| value: 30 | ||
| - ${{ if in(parameters.testGroup, 'gcstress0x3-gcstress0xc') }}: | ||
| - name: timeoutPerTestCollectionInMinutes | ||
| value: 240 | ||
| - name: timeoutPerTestInMinutes | ||
| value: 60 | ||
| - ${{ if in(parameters.testGroup, 'gcstress-extra', 'r2r-extra') }}: | ||
| - name: timeoutPerTestCollectionInMinutes | ||
| value: 300 | ||
| - name: timeoutPerTestInMinutes | ||
| value: 90 | ||
| - ${{ if eq(parameters.testGroup, 'ilasm') }}: | ||
| # ilasm-ildasm round trip testing runs every test twice, plus runs ilasm and ildasm, so double the 'outerloop' timeout numbers. | ||
| - name: timeoutPerTestInMinutes | ||
| value: 20 | ||
| - name: timeoutPerTestCollectionInMinutes | ||
| value: 240 | ||
| - ${{ if in(parameters.testGroup, 'clrinterpreter') }}: | ||
| - name: timeoutPerTestCollectionInMinutes | ||
| value: 180 | ||
| - name: timeoutPerTestInMinutes | ||
| value: 30 | ||
| - ${{ if in(parameters.testGroup, 'pgo', 'pgostress') }}: | ||
| - name: timeoutPerTestCollectionInMinutes | ||
| value: 120 | ||
| - ${{ if in(parameters.testGroup, 'jit-cfg') }}: | ||
| - name: timeoutPerTestCollectionInMinutes | ||
| value: 120 | ||
|
|
||
| - name: runtimeFlavorArgs | ||
| value: '' | ||
|
|
||
| - ${{ if eq(parameters.runtimeFlavor, 'mono') }}: | ||
| - name: runtimeFlavorArgs | ||
| value: '-mono' | ||
|
|
||
| - name: runtimeVariantArg | ||
| value: '' | ||
|
|
||
| - ${{ if ne(parameters.runtimeVariant, '') }}: | ||
| - name: runtimeVariantArg | ||
| value: '/p:RuntimeVariant=${{ parameters.runtimeVariant }}' | ||
|
|
||
| - name: priorityArg | ||
| value: '' | ||
|
|
||
| # 'innerloop' and 'clrinterpreter' jobs run the Priority 0 tests; everything else runs the Priority 1 tests. | ||
| # 'gc-standalone' is forced to run pri0 as well to start with. | ||
| - ${{ if and(ne(parameters.testGroup, 'innerloop'), ne(parameters.testGroup, 'clrinterpreter'), ne(parameters.testGroup, 'gc-standalone'), ne(parameters.testGroup, 'gc-standalone-server') ) }}: | ||
| - ${{ if ne(parameters.osGroup, 'windows') }}: | ||
| - name: priorityArg | ||
| value: 'priority1' | ||
| - ${{ if eq(parameters.osGroup, 'windows') }}: | ||
| - name: priorityArg | ||
| value: '-priority=1' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is an eng/common file which needs to be updated in Arcade: https://github.com/dotnet/arcade/blob/b888df17a4acb65630c1e9ad5e94f22a33b62ab0/eng/common/templates/job/job.yml#L104
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Arcade PR is here: dotnet/arcade#12524