Skip to content

[Perf] Linux/x64: 7 Regressions on 12/11/2022 2:11:02 PM #79606

Description

@performanceautofiler

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d0a1ad2a7b6363bf68a6346efa3c9754f2711970
Compare cfdf0f67485ae41f4454a687b1e01ccb32e5e799
Diff Diff

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ctor_string - Duration of single invocation 266.15 ns 302.71 ns 1.14 0.03 False
ctor_capacity - Duration of single invocation 139.13 ns 154.88 ns 1.11 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_StringBuilder.ctor_string(length: 100)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 302.70722940489685 > 281.68682770926114.
IsChangePoint: Marked as a change because one of 11/9/2022 12:33:05 PM, 12/3/2022 2:55:34 AM, 12/10/2022 11:25:25 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsRegressionStdDev: Marked as regression because -3.539056613080661 (T) = (0 -293.4867167865786) / Math.Sqrt((376.89353950198995 / (50)) + (159.82589452938768 / (8))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (8) - 2, .025) and -0.06752599941751249 = (274.9223128492584 - 293.4867167865786) / 274.9223128492584 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_StringBuilder.ctor_capacity(length: 100)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 154.87664426923203 > 140.44646051884087.
IsChangePoint: Marked as a change because one of 11/15/2022 5:29:43 PM, 12/2/2022 5:21:30 AM, 12/10/2022 11:25:25 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsRegressionStdDev: Marked as regression because -14.120489701362283 (T) = (0 -155.87677184521672) / Math.Sqrt((6.1196510690500014 / (50)) + (15.980723966488073 / (8))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (8) - 2, .025) and -0.1519369106908859 = (135.31710842716902 - 155.87677184521672) / 135.31710842716902 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline d0a1ad2a7b6363bf68a6346efa3c9754f2711970
Compare cfdf0f67485ae41f4454a687b1e01ccb32e5e799
Diff Diff

Regressions in System.Tests.Perf_Random

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Next_int_unseeded - Duration of single invocation 144.29 ns 154.80 ns 1.07 0.15 False
Next_int - Duration of single invocation 83.66 ns 95.03 ns 1.14 0.15 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Random*'
Details

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Random.Next_int_unseeded


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 154.8002019210073 > 148.68770389385432.
IsChangePoint: Marked as a change because one of 10/12/2022 11:17:15 AM, 11/15/2022 8:48:03 PM, 12/2/2022 2:38:36 AM, 12/10/2022 11:25:25 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsRegressionStdDev: Marked as regression because -8.23705883114742 (T) = (0 -156.02817803647324) / Math.Sqrt((15.5969647820477 / (50)) + (13.077857982450045 / (8))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (8) - 2, .025) and -0.07951402932122628 = (144.53557230245556 - 156.02817803647324) / 144.53557230245556 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Random.Next_int

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 95.03474177871958 > 87.15511640308564.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/2/2022 5:21:30 AM, 12/10/2022 11:25:25 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsRegressionStdDev: Marked as regression because -7.229688622162542 (T) = (0 -91.10217203871278) / Math.Sqrt((7.441373556492987 / (50)) + (11.6228926740883 / (8))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (8) - 2, .025) and -0.11164694509064729 = (81.95243322625603 - 91.10217203871278) / 81.95243322625603 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d0a1ad2a7b6363bf68a6346efa3c9754f2711970
Compare cfdf0f67485ae41f4454a687b1e01ccb32e5e799
Diff Diff

Regressions in System.Collections.CtorGivenSizeNonGeneric

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 333.70 ns 373.57 ns 1.12 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorGivenSizeNonGeneric*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.CtorGivenSizeNonGeneric.Queue(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 373.5733747842816 > 352.2351794534928.
IsChangePoint: Marked as a change because one of 11/17/2022 4:58:45 AM, 12/10/2022 11:25:25 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsRegressionStdDev: Marked as regression because -36.23606847478164 (T) = (0 -374.9997248543027) / Math.Sqrt((35.68562305212292 / (50)) + (5.431972813048347 / (8))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (8) - 2, .025) and -0.12871348093271726 = (332.2364188867666 - 374.9997248543027) / 332.2364188867666 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline d0a1ad2a7b6363bf68a6346efa3c9754f2711970
Compare cfdf0f67485ae41f4454a687b1e01ccb32e5e799
Diff Diff

Regressions in System.Collections.CtorDefaultSizeNonGeneric

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 131.68 ns 174.45 ns 1.32 0.08 False
Hashtable - Duration of single invocation 123.41 ns 135.62 ns 1.10 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorDefaultSizeNonGeneric*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.CtorDefaultSizeNonGeneric.Queue


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 174.45280287747184 > 136.61023355862744.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsRegressionStdDev: Marked as regression because -40.91559679811722 (T) = (0 -170.90649085293063) / Math.Sqrt((4.949346711821823 / (50)) + (7.115001059125946 / (8))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (8) - 2, .025) and -0.3123467275033625 = (130.22967731863585 - 170.90649085293063) / 130.22967731863585 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorDefaultSizeNonGeneric.Hashtable

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 135.61752535817428 > 130.09362569212945.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/11/2022 2:11:02 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM.
IsRegressionStdDev: Marked as regression because -12.736432008276347 (T) = (0 -132.76898269677437) / Math.Sqrt((5.269002748669535 / (51)) + (2.146359250513268 / (7))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (7) - 2, .025) and -0.065439224718225 = (124.61431831730015 - 132.76898269677437) / 124.61431831730015 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions