Skip to content

[Perf] Linux/x64: 9 Regressions on 11/18/2022 7:31:41 AM #78827

Description

@performanceautofiler

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 911cc41358aa3a99c24fa85dca426fafa3bef555
Compare 82054d0a71622a22fdfd278d957019a3d69c4040
Diff Diff

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
XmlSerializer_ - Duration of single invocation 34.50 μs 37.20 μs 1.08 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;SimpleStructWithProperties&gt;*'
Details

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<SimpleStructWithProperties>.XmlSerializer_


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.199494570405726 > 36.317790635636776.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/18/2022 6:07:58 AM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -22.710939893340225 (T) = (0 -37228.963743060296) / Math.Sqrt((185062.70953078516 / (29)) + (30710.694009210805 / (18))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (18) - 2, .025) and -0.05804555392502422 = (35186.541453675854 - 37228.963743060296) / 35186.541453675854 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 911cc41358aa3a99c24fa85dca426fafa3bef555
Compare 82054d0a71622a22fdfd278d957019a3d69c4040
Diff Diff

Regressions in System.Reflection.Invoke

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Field_Set_class - Duration of single invocation 4.26 μs 4.88 μs 1.15 0.01 False
StaticMethod4_ByRefParams_int_string_struct_class - Duration of single invocation 3.96 μs 10.66 μs 2.69 0.01 False
StaticMethod5_ByRefParams_int_string_struct_class_bool - Duration of single invocation 4.99 μs 13.85 μs 2.77 0.02 False
Property_Get_class - Duration of single invocation 55.01 ns 61.59 ns 1.12 0.25 False
Method0_NoParms - Duration of single invocation 1.33 μs 1.47 μs 1.10 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Invoke*'
Details

Payloads

Baseline
Compare

Histogram

System.Reflection.Invoke.Field_Set_class


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.881291129687499 > 4.507803969921874.
IsChangePoint: Marked as a change because one of 11/18/2022 6:07:58 AM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -52.13861959571211 (T) = (0 -4972.768487322255) / Math.Sqrt((1761.27582811406 / (29)) + (1737.0505871662383 / (18))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (18) - 2, .025) and -0.15137522296280304 = (4318.981673520786 - 4972.768487322255) / 4318.981673520786 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.StaticMethod4_ByRefParams_int_string_struct_class

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.664027450892858 > 4.159710839921875.
IsChangePoint: Marked as a change because one of 11/18/2022 6:07:58 AM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -257.2411492030293 (T) = (0 -10727.2056194171) / Math.Sqrt((4504.377009562933 / (29)) + (9598.885376745995 / (18))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (18) - 2, .025) and -1.6973667527872534 = (3976.9177136673843 - 10727.2056194171) / 3976.9177136673843 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.StaticMethod5_ByRefParams_int_string_struct_class_bool

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.846075135416665 > 5.238917055859376.
IsChangePoint: Marked as a change because one of 11/18/2022 6:07:58 AM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -328.8497895160741 (T) = (0 -13666.393086988704) / Math.Sqrt((4455.796688639803 / (29)) + (9659.232147972647 / (18))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (18) - 2, .025) and -1.718861961811631 = (5026.512297771277 - 13666.393086988704) / 5026.512297771277 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.Property_Get_class

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 61.59446989243531 > 60.64124227120833.
IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -6.881911696879265 (T) = (0 -62.49095342617084) / Math.Sqrt((7.11541943712059 / (19)) + (2.835592497393372 / (28))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (28) - 2, .025) and -0.08220507556126465 = (57.744095677763404 - 62.49095342617084) / 57.744095677763404 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Invoke.Method0_NoParms

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.470787190151515 > 1.3991757950260422.
IsChangePoint: Marked as a change because one of 11/15/2022 5:29:43 PM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -16.072001296700833 (T) = (0 -1444.216785103397) / Math.Sqrt((376.9146763962987 / (18)) + (898.8628056428271 / (29))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (29) - 2, .025) and -0.08719137483155047 = (1328.3924233920307 - 1444.216785103397) / 1328.3924233920307 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 911cc41358aa3a99c24fa85dca426fafa3bef555
Compare 82054d0a71622a22fdfd278d957019a3d69c4040
Diff Diff

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<ClassImplementingIXmlSerialiable>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
XmlSerializer_ - Duration of single invocation 20.19 μs 22.22 μs 1.10 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;ClassImplementingIXmlSerialiable&gt;*'
Details

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<ClassImplementingIXmlSerialiable>.XmlSerializer_


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.224132927412857 > 21.18279946905933.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/18/2022 6:07:58 AM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -42.19504169192833 (T) = (0 -22299.07446752603) / Math.Sqrt((46868.473286063374 / (29)) + (11363.66805709808 / (18))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (18) - 2, .025) and -0.09854617388787648 = (20298.713879824583 - 22299.07446752603) / 20298.713879824583 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 911cc41358aa3a99c24fa85dca426fafa3bef555
Compare 82054d0a71622a22fdfd278d957019a3d69c4040
Diff Diff

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<XmlElement>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
XmlSerializer_ - Duration of single invocation 18.13 μs 20.28 μs 1.12 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;XmlElement&gt;*'
Details

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<XmlElement>.XmlSerializer_


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.284128040132064 > 18.96606242714838.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 11/18/2022 6:07:58 AM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -31.878281753613635 (T) = (0 -20093.224263958327) / Math.Sqrt((40119.669852435836 / (29)) + (38003.141152651966 / (18))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (18) - 2, .025) and -0.10349568972727433 = (18208.702082854812 - 20093.224263958327) / 18208.702082854812 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 911cc41358aa3a99c24fa85dca426fafa3bef555
Compare 82054d0a71622a22fdfd278d957019a3d69c4040
Diff Diff

Regressions in BenchmarksGame.SpectralNorm_3

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
RunBench - Duration of single invocation 3.40 ms 4.73 ms 1.39 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BenchmarksGame.SpectralNorm_3*'
Details

Payloads

Baseline
Compare

Histogram

BenchmarksGame.SpectralNorm_3.RunBench


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.7280149120065795 > 3.5410243465625.
IsChangePoint: Marked as a change because one of 11/18/2022 6:07:58 AM, 11/22/2022 2:20:42 AM falls between 11/9/2022 12:33:05 PM and 11/22/2022 2:20:42 AM.
IsRegressionStdDev: Marked as regression because -5.683203975262063 (T) = (0 -4365224.6557979565) / Math.Sqrt((127508963466.9081 / (29)) + (349671376416.0141 / (18))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (18) - 2, .025) and -0.25148418305304293 = (3488038.214872861 - 4365224.6557979565) / 3488038.214872861 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions