[Perf] Linux/x64: 4 Regressions on 9/27/2022 4:04:11 PM

[performanceautofiler]

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	007563949e53bad52dfcdf95927b0ce00101df4d
Compare	1f688ed75133664aa20c838fa451e2ecf9f1f424
Diff	Diff

Regressions in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Ctor_Span - Duration of single invocation	38.17 ns	113.39 ns	2.97	0.03	False
Ctor_Bytes - Duration of single invocation	41.32 ns	115.20 ns	2.79	0.03	False
GetAddressBytes - Duration of single invocation	31.73 ns	84.37 ns	2.66	0.03	False
TryWriteBytes - Duration of single invocation	22.94 ns	73.31 ns	3.20	0.03	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*'

Details

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Span(address: [16, 32, 48, 64, 80, ...])

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 113.38536472819986 > 41.081126953527246.
IsChangePoint: Marked as a change because one of 9/27/2022 11:22:50 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -573.9698716870419 (T) = (0 -113.15522885014654) / Math.Sqrt((0.2683060587789468 / (26)) + (0.2386099705281585 / (38))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (38) - 2, .025) and -1.8860726844472462 = (39.207338560781444 - 113.15522885014654) / 39.207338560781444 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Bytes(address: [16, 32, 48, 64, 80, ...])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 115.20384776238377 > 43.389346375651755.
IsChangePoint: Marked as a change because one of 9/27/2022 11:22:50 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -476.25748940860007 (T) = (0 -115.28240541792304) / Math.Sqrt((0.4289559560285226 / (26)) + (0.29926879395462297 / (40))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (40) - 2, .025) and -1.7757742566452588 = (41.531621363637434 - 115.28240541792304) / 41.531621363637434 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.GetAddressBytes(address: 1020:3040:5060:7080:9010:1112:1314:1516)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 84.37168810820093 > 33.8965873184253.
IsChangePoint: Marked as a change because one of 9/27/2022 11:22:50 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -315.42954157527174 (T) = (0 -83.199049558771) / Math.Sqrt((0.4863059027877561 / (26)) + (0.2684891863688202 / (39))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (39) - 2, .025) and -1.5410693136900087 = (32.74174738584902 - 83.199049558771) / 32.74174738584902 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.TryWriteBytes(address: 1020:3040:5060:7080:9010:1112:1314:1516)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 73.30686374550373 > 24.02289664826891.
IsChangePoint: Marked as a change because one of 9/27/2022 11:22:50 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -708.8002141196165 (T) = (0 -73.33756683688654) / Math.Sqrt((0.12819942483917043 / (26)) + (0.004097138068264754 / (38))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (38) - 2, .025) and -2.1851416171381857 = (23.024899879578832 - 73.33756683688654) / 23.024899879578832 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	007563949e53bad52dfcdf95927b0ce00101df4d
Compare	1f688ed75133664aa20c838fa451e2ecf9f1f424
Diff	Diff

Regressions in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Ctor_Span - Duration of single invocation	243.21 ns	392.14 ns	1.61	0.06	False
GetAddressBytes - Duration of single invocation	248.28 ns	354.23 ns	1.43	0.07	False
Ctor_Bytes - Duration of single invocation	242.96 ns	394.99 ns	1.63	0.08	False
TryWriteBytes - Duration of single invocation	222.36 ns	335.05 ns	1.51	0.08	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*'

Details

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Span(address: [16, 32, 48, 64, 80, ...])

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 392.1427433201462 > 250.2405288243561.
IsChangePoint: Marked as a change because one of 9/27/2022 11:22:50 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -45.76536029112274 (T) = (0 -402.4174632200849) / Math.Sqrt((14.251018938870484 / (26)) + (474.898411406718 / (39))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (39) - 2, .025) and -0.6826093528551538 = (239.16274002473511 - 402.4174632200849) / 239.16274002473511 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.GetAddressBytes(address: 1020:3040:5060:7080:9010:1112:1314:1516)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 354.2298052379228 > 262.21331826690056.
IsChangePoint: Marked as a change because one of 9/27/2022 11:22:50 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -31.22820454535586 (T) = (0 -363.7494285011303) / Math.Sqrt((43.32684371596665 / (26)) + (419.00453970237754 / (38))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (38) - 2, .025) and -0.44063394817017265 = (252.49261199428778 - 363.7494285011303) / 252.49261199428778 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Bytes(address: [16, 32, 48, 64, 80, ...])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 394.9946998791459 > 261.4751401554384.
IsChangePoint: Marked as a change because one of 9/27/2022 11:22:50 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -58.336651069703755 (T) = (0 -411.0494190813555) / Math.Sqrt((32.545727298506435 / (26)) + (248.45163104825795 / (38))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (38) - 2, .025) and -0.6559288406395734 = (248.22891478995854 - 411.0494190813555) / 248.22891478995854 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.TryWriteBytes(address: 1020:3040:5060:7080:9010:1112:1314:1516)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 335.0458643157161 > 231.34897044519997.
IsChangePoint: Marked as a change because one of 9/27/2022 11:22:50 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -41.21305130934805 (T) = (0 -331.0321460972355) / Math.Sqrt((12.220282934691816 / (26)) + (241.61349844599422 / (37))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (37) - 2, .025) and -0.49119060198265213 = (221.99184038385363 - 331.0321460972355) / 221.99184038385363 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	007563949e53bad52dfcdf95927b0ce00101df4d
Compare	1f688ed75133664aa20c838fa451e2ecf9f1f424
Diff	Diff

Regressions in PerfLabTests.LowLevelPerf

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
NewDelegateClassEmptyInstanceFn - Duration of single invocation	22.90 ms	24.57 ms	1.07	0.08	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'

Details

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

PerfLabTests.LowLevelPerf.NewDelegateClassEmptyInstanceFn

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.56930045714286 > 24.326842938461535.
IsChangePoint: Marked as a change because one of 9/27/2022 11:22:50 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -10.919075243687148 (T) = (0 -24388378.06581188) / Math.Sqrt((174748518699.47052 / (26)) + (422740347230.95245 / (39))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (39) - 2, .025) and -0.06307178463446259 = (22941421.6596839 - 24388378.06581188) / 22941421.6596839 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[kotlarmilos]

Looks like regression - @stephentoub could you take a look?

31e0bd1

[stephentoub]

It's definitely not 31e0bd1, which is changing Enumerable.Min/Max.

It's very likely 0075639...1f688ed, which is changing IPAddress. The change relies on Vector128, and in particular Vector128.LoadUnsafe, ShiftLeft, ShiftRight, and StoreUnsafe. Are those all appropriately optimized in whatever mono configuration this is?

[kotlarmilos]

Not sure if they are optimized for mono interpreter.

cc @BrzVlad

[kotlarmilos]

Regression in change from _numbers[i] = (ushort)(address[i * 2] * 256 + address[i * 2 + 1]); to _numbers[i] = BinaryPrimitives.ReadUInt16BigEndian(address.Slice(i * 2));

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.