Skip to content

[Testing, CI] Increased threshold value to make Resizetizer unit tests pass on arm64 machines - #27684

Merged
rmarinho merged 4 commits into
mainfrom
unknown repository
Feb 11, 2025
Merged

[Testing, CI] Increased threshold value to make Resizetizer unit tests pass on arm64 machines#27684
rmarinho merged 4 commits into
mainfrom
unknown repository

Conversation

@ghost

@ghost ghost commented Feb 10, 2025 •
edited by ghost
Loading

Copy link
Copy Markdown

Description of Change

This pull request includes changes to update the architecture for macOS builds and adjust image error thresholds in unit tests. The most important changes are as follows:

Unit Test Adjustments:

image

Issues Fixed

Fixes #26516

@dotnet-policy-service

dotnet-policy-service Bot commented Feb 10, 2025

Copy link
Copy Markdown
Contributor

Hey there @anandhan-rajagopal! Thank you so much for your PR! Someone from the team will get assigned to your PR shortly and we'll get it reviewed.

@dotnet-policy-service dotnet-policy-service Bot added the community ✨ Community Contribution label Feb 10, 2025
@ghost
ghost requested a review from rmarinho February 10, 2025 16:25
@ghost ghost added partner/syncfusion Issues / PR's with Syncfusion collaboration area-testing Unit tests, device tests labels Feb 10, 2025
@jsuarezruiz

jsuarezruiz commented Feb 10, 2025

Copy link
Copy Markdown
Contributor

/azp run

@azure-pipelines

azure-pipelines Bot commented Feb 10, 2025

Copy link
Copy Markdown
Azure Pipelines successfully started running 3 pipeline(s).

@ghost
ghost marked this pull request as ready for review February 10, 2025 16:37
Copilot AI review requested due to automatic review settings February 10, 2025 16:37
@ghost
ghost self-requested a review as a code owner February 10, 2025 16:37
@ghost
ghost requested a review from jfversluis February 10, 2025 16:37
Copilot AI reviewed Feb 10, 2025
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comment thread eng/pipelines/handlers.yml Outdated
rmarinho
rmarinho reviewed Feb 10, 2025
View reviewed changes
Comment thread src/SingleProject/Resizetizer/test/UnitTests/BaseTest.cs
private const string TestFolderName = "Microsoft.Maui.Resizetizer.Tests";
private const string TestImagesFolderName = "imageresults";
private const double ImageErrorThreshold = 0.0027;
private const double ImageErrorThreshold = 0.27;

@rmarinho rmarinho Feb 10, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do we think about this diff @mattleibow @BretJohnson ?

@jsuarezruiz jsuarezruiz Feb 11, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@anandhan-rajagopal Could you share more details? Why required the change?

@ghost ghost Feb 11, 2025

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jsuarezruiz The issue has been reproduced on ARM-based operating systems, where the failures result from differences in image comparison, with a maximum threshold of 0.27 percent for each test. Each test fails due to image discrepancies at several values, so I selected the maximum difference observed, which appears to be related to edge pixel variations. Upon investigation, it seems that these differences are likely influenced by the screen resolution of the operating system.

@rmarinho

rmarinho commented Feb 10, 2025

Copy link
Copy Markdown
Member

Seems we have other tests failing now related with Skia cc @mattleibow

Microsoft.Maui.Graphics.Tests.SkiaSharpScenarioTests.Scenario(scenarioName: "TestPattern1")
``

@ghost

ghost commented Feb 11, 2025

Copy link
Copy Markdown
Author

Seems we have other tests failing now related with Skia cc @mattleibow

Microsoft.Maui.Graphics.Tests.SkiaSharpScenarioTests.Scenario(scenarioName: "TestPattern1")
``

@rmarinho Identical pixel-level variations in the image were observed, so a maximum threshold value of 0.07 was added for these tests.

@rmarinho

rmarinho commented Feb 11, 2025

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

azure-pipelines Bot commented Feb 11, 2025

Copy link
Copy Markdown
Azure Pipelines successfully started running 3 pipeline(s).

@rmarinho

rmarinho commented Feb 11, 2025

Copy link
Copy Markdown
Member

/azp run MAUI-public

@azure-pipelines

azure-pipelines Bot commented Feb 11, 2025

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@rmarinho
rmarinho merged commit 804a549 into dotnet:main Feb 11, 2025
@github-actions github-actions Bot locked and limited conversation to collaborators Mar 14, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@jfversluis jfversluis Awaiting requested review from jfversluis jfversluis was automatically assigned from dotnet/dotnet-maui-reviewers

Assignees

No one assigned

Labels

area-testing Unit tests, device tests community ✨ Community Contribution partner/syncfusion Issues / PR's with Syncfusion collaboration

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[testing] Resizetizer unit tests failing on ARM

3 participants

@jsuarezruiz @rmarinho