Skip to content

Refactoring/inputs jest to playwright#608

Merged
bedrich-schindler merged 1 commit into
nextfrom
refactoring/inputs-jest-to-playwright
Apr 15, 2025
Merged

Refactoring/inputs jest to playwright#608
bedrich-schindler merged 1 commit into
nextfrom
refactoring/inputs-jest-to-playwright

Conversation

@lukasbriza

Copy link
Copy Markdown
Contributor

Rewrite jest tests for Inputs components into playwright.

I have a few questions for @mbohal or @bedrich-schindler.

When running full-page screenshots (in the case of SelectField), the rendering seems inconsistent. The images for the full page are re-generated every time I run the tests with the --with-update flag. Also, when testing components within the FormLayout context, they look oddly rendered when the horizontal layout is applied.

Should we address/fix this issue directly in this PR? Do you consider it a bug?

Additionally, would you like to add tests for ButtonGroup and InputGroup contexts at the component level, similar to how we are testing the FormLayout context?

Closes #607

Comment thread tests/playwright/propTests/labelPropTest.tsx Outdated
Comment thread src/components/Toggle/__tests__/Toggle.spec.tsx Outdated
Comment thread src/components/Toggle/__tests__/Toggle.spec.tsx
Comment thread src/components/Toggle/__tests__/_propTests/disabledWithCheckedPropTest.ts Outdated
Comment thread src/components/Radio/__tests__/Radio.story.tsx Outdated
Comment thread src/components/SelectField/__tests__/SelectField.spec.tsx
Comment thread src/components/SelectField/__tests__/SelectField.spec.tsx Outdated
Comment thread src/components/SelectField/__tests__/SelectField.spec.tsx Outdated
Comment thread src/components/SelectField/__tests__/SelectField.spec.tsx Outdated
Comment thread src/components/SelectField/__tests__/SelectField.spec.tsx Outdated
@lukasbriza

Copy link
Copy Markdown
Contributor Author

I had to add await page.waitForTimeout(500); inside onBeforeSnapshot, and also include the { maxDiffPixelRatio: 0.001 } option in toMatchSnapshot within the full-page block to capture snapshots of the opened SelectField. These tests are getting rewritten on every update, but I have no idea why. I think it might be some inconsistency or an issue with the slider option and how it's rendered in the browser or Playwright.

@bedrich-schindler

Copy link
Copy Markdown
Contributor

I had to add await page.waitForTimeout(500); inside onBeforeSnapshot, and also include the { maxDiffPixelRatio: 0.001 } option in toMatchSnapshot within the full-page block to capture snapshots of the opened SelectField. These tests are getting rewritten on every update, but I have no idea why. I think it might be some inconsistency or an issue with the slider option and how it's rendered in the browser or Playwright.

@lukasbriza, I suggested change over there:

image

Did you try it?

@lukasbriza

Copy link
Copy Markdown
Contributor Author

I had to add await page.waitForTimeout(500); inside onBeforeSnapshot, and also include the { maxDiffPixelRatio: 0.001 } option in toMatchSnapshot within the full-page block to capture snapshots of the opened SelectField. These tests are getting rewritten on every update, but I have no idea why. I think it might be some inconsistency or an issue with the slider option and how it's rendered in the browser or Playwright.

@lukasbriza, I suggested change over there:
image

Did you try it?

Yes i tried it. I added { animations: "disabled" } into every screenshot method in SelectField tests and it did not worked on that issue. Snapshots are still rewritten every time. Adding { maxDiffPixelRatio: 0.001 } was only thing that helped in combination with timeout on 500ms.

Comment thread tests/playwright/propTests/helperTextAndValidationTextPropType.ts Outdated
Comment thread tests/playwright/propTests/helperTextAndValidationTextPropType.ts Outdated
Comment thread tests/playwright/propTests/validationStatePropTest.ts
@lukasbriza
lukasbriza force-pushed the refactoring/inputs-jest-to-playwright branch 2 times, most recently from 57d7825 to b576522 Compare April 1, 2025 07:05
@bedrich-schindler

bedrich-schindler commented Apr 3, 2025

Copy link
Copy Markdown
Contributor

I had to add await page.waitForTimeout(500); inside onBeforeSnapshot, and also include the { maxDiffPixelRatio: 0.001 } option in toMatchSnapshot within the full-page block to capture snapshots of the opened SelectField. These tests are getting rewritten on every update, but I have no idea why. I think it might be some inconsistency or an issue with the slider option and how it's rendered in the browser or Playwright.

I have tried several things but unfortunately, I have the same problem as you report. I do not know why it worked when I tried it with disabled animations. It may be the flakiness.

That being said, please create bug report into the Playwright. You must provide reproducible demo with the bug and detailed description what is wrong. I would recommend to paste few examples of snapshots so that they can clearly see the error. I would also inform them about our attempts (disabling animations, timeout, waiting for documents.fonts....) I already removed the reproducible repository from my Github Profile, but I have local copy of that. I will send it to you so you can just replace my issue with this bug and sent it do Playwright team so than can review that and hopefully fix that or give us a hint.

Example of the bug report: microsoft/playwright#34441

Please, link the issue into the code and comment out the full page tests that are broken due to this. Also, create issue in React UI repository that is linked to Playwright issue and that should fix those commented tests. Please, add this link into the code as well.

Base automatically changed from task/587 to next April 15, 2025 11:09
@bedrich-schindler
bedrich-schindler force-pushed the refactoring/inputs-jest-to-playwright branch from b576522 to 63101c4 Compare April 15, 2025 12:57
@bedrich-schindler
bedrich-schindler merged commit 0cfcaa8 into next Apr 15, 2025
@bedrich-schindler
bedrich-schindler deleted the refactoring/inputs-jest-to-playwright branch April 15, 2025 13:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants