Skip to content

[*] Chore: Mark additional tests as flaky from #6535 test runs - #6536

Merged
etrepum merged 2 commits into
meta:mainfrom
etrepum:add-more-flaky-tests
Aug 21, 2024
Merged

etrepum merged 2 commits into
meta:mainfrom
etrepum:add-more-flaky-tests

Conversation

@etrepum

@etrepum etrepum commented Aug 21, 2024

Copy link
Copy Markdown
Collaborator

Description

We still have a lot of tests that are flaky, which is a considerable burden on contributions and reviewing them. This is just the ones that flaked in #6535 on this particular run https://github.com/facebook/lexical/actions/runs/10481362302/job/29033662935

Test plan

Before

 9 flaky
    [chromium] › packages/lexical-playground/__tests__/e2e/CopyAndPaste/lexical/ListsCopyAndPaste.spec.mjs:111:3 › Lists CopyAndPaste › Copy and paste of partial list items into the list 
    [chromium] › packages/lexical-playground/__tests__/e2e/HorizontalRule.spec.mjs:30:3 › HorizontalRule › Can create a horizontal rule and move selection around it 
    [chromium] › packages/lexical-playground/__tests__/e2e/Links.spec.mjs:1926:3 › Links › Can handle pressing Enter inside a Link 
    [chromium] › packages/lexical-playground/__tests__/e2e/Links.spec.mjs:1960:3 › Links › Can handle pressing Enter inside a Link containing multiple TextNodes 
    [chromium] › packages/lexical-playground/__tests__/e2e/Selection.spec.mjs:877:3 › Selection › shift+arrowdown into a table, when the table is the last node, selects the whole table 
    [chromium] › packages/lexical-playground/__tests__/e2e/Selection.spec.mjs:927:3 › Selection › shift+arrowdown into a table, when the table is the only node, selects the whole table 
    [chromium] › packages/lexical-playground/__tests__/e2e/Tab.spec.mjs:20:3 › Tab › can tab + IME ─
    [chromium] › packages/lexical-playground/__tests__/e2e/TextFormatting.spec.mjs:1114:3 › TextFormatting › Regression #2523: can toggle format when selecting a TextNode edge followed by a non TextNode;  
    [chromium] › packages/lexical-playground/__tests__/regression/429-swapping-emoji.spec.mjs:21:3 › Regression test #429 › Can add new lines before the line with emoji 

After

These 9 tests are tagged @flaky

@vercel

vercel Bot commented Aug 21, 2024 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
lexical ✅ Ready (Inspect) Visit Preview 💬 Add feedback Aug 21, 2024 1:45pm
lexical-playground ✅ Ready (Inspect) Visit Preview 💬 Add feedback Aug 21, 2024 1:45pm

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 21, 2024
@etrepum etrepum added the extended-tests Run extended e2e tests on a PR label Aug 21, 2024
@github-actions

github-actions Bot commented Aug 21, 2024 •

Copy link
Copy Markdown

size-limit report 📦

Path Size
lexical - cjs 29.38 KB (0%)
lexical - esm 29.24 KB (0%)
@lexical/rich-text - cjs 37.74 KB (0%)
@lexical/rich-text - esm 31.05 KB (0%)
@lexical/plain-text - cjs 36.41 KB (0%)
@lexical/plain-text - esm 28.42 KB (0%)
@lexical/react - cjs 39.64 KB (0%)
@lexical/react - esm 32.51 KB (0%)

@etrepum

etrepum commented Aug 21, 2024

Copy link
Copy Markdown
Collaborator Author

The plain text tests are expected to fail until #6535 is merged and this is updated

@potatowagon

Copy link
Copy Markdown
Contributor

the criteria to mark flaky (decided with @Sahejkm ) was if a test failed on a PR due to flakiness. however also ok to adjust the criteria to occasionally include non failing tests with flaky runs, since it was a bother

@etrepum

etrepum commented Aug 21, 2024

Copy link
Copy Markdown
Collaborator Author

Every test that has a flaky run has a chance of failing, it's just probability. The more we have the more often it's going to be a problem.

@etrepum
etrepum added this pull request to the merge queue Aug 21, 2024
Merged via the queue into meta:main with commit 24cf9e3 Aug 21, 2024
@etrepum
etrepum deleted the add-more-flaky-tests branch August 21, 2024 23:40

This branch was successfully deployed

2 active deployments
Preview – lexical-playground — 2e24b50d Deployed Aug 21, 2024 by vercel[bot]
Preview – lexical — 2e24b50d Deployed Aug 21, 2024 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. extended-tests Run extended e2e tests on a PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants