Skip to content

Print baseline diffs on test failures#57852

Merged
jakebailey merged 4 commits intomicrosoft:mainfrom
jakebailey:baseline-diff-on-failed-test
Mar 19, 2024
Merged

Print baseline diffs on test failures#57852
jakebailey merged 4 commits intomicrosoft:mainfrom
jakebailey:baseline-diff-on-failed-test

Conversation

@jakebailey
Copy link
Member

Just on Windows/Node14, we're seeing this test flake:

92286 passing (9m)
  1 failing

  1) 
       
         unittests:: tsserver:: events:: ProjectsUpdatedInBackground
           when event handler is not set but session is created with canUseEvents = true
             without noGetErrOnBackgroundUpdate, diagnostics for open files are queued
               with --outFile setting
                 when both options are not set:
     Error: New baseline created at tests\baselines\local\tsserver\events\projectUpdatedInBackground\without-noGetErrOnBackgroundUpdate-and-when-both-options-are-not-set.js
      at writeComparison (src\harness\harnessIO.ts:1470:27)
      at Object.runBaseline2 [as runBaseline] (src\harness\harnessIO.ts:1489:9)
      at baselineTsserverLogs (src\testRunner\unittests\helpers\tsserver.ts:34:22)
      at Context.<anonymous> (src\testRunner\unittests\tsserver\events\projectUpdatedInBackground.ts:78:21)
      at processImmediate (internal/timers.js:[46](https://github.com/microsoft/TypeScript/actions/runs/8347717870/job/22847975351#step:5:47)4:21)

I'm not sure we can figure this out locally, but we can at least print out the baseline diff when tests fail.

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Mar 19, 2024
@jakebailey
Copy link
Member Author

Will make a bad change quick and then revert it to see that this works.

@jakebailey
Copy link
Member Author

Oh yeah this PR is wrong. Will fix

@jakebailey jakebailey merged commit 8fa18b2 into microsoft:main Mar 19, 2024
@jakebailey jakebailey deleted the baseline-diff-on-failed-test branch March 19, 2024 22:09
@microsoft microsoft locked as resolved and limited conversation to collaborators Oct 16, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments