make sure BindDuringTcpWait_Succeeds does not run with other tests

[wfurt]

I check run log and the test runs at the end as @bradwilson described. This should avoid conflicts with other tests.

Right now it is only one test but I put CollectionDefinition to HelperBase so it is easy to add more as we find mode cases like this.

      System.Net.Sockets.Tests.NoParallelTests.BindDuringTcpWait_Succeeds [STARTING]
      System.Net.Sockets.Tests.NoParallelTests.BindDuringTcpWait_Succeeds [FINISHED] Time: 0.0012822s
    Finished:    System.Net.Sockets.Tests
  === TEST EXECUTION SUMMARY ===
     System.Net.Sockets.Tests  Total: 920, Errors: 0, Failed: 0, Skipped: 7, Time: 6.013s

fixes #40475

[davidsh]

This seems like a useful pattern to document more broadly in our "how to write unit tests for CoreFx" documentation.

[tmds]

@wfurt I'm not sure this does what you want it to.
My understanding is this means that tests in this collection do not run in parallel.
By default the collections are determined by tests grouped in the same class.
So I think the effect of this change is BindDuringTcpWait_Succeeds runs in parallel with more tests.

[wfurt]

The trick comes from DisableParallelization = true @tmds. I was confused as well by normal documentation but the interesting part comes from https://xunit.net/releases/2.3-beta5

Added the ability to disable cross-collection parallelization for individual test collections, via the test collection definition (by setting [CollectionDefinition(DisableParallelization = true)]). Parallel-capable test collections will be run first (in parallel), followed by parallel-disabled test collections (run sequentially).

I verified that behavior and observed that my class and test will be at the end. When DisableParallelization is removed new test is mingled inside other tests. https://xunit.net/docs/running-tests-in-parallel.html does not really explain how that parameter works.

As far as the documentation: I think that needs more work in general. My plan is to wait and verify that #40475 is truly fixed and then I can update testing notes.

[bradwilson]

Collection definition classes are intended to be empty classes. I'm not sure why you've attached this to some seemingly random existing class. Create a new empty class to attach this to instead.

[wfurt]

Unless I missed something It did not seems like I can attach [Collection] attribute to specific test @bradwilson. So I still needed class where test can live.

[wfurt]

BTW I was looking for way how to update running-tests-in-parallel.html but it does not seems it lives on github. I think the fact that DisableParallelization=true collections run at the end in sequence is crucial piece of information.

[bradwilson]

It's here: https://github.com/xunit/xunit.github.io/blob/master/docs/running-tests-in-parallel.html

[bradwilson]

I meant specifically this line ([CollectionDefinition]), not the test ([Collection]).

[wfurt]

updated. And thanks for the doc pointer @bradwilson.