TearDownTest called before all sub-tests are done?

[marco-alban-atomic]

Should this fork address stretchr/testify#934 such that TearDownTest gets called after all the subtests are done? The line in the readme below this header: https://github.com/huma-engineering/testify#parallel-test-suites seems to imply this is the case.

This reproducible example shows that TearDownTest is called before the subsets are complete:

package example

import (
	"math/rand"
	"testing"
	"time"

	"github.com/huma-engineering/testify/v2/suite"
)

func TestSuiteParallelSubTests(t *testing.T) {
	suite.Run(t, &Suite{})
}

type Suite struct{}

func (s *Suite) TearDownSuite(t *suite.T) {
	t.Log(">> suite tear down", time.Now().Format(time.ANSIC))
}

func (s *Suite) SetupTest(t *suite.T) {
	t.Log(">> single test setup", t.Name())
}

func (s *Suite) TearDownTest(t *suite.T) {
	t.Log(">> single test tear down", t.Name(), time.Now().Format(time.ANSIC))
}

func (s *Suite) TestOne(t *suite.T) {
	for _, v := range []string{"sub1", "sub2", "sub3"} {
		t.Run(v, func(a *suite.T) {
			a.Parallel()
			r := rand.Intn(3)
			a.Log("started to run: ", a.Name(), "will wait: ", r, time.Now().Format(time.ANSIC))
			time.Sleep(time.Duration(r) * time.Second)
			a.Log("end  run: ", a.Name(), time.Now().Format(time.ANSIC))
		})
	}
}

func (s *Suite) TestTwo(t *suite.T) {
	for _, v := range []string{"sub1", "sub2", "sub3"} {
		t.Run(v, func(a *suite.T) {
			a.Parallel()
			r := rand.Intn(3)
			a.Log("started to run: ", a.Name(), "will wait: ", r, time.Now().Format(time.ANSIC))
			time.Sleep(time.Duration(r) * time.Second)
			a.Log("end  run: ", a.Name(), time.Now().Format(time.ANSIC))
		})
	}
}

The output of a sample run is:

=== RUN   TestSuiteParallelSubTests
=== CONT  TestSuiteParallelSubTests
    suite.go:64: >> suite tear down Fri May 31 01:52:06 2024
--- PASS: TestSuiteParallelSubTests (4.00s)
=== RUN   TestSuiteParallelSubTests/All
    --- PASS: TestSuiteParallelSubTests/All (4.00s)
=== RUN   TestSuiteParallelSubTests/All/TestOne
    suite.go:64: >> single test setup TestSuiteParallelSubTests/All/TestOne
=== CONT  TestSuiteParallelSubTests/All/TestOne
    suite.go:64: >> single test tear down TestSuiteParallelSubTests/All/TestOne Fri May 31 01:52:02 2024
        --- PASS: TestSuiteParallelSubTests/All/TestOne (0.00s)
=== RUN   TestSuiteParallelSubTests/All/TestOne/sub1
=== PAUSE TestSuiteParallelSubTests/All/TestOne/sub1
=== CONT  TestSuiteParallelSubTests/All/TestOne/sub1
=== CONT  TestSuiteParallelSubTests/All/TestOne/sub1
    suite.go:64: started to run:  TestSuiteParallelSubTests/All/TestOne/sub1 will wait:  2 Fri May 31 01:52:02 2024
=== CONT  TestSuiteParallelSubTests/All/TestOne/sub1
    suite.go:64: end  run:  TestSuiteParallelSubTests/All/TestOne/sub1 Fri May 31 01:52:04 2024
            --- PASS: TestSuiteParallelSubTests/All/TestOne/sub1 (2.00s)
=== RUN   TestSuiteParallelSubTests/All/TestOne/sub2
=== PAUSE TestSuiteParallelSubTests/All/TestOne/sub2
=== CONT  TestSuiteParallelSubTests/All/TestOne/sub2
    suite.go:64: started to run:  TestSuiteParallelSubTests/All/TestOne/sub2 will wait:  2 Fri May 31 01:52:02 2024
=== CONT  TestSuiteParallelSubTests/All/TestOne/sub2
    suite.go:64: end  run:  TestSuiteParallelSubTests/All/TestOne/sub2 Fri May 31 01:52:04 2024
            --- PASS: TestSuiteParallelSubTests/All/TestOne/sub2 (2.00s)
=== RUN   TestSuiteParallelSubTests/All/TestOne/sub3
=== PAUSE TestSuiteParallelSubTests/All/TestOne/sub3
=== CONT  TestSuiteParallelSubTests/All/TestOne/sub3
=== CONT  TestSuiteParallelSubTests/All/TestOne/sub3
    suite.go:64: started to run:  TestSuiteParallelSubTests/All/TestOne/sub3 will wait:  0 Fri May 31 01:52:02 2024
    suite.go:64: end  run:  TestSuiteParallelSubTests/All/TestOne/sub3 Fri May 31 01:52:02 2024
            --- PASS: TestSuiteParallelSubTests/All/TestOne/sub3 (0.00s)
=== RUN   TestSuiteParallelSubTests/All/TestTwo
    suite.go:64: >> single test setup TestSuiteParallelSubTests/All/TestTwo
=== CONT  TestSuiteParallelSubTests/All/TestTwo
    suite.go:64: >> single test tear down TestSuiteParallelSubTests/All/TestTwo Fri May 31 01:52:04 2024
        --- PASS: TestSuiteParallelSubTests/All/TestTwo (0.00s)
=== RUN   TestSuiteParallelSubTests/All/TestTwo/sub1
=== PAUSE TestSuiteParallelSubTests/All/TestTwo/sub1
=== CONT  TestSuiteParallelSubTests/All/TestTwo/sub1
    suite.go:64: started to run:  TestSuiteParallelSubTests/All/TestTwo/sub1 will wait:  2 Fri May 31 01:52:04 2024
=== CONT  TestSuiteParallelSubTests/All/TestTwo/sub1
    suite.go:64: end  run:  TestSuiteParallelSubTests/All/TestTwo/sub1 Fri May 31 01:52:06 2024
            --- PASS: TestSuiteParallelSubTests/All/TestTwo/sub1 (2.00s)
=== RUN   TestSuiteParallelSubTests/All/TestTwo/sub2
=== PAUSE TestSuiteParallelSubTests/All/TestTwo/sub2
=== CONT  TestSuiteParallelSubTests/All/TestTwo/sub2
    suite.go:64: started to run:  TestSuiteParallelSubTests/All/TestTwo/sub2 will wait:  0 Fri May 31 01:52:04 2024
    suite.go:64: end  run:  TestSuiteParallelSubTests/All/TestTwo/sub2 Fri May 31 01:52:04 2024
            --- PASS: TestSuiteParallelSubTests/All/TestTwo/sub2 (0.00s)
=== RUN   TestSuiteParallelSubTests/All/TestTwo/sub3
=== PAUSE TestSuiteParallelSubTests/All/TestTwo/sub3
=== CONT  TestSuiteParallelSubTests/All/TestTwo/sub3
    suite.go:64: started to run:  TestSuiteParallelSubTests/All/TestTwo/sub3 will wait:  1 Fri May 31 01:52:04 2024
=== CONT  TestSuiteParallelSubTests/All/TestTwo/sub3
    suite.go:64: end  run:  TestSuiteParallelSubTests/All/TestTwo/sub3 Fri May 31 01:52:05 2024
            --- PASS: TestSuiteParallelSubTests/All/TestTwo/sub3 (1.00s)
PASS

Process finished with the exit code 0

In particular note that:

single test tear down TestSuiteParallelSubTests/All/TestOne Fri May 31 01:52:02 2024

is called before:

suite.go:64: end  run:  TestSuiteParallelSubTests/All/TestOne/sub1 Fri May 31 01:52:04 2024

[marco-alban-atomic]

any thoughts on this @denist-huma ?

[denist-huma]

Thank you for spotting this and having time to provide an example, @marco-alban-atomic!
I've used this test library in one project to speed up my tests.