Full schema insertion + schema testing

Here, introduce a full version of schema injection as initially started
in #798, and exposing a new `Schema` option to make it configurable even
outside the test suite.

This involved fixing a lot of bugs from #798, and the only way to make
it possible to root them all out was to make full use of schemas across
the entire test suite. The original "test DB manager" system has been
replaced with a new `riverschematest.TestSchema` helper that generates a
schema for use with a test case or prefers an existing idle one that was
already generated for the same test run.

`TestSchema` runs migrations for generated schemas which also means we
don't need to use `testdbman` anymore, with tests capable of
bootstrapping themselves at run time. We reuse schemas to avoid this
extra work when possible, but migrating a new schema is also
surprisingly fast, taking up to 50ms, but getting more down to a stable
~10ms once things are warmed up. Here's a series of sample timings:

    $ go test ./rivermigrate -run TestMigrator/MigrateUpDefault -test.v -count 50 | grep 'migrations ran in'
        river_migrate_test.go:492: migrations ran in 45.571209ms
        river_migrate_test.go:492: migrations ran in 24.642458ms
        river_migrate_test.go:492: migrations ran in 16.749708ms
        river_migrate_test.go:492: migrations ran in 22.970375ms
        river_migrate_test.go:492: migrations ran in 16.201375ms
        river_migrate_test.go:492: migrations ran in 15.727625ms
        river_migrate_test.go:492: migrations ran in 13.291333ms
        river_migrate_test.go:492: migrations ran in 13.680708ms
        river_migrate_test.go:492: migrations ran in 14.867416ms
        river_migrate_test.go:492: migrations ran in 15.631916ms
        river_migrate_test.go:492: migrations ran in 13.873791ms
        river_migrate_test.go:492: migrations ran in 14.8645ms
        river_migrate_test.go:492: migrations ran in 14.92575ms
        river_migrate_test.go:492: migrations ran in 12.541834ms
        river_migrate_test.go:492: migrations ran in 14.753875ms
        river_migrate_test.go:492: migrations ran in 12.694334ms
        river_migrate_test.go:492: migrations ran in 13.955917ms
        river_migrate_test.go:492: migrations ran in 12.126458ms
        river_migrate_test.go:492: migrations ran in 14.095958ms
        river_migrate_test.go:492: migrations ran in 13.273375ms
        river_migrate_test.go:492: migrations ran in 13.988917ms
        river_migrate_test.go:492: migrations ran in 13.141459ms
        river_migrate_test.go:492: migrations ran in 12.394417ms
        river_migrate_test.go:492: migrations ran in 11.539208ms
        river_migrate_test.go:492: migrations ran in 11.577834ms
        river_migrate_test.go:492: migrations ran in 10.883375ms
        river_migrate_test.go:492: migrations ran in 10.547417ms
        river_migrate_test.go:492: migrations ran in 12.330375ms
        river_migrate_test.go:492: migrations ran in 11.54575ms
        river_migrate_test.go:492: migrations ran in 11.437458ms
        river_migrate_test.go:492: migrations ran in 10.957ms
        river_migrate_test.go:492: migrations ran in 10.589083ms
        river_migrate_test.go:492: migrations ran in 9.758583ms

Removal of the "test DB manager" system also means that we can ungate
test from `-p 1` because they're all able to run in parallel now. The
limiting factor I ran in is that we need to keep max pool connections
within each package's tests to a relatively modest number (I found 15
seemed to maximum success) so parallel packages don't exceed the default
Postgres configuration of 100 connections.

Something that can be kind of annoying is that in case a schema isn't
used properly somewhere in a test case (i.e. `TestSchema` is run, but
then not used), inserts/operations will go the default schema, which
will leave debris there, and that will interfere with test cases using
`TestTx` (with test DB manager, all debris would go to a different
database so you wouldn't notice). To remediate this, I've added a
cleanup hook to `TestSchema` that looks for leftovers that may have been
added to the default schema. This isn't perfect because those leftovers
may have come from another test case running in parallel or which ran
previously, but it helps to zero in on the original source of the issue.

Author: brandur

.github/workflows/ci.yaml

@@ -8,7 +8,7 @@ env:
   DATABASE_URL: postgres://postgres:postgres@127.0.0.1:5432/river_dev?sslmode=disable
 
   # Test database.
-  TEST_DATABASE_URL: postgres://postgres:postgres@127.0.0.1:5432/river_test?sslmode=disable
+  TEST_DATABASE_URL: postgres://postgres:postgres@127.0.0.1:5432/river_test?pool_max_conns=15&sslmode=disable
 
 on:
   push:
@@ -42,6 +42,7 @@ jobs:
       postgres:
         image: postgres:${{ matrix.postgres-version }}
         env:
+          # POSTGRES_INITDB_ARGS: "-c max_connections=1500"
           POSTGRES_PASSWORD: postgres
         options: >-
           --health-cmd pg_isready
@@ -62,42 +63,16 @@ jobs:
       - name: Display Go version
         run: go version
 
-      - name: Set up test DBs
-        run: go run ./internal/cmd/testdbman create
-        env:
-          PGHOST: 127.0.0.1
-          PGPORT: 5432
-          PGUSER: postgres
-          PGPASSWORD: postgres
-          PGSSLMODE: disable
-
-      - name: Test
-        working-directory: .
-        run: go test -p 1 -race ./... -timeout 2m
-
-      - name: Test cmd/river
-        working-directory: ./cmd/river
-        run: go test -race ./... -timeout 2m
-
-      - name: Test riverdriver
-        working-directory: ./riverdriver
-        run: go test -race ./... -timeout 2m
-
-      - name: Test riverdriver/riverdatabasesql
-        working-directory: ./riverdriver/riverdatabasesql
-        run: go test -race ./... -timeout 2m
-
-      - name: Test riverdriver/riverpgxv5
-        working-directory: ./riverdriver/riverpgxv5
-        run: go test -race ./... -timeout 2m
+      - name: Set up database
+        run: |
+          psql -c "CREATE DATABASE river_test" postgres://postgres:postgres@localhost:5432
+          go run github.com/riverqueue/river/cmd/river@latest migrate-up --database-url "$TEST_DATABASE_URL"
 
-      - name: Test rivershared
-        working-directory: ./rivershared
-        run: go test -race ./... -timeout 2m
+      - name: Temporary verbose jobcompleter run TODO remove
+        run: go test ./internal/jobcompleter -test.v
 
-      - name: Test rivertype
-        working-directory: ./rivertype
-        run: go test -race ./... -timeout 2m
+      - name: Test
+        run: make test/race
 
   cli:
     strategy:

.golangci.yaml

@@ -88,6 +88,7 @@ linters:
         - r
         - sb # common convention for string builder
         - t
+        - tb
         - tt # common convention for table tests
         - tx
         - w

Makefile

@@ -53,14 +53,14 @@ $(foreach mod,$(submodules),$(eval $(call lint-target,$(mod))))
 .PHONY: test
 test:: ## Run test suite for all submodules
 define test-target
-    test:: ; cd $1 && go test ./... -p 1
+    test:: ; cd $1 && go test ./...
 endef
 $(foreach mod,$(submodules),$(eval $(call test-target,$(mod))))
 
 .PHONY: test/race
 test/race:: ## Run test suite for all submodules with race detector
 define test-race-target
-    test/race:: ; cd $1 && go test ./... -p 1 -race
+    test/race:: ; cd $1 && go test ./... -race
 endef
 $(foreach mod,$(submodules),$(eval $(call test-race-target,$(mod))))