fix(sqlalchemy-spanner): robust session-isolated test databases and clean lifecycle management

[chalmerlowe]

Problem

We recently updated our various system CI session to run in parallel across multiple cores. When running Cloud Spanner SQLAlchemy dialect test suites (both compliance and system tests) in parallel or across multiple Nox sessions, the test runs shared a single hardcoded database using a static configuration file (test.cfg). This caused race conditions and test pollution:

• Intermittent 404 errors due to one test session's teardown dropping the database while another session was actively running tests against it.
• Schema conflicts (400 Duplicate name in schema) when multiple test runs attempted to create identical test tables simultaneously.
• Environment state leakage across parameterized Nox sessions due to global process-wide environment variable manipulation.

Solution

This pull request introduces session-isolated test execution for SQLAlchemy Spanner:

1. Ephemeral Unique Databases: Each Nox session (compliance_test_14, compliance_test_20, migration, and system) now provisions its own dedicated Spanner database named with a timestamp and a 6-character random hex suffix (e.g., sp_test_1787306641_a3f2b1), ensuring test isolation and parallelization safety.
2. Explicit Configuration Passing: Replaced global process-wide environment mutation (os.environ.*) with explicit configuration passing via command-line arguments and subprocess env dictionaries (env={"SQLALCHEMY_SPANNER_CONFIG": config_file}), preventing state bleed across runs.
3. Modern Path Management: Adopted Python's pathlib for clean, modern file existence checks and unlinking across database creation and teardown scripts.
4. Streamlined Nox Lifecycle: Refactored Nox session execution to dispatch directly to dedicated test functions without duplicate wrapper teardown logic.

[gemini-code-assist]

Code Review

This pull request introduces isolated, parallel test sessions for the Spanner SQLAlchemy dialect by dynamically provisioning unique test databases and configuration files. Key feedback from the review highlights critical issues with duplicate database creation in create_test_database.py that leads to leaked databases, potential race conditions from using a static session metadata file (recommending session-isolated metadata files and path normalization), environment variable pollution from os.environ.setdefault in noxfile.py, and an unused configuration variable in the system test session.

[chalmerlowe]

Note

Many of the changes in this PR are focused on trying to ensure that one session does not clobber another session's database as we run sessions in parallel. So we have multiple changes similar to this, where we move away from using a single global test.cfg file that used to be overwritten with the current DB name as CI checks moved from one session to another. We move to using highly unique DB names that are handled within a session using an environment variable that is local to that session.

[chalmerlowe]

Note

We had two conflicting problems:

• If we accidentally leak Databases (i.e. if cleanup does not happen properly) then we quickly run into the 100 DB limit imposed by Spanner.
• We used to be pretty robust about deleting DBs before any session starts regardless of who created it and when. We tried to use various timings with different levels of success (If we don't wait long enough, a slow running test may have their DB deleted. If we wait too long, then DBs may start stacking up.)

This approach attempts to be a reasonable compromise:

• When a given CI check starts, we do a global delete up front when the first session starts of any DB older than 4 hours. This global delete covers all DBs regardless of name, etc and are meant specifically to clean up if we have leaks OR lingering DBs.
• For the remainder of that CI check we avoid additional global deletes to avoid clobbering.
• One reason for the four hour delay is to help ensure that manual commits early in the day don't clobber continuous OR nightly CI checks that might still be running in the early morning.

Note

Other tests in the PR are more tailored and focused as much as possible on ensuring that individual sessions don't leak.