Skip to content

chore: organize tests and CI by subsystem #113

Closed

Description

@abruption

The test suite is healthy but increasingly difficult to navigate: 23 Python test modules contain about 5,811 lines in one directory, with test_session_peer.py around 1,228 lines and test_codex.py around 764 lines. CI selects some optional suites by manually listing module names while the base matrix uses broad discovery. This makes ownership and coverage boundaries harder to see as relay, MCP, adapters and packaging expand.

Reorganize tests and CI by subsystem without changing product behavior.

Proposed layout

tests/
  core/
  adapters/
  codex/
  relay/
  control_integration/
  mcp/
  packaging/
  docs/
  fixtures/

The exact names may change, but each suite must have one clear owner and invocation contract.

Scope

  • Split oversized test modules along existing behavioral boundaries.
  • Move relay prototype assertions retained by chore: separate relay prototype history, deployment examples, and operator docs #111 into the product relay suites rather than creating a permanent prototype group.
  • Replace fragile hand-maintained CI module lists with documented suite entry points or discovery roots.
  • Keep fast core tests separate from dependency-bearing relay/MCP/control integration tests.
  • Add explicit distribution-content checks for intended wheel/sdist files and excluded prototype/secret/state artifacts.
  • Preserve fixture privacy and deterministic no-network defaults.

Acceptance criteria

  • python -m unittest discover -s tests still provides a useful complete local default with documented optional-dependency behavior.
  • CI jobs map directly to named directories or stable suite commands.
  • No test is silently dropped; before/after test inventory is recorded in the PR.
  • Existing OS and Python/Node matrices continue to pass.
  • Real-network, OAuth, model and production-service tests remain opt-in and cannot run from default discovery.
  • Package tests verify the complete intended artifact contract rather than only a few sentinel files.
  • Documentation translation checks remain fast and independently runnable.

Non-goals

This can follow #111 and may proceed before #112 if it avoids renaming source-facing fixtures twice.

Activity

  1. added this to the v1.0.0 milestone on Sep 20, 2026
  2. abruption commented on Sep 20, 2026

    @abruption
    OwnerAuthor

    Target release: v1.0.0 stable. Implement this before the core extraction in #112 so the refactor starts with explicit subsystem suites and a recorded before/after test inventory. File moves must preserve the alpha CLI and keep all live-network/OAuth/model tests opt-in.

  3. abruption commented on Sep 22, 2026

    @abruption
    OwnerAuthor

    Implementation review is now in #133.

    Two proposed directory names were deliberately adjusted while preserving the requested ownership boundaries:

    • tests/mcp became tests/mcp_integration
    • tests/packaging became tests/distribution

    With the required python -m unittest discover -s tests command, the start directory is placed on sys.path; the shorter names shadow the installed mcp and packaging libraries. The collision was reproduced during the reorganization, and the non-conflicting names restore the exact default-discovery contract.

    Inventory evidence: 439 existing test methods before, 442 after, zero removed, and exactly three new distribution-contract tests. Runtime default discovery moved from 365 tests/10 skips to 368 tests/13 skips for those opt-in artifact checks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions