[Coverage] Phase 5: Fix Critical Failing Tests (42 failures → 0)

[jeremyeder]

Cold-Start Issue: Phase 5 - Fix Critical Failing Tests (42 failures → 0)

Current Status (2025-11-23)

Test Results: 42 failed, 341 passed, 16 skipped
Current Coverage: 54% (target: 90%)
Root Cause: Test fixtures using old Assessment JSON schema format (missing 'config' field)

Objective

Fix the 42 failing tests that are blocking 90% coverage threshold by updating test fixtures to match current Assessment model schema.

Estimated Time: 30-45 minutes
Priority: P1 (critical for 90% coverage)

---

Analysis

Coverage Impact

Critical under-covered modules that will benefit from test fixes:

• cli/learn.py: 65% → 95%+ (13 failing tests)
• cli/align.py: 40% → 90%+ (10 failing tests)
• services/learning_service.py: 30% → 85%+ (8 failing tests)

Failing Test Breakdown

High Impact (31 failures - directly affect coverage):

• tests/unit/test_cli_learn.py: 13 failures
• tests/unit/test_cli_align.py: 10 failures
• tests/unit/test_learning_service.py: 8 failures

Medium Impact (11 failures):

• tests/unit/test_cli_validation.py: 5 failures
• tests/unit/test_code_sampler.py: 1 failure
• tests/unit/learners/test_llm_enricher.py: 2 failures
• tests/unit/test_research_formatter.py: 2 failures
• tests/integration/test_research_cli.py: 2 failures

---

Root Cause

All failing tests are using old Assessment JSON schema format. The Assessment model now requires:

Missing Field: config (required, can be null)

Example of broken fixture:

{
    "schema_version": "1.0.0",
    "timestamp": "2025-11-22T06:00:00",
    "repository": {...},
    "overall_score": 85.0,
    "findings": [...],
    "duration_seconds": 1.5
    // MISSING: "config": null
}

Current Assessment.to_dict() includes:

{
    "config": self.config.to_dict() if self.config else None,
    # ... other fields
}

---

Solution: Shared Fixture Module

Create tests/fixtures/assessment_fixtures.py with factory functions for valid test data.

Implementation Plan

Step 1: Create shared fixtures module

"""Shared test fixtures for creating valid Assessment JSON data."""

def create_test_assessment_json(overall_score=85.0, num_findings=2):
    """Create valid Assessment JSON matching current schema.
    
    Returns dict that can be serialized to JSON and loaded by LearningService.
    """
    return {
        "schema_version": "1.0.0",
        "timestamp": "2025-11-22T06:00:00",
        "repository": {
            "name": "test-repo",
            "path": "/tmp/test",
            "url": None,
            "branch": "main",
            "commit_hash": "abc123",
            "languages": {"Python": 100},
            "total_files": 10,
            "total_lines": 500,
        },
        "overall_score": overall_score,
        "certification_level": "Gold",
        "attributes_assessed": num_findings,
        "attributes_not_assessed": 0,
        "attributes_total": num_findings,
        "findings": [create_test_finding_json() for _ in range(num_findings)],
        "config": None,  # CRITICAL: Must be present
        "duration_seconds": 1.5,
        "discovered_skills": [],  # Optional but good to include
    }

def create_test_finding_json(status="pass", score=90.0):
    """Create valid Finding JSON."""
    return {
        "attribute": {
            "id": "test_attr",
            "name": "Test Attribute",
            "category": "Documentation",
            "tier": 1,
            "description": "Test description",
            "criteria": "Test criteria",
            "default_weight": 1.0,
        },
        "status": status,
        "score": score if status in ("pass", "fail") else None,
        "measured_value": "present",
        "threshold": "present",
        "evidence": ["Test evidence"],
        "error_message": None,
    }

Step 2: Update all test files to use fixtures

from tests.fixtures.assessment_fixtures import create_test_assessment_json

def test_something(temp_repo):
    assessment_data = create_test_assessment_json(overall_score=95.0)
    
    # Write to file
    assessment_file = temp_repo / ".agentready" / "assessment-latest.json"
    with open(assessment_file, "w") as f:
        json.dump(assessment_data, f)

Step 3: Run tests to verify fixes

# Run previously failed tests
pytest --lf -v

# Check coverage improvement
pytest --cov=src/agentready --cov-report=term --cov-branch

---

Files to Update

Priority 1 (High Impact - 31 tests)

1. tests/unit/test_cli_learn.py - Update all 13 fixtures
2. tests/unit/test_cli_align.py - Update all 10 fixtures
3. tests/unit/test_learning_service.py - Update all 8 fixtures

Priority 2 (Medium Impact - 9 tests)

4. tests/unit/test_cli_validation.py - Update 5 fixtures
5. tests/unit/test_code_sampler.py - Update 1 fixture
6. tests/unit/learners/test_llm_enricher.py - Update 2 fixtures
7. tests/unit/test_research_formatter.py - Fix 2 formatter tests (different issue)
8. tests/integration/test_research_cli.py - Fix 2 integration tests (different issue)

---

Expected Outcome

✅ 42 failures → 0 failures
✅ Coverage: 54% → 90%+
✅ Prevention: Shared fixtures prevent future schema breakage
✅ Maintainability: Single source of truth for test data

---

Acceptance Criteria

• Create tests/fixtures/assessment_fixtures.py with factory functions
• Update all 42 failing tests to use new fixtures
• All tests pass: pytest exit code 0
• Coverage ≥ 90%: pytest --cov=src/agentready --cov-fail-under=90
• No test takes > 2 seconds (keep tests fast)
• Test suite completes in < 30 seconds total

---

Updated: 2025-11-23 by Claude Code
Analysis: Issue #109 ready for implementation

[github-actions]

🎉 This issue has been resolved in version 1.27.1 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀