Testing Strategy and Policy

Testing Strategy and Policy

Purpose: Practical testing guide for RiceCoder contributors.

---

Test Organization Policy

1. Unit Tests: In-Module (#[cfg(test)])

Location: Same file as implementation code

When to use:

• Testing individual functions and structs
• Testing internal logic and edge cases
• Testing private APIs

Structure:

// src/module.rs

pub fn my_function(input: &str) -> String {
    // implementation
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_my_function_basic() {
        assert_eq!(my_function("input"), "expected");
    }

    #[test]
    fn test_my_function_edge_case() {
        assert_eq!(my_function(""), "");
    }
}

Benefits:

• Tests live near code
• Easy to test private functions
• Clear module scope

---

2. Integration Tests: tests/ Directory

Location: crates/<crate-name>/tests/ or workspace tests/

When to use:

• Testing public APIs
• Testing multi-module interactions
• Testing cross-crate functionality
• Testing end-to-end workflows

Structure:

crates/ricecoder-application/
├── src/
│   └── services/
│       └── session_service.rs     # Unit tests here
└── tests/
    └── service_tests.rs            # Integration tests here

Example:

// tests/service_tests.rs

use ricecoder_application::services::SessionService;

#[test]
fn test_session_creation() {
    let service = SessionService::new();
    let session = service.create_session("test-user");
    assert!(session.is_ok());
}

Benefits:

• Tests public APIs only
• Simulates real usage
• Catches integration bugs

---

3. Property Tests: Using proptest

Location: In-module (#[cfg(test)]) or dedicated *_properties.rs files

When to use:

• Testing invariants across many inputs
• Testing edge cases automatically
• Validating data consistency

Dependencies: Already in workspace Cargo.toml:

[dependencies]
proptest = { workspace = true }

Example:

use proptest::prelude::*;

proptest! {
    #[test]
    fn test_roundtrip_serialization(data: Vec<u8>) {
        let encoded = encode(&data);
        let decoded = decode(&encoded)?;
        assert_eq!(data, decoded);
    }
}

Benefits:

• Finds edge cases automatically
• Validates invariants
• Reduces test maintenance

---

Test Quarantine Policy

Source: .ai/config/test-quarantine.toml

Purpose

Prevent flaky tests from blocking CI/CD while tracking them for eventual fixes.

Quarantine Rules

Setting	Value	Description
Attempt Threshold	3	Fix attempts before quarantine
Expiration Days	7	Days before retry
Max Cycles	3	Cycles before human review
Auto-Skip Default	true	Skip in default runs
Include in Full Runs	true	Run in full test suite

Workflow

1. Test fails repeatedly (3+ times)
2. Auto-quarantine - Test marked as quarantined
3. Default runs skip - cargo test skips quarantined tests
4. Full runs include - cargo test -- --include-ignored runs all tests
5. Expiration - After 7 days, test is retried automatically
6. Escalation - After 3 cycles, test flagged for human review

Commands

# Default test run (skips quarantined)
cargo test

# Full test run (includes quarantined)
cargo test -- --include-ignored

# Run specific quarantined test
cargo test quarantined_test_name -- --ignored

# Check quarantine status
cat .ai/test-quarantine-registry.yaml

Registry Location

Quarantined tests tracked in: .ai/test-quarantine-registry.yaml

---

Coverage Expectations

By Layer (Domain-Driven Design)

Layer	Min Coverage	Target	Focus
Domain	85%	90%	Business logic, entities, value objects
Application	80%	85%	Use cases, services, DTOs
Infrastructure	70%	75%	Repositories, external APIs, I/O
Presentation	60%	70%	CLI, TUI, handlers

By Crate Type

Crate Type	Min Coverage	Example Crates
Core Logic	85%	ricecoder-domain, ricecoder-application
Integration	75%	ricecoder-mcp, ricecoder-github
Infrastructure	70%	ricecoder-storage, ricecoder-persistence
UI/CLI	65%	ricecoder-cli, ricecoder-tui

Current Status

Overall Coverage: ~85%+ (Phase 7 complete)

---

How to Run Tests

Basic Commands

# Run all workspace tests
cargo test --workspace

# Run tests for specific crate
cargo test --package ricecoder-application

# Run specific test
cargo test test_session_creation

# Run with output
cargo test -- --nocapture

# Run with parallel control
cargo test -- --test-threads=4

Advanced Commands

# Run with coverage (requires cargo-llvm-cov)
cargo llvm-cov --workspace

# Run only unit tests
cargo test --lib

# Run only integration tests
cargo test --test '*'

# Run only doc tests
cargo test --doc

# Run property tests
cargo test --features proptest

Watch Mode (Development)

# Install cargo-watch
cargo install cargo-watch

# Auto-run tests on file changes
cargo watch -x test

# Run specific crate tests on changes
cargo watch -x 'test --package ricecoder-domain'

---

CI/CD Test Requirements

GitHub Actions Workflow

Location: .github/workflows/ci.yml (if enabled)

Required Checks

1. Unit Tests - Must pass in all workspace crates
2. Integration Tests - Must pass in tests/ directory
3. Property Tests - Must pass with proptest enabled
4. Coverage Report - Must meet minimum thresholds
5. Clippy Warnings - Zero warnings allowed
6. Rustfmt - Code must be formatted

Pipeline Stages

# Example CI pipeline
stages:
  - name: Build
    command: cargo build --workspace

  - name: Test
    command: cargo test --workspace

  - name: Coverage
    command: cargo llvm-cov --workspace --lcov --output-path coverage.lcov

  - name: Clippy
    command: cargo clippy --workspace -- -D warnings

  - name: Format
    command: cargo fmt --all -- --check

Quarantined Tests in CI

• Default runs - Skip quarantined tests
• Full runs - Include quarantined tests for tracking
• Nightly builds - Run full suite with quarantined tests

---

Performance Validation

Performance Targets

Metric	Target	Command
Startup Time	< 3s	./scripts/run-performance-validation.sh
Response Time	< 500ms	Included in validation
Memory Usage	< 300MB	Included in validation

Baseline Tracking

# Run performance validation
./scripts/run-performance-validation.sh

# Update baselines (after performance improvements)
./scripts/update-performance-baselines.sh

# Check for regressions
ricecoder-performance check-regression \
  --binary ./target/release/ricecoder \
  --baseline performance-baselines.json

Baseline File: performance-baselines.json (workspace root)

---

Test Debugging

Common Issues

"Test failed but passes locally"

Causes:

• Timing issues (flaky tests)
• Environment differences
• Race conditions

Solutions:

1. Add explicit waits/timeouts
2. Use tokio::time::sleep for async tests
3. Quarantine if consistently flaky

"Property test fails randomly"

Causes:

• Non-deterministic behavior
• Seed-dependent failures

Solutions:

1. Run with fixed seed: PROPTEST_CASES=1000 cargo test
2. Add invariant checks
3. Simplify property

"Integration test timeout"

Causes:

• Slow external services
• Resource exhaustion

Solutions:

1. Use test doubles/mocks
2. Increase timeout: #[tokio::test(timeout = 5000)]
3. Run in isolation

---

Best Practices

1. Test Naming

// Good: Descriptive names
#[test]
fn test_create_session_with_valid_user_id() { }

// Bad: Generic names
#[test]
fn test1() { }

2. Test Organization

// Group related tests in modules
#[cfg(test)]
mod session_tests {
    use super::*;

    mod creation {
        use super::*;

        #[test]
        fn test_creates_with_default_config() { }

        #[test]
        fn test_fails_with_invalid_user() { }
    }

    mod deletion {
        use super::*;

        #[test]
        fn test_deletes_existing_session() { }
    }
}

3. Test Data

// Use builders for complex test data
struct SessionBuilder {
    user_id: String,
    config: Config,
}

impl SessionBuilder {
    fn new() -> Self {
        Self {
            user_id: "test-user".into(),
            config: Config::default(),
        }
    }

    fn with_user(mut self, user_id: impl Into<String>) -> Self {
        self.user_id = user_id.into();
        self
    }

    fn build(self) -> Session {
        Session::new(self.user_id, self.config)
    }
}

#[test]
fn test_example() {
    let session = SessionBuilder::new()
        .with_user("custom-user")
        .build();
    // ...
}

4. Async Tests

// Use tokio::test for async tests
#[tokio::test]
async fn test_async_operation() {
    let result = async_function().await;
    assert!(result.is_ok());
}

// Use timeout for flaky async tests
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_with_timeout() {
    tokio::time::timeout(
        Duration::from_secs(5),
        async_operation()
    ).await.expect("timeout");
}

---

Resources

• Workspace Structure: See README.md for crate organization
• Test Quarantine Config: .ai/config/test-quarantine.toml
• Performance Baselines: performance-baselines.json
• Quarantine Registry: .ai/test-quarantine-registry.yaml
• Rust Testing Guide: https://doc.rust-lang.org/book/ch11-00-testing.html
• Proptest Documentation: https://docs.rs/proptest/latest/proptest/

---

Quick Reference

# Essential commands
cargo test --workspace                 # Run all tests
cargo test --package <crate>          # Run crate tests
cargo test -- --include-ignored       # Include quarantined tests
cargo llvm-cov --workspace            # Generate coverage

# Performance validation
./scripts/run-performance-validation.sh

# Watch mode
cargo watch -x test

# Debugging
cargo test -- --nocapture             # Show println! output
RUST_BACKTRACE=1 cargo test          # Show backtraces

---

Last Updated: Phase 7 (v0.1.7) - December 9, 2025