Repository Quality Report: Test Classification & Build Tag Hygiene (2026-03-16)

[github-actions[bot]]

🎯 Repository Quality Improvement Report — Test Classification & Build Tag Hygiene

Analysis Date: 2026-03-16
Focus Area: Test Classification & Build Tag Hygiene
Strategy Type: Custom (first run — no prior history)
Custom Area: Yes — gh-aw enforces a mandatory //go:build tag convention for all test files. The AGENTS.md explicitly documents this requirement and notes it has caused CI failures. This analysis quantifies compliance and identifies the remaining violations.

---

Executive Summary

Of 942 test files in pkg/ and cmd/, 5 files (0.5%) are missing required build tags. While the compliance rate is strong, these 5 files can cause subtle CI failures: tests may run in the wrong environment (unit vs integration), or the build tag enforcement script may silently mis-classify shared helpers. Two of the missing-tag files are shared test helpers whose comment documentation claims they are "used by both unit and integration tests" — yet without a tag they are excluded from integration test builds under the current convention.

Overall test investment is excellent: 304,295 lines of test code vs 133,672 lines of source code (2.27× test-to-source ratio). The any migration is complete (0 remaining interface{} occurrences), and only 9 TODO/FIXME comments remain in production code. The focus area for immediate action is the missing build tags.

---

Full Analysis Report

Focus Area: Test Classification & Build Tag Hygiene

Current State Assessment

Metric	Value	Status
Total test files	942	✅
Test files with correct build tags	937	✅
Test files missing build tags	5	❌
Compliance rate	99.5%	⚠️
Test-to-source line ratio	2.27×	✅
Source files (*.go, non-test)	583	✅
interface{} occurrences	0	✅
TODO/FIXME in production code	9	✅
Files exceeding 400 lines (non-test)	20	⚠️
Largest file (pkg/constants/constants.go)	1,031 lines	⚠️

Findings

Strengths

• 99.5% build tag compliance — almost all test files correctly annotated
• Exceptional test investment — more than double the source code in tests
• Complete any migration — zero remaining interface{} occurrences
• Low technical debt — only 9 TODO/FIXME comments across all production Go files
• Well-distributed test packages — all major packages have tests

Areas for Improvement

1. ❌ HIGH — 5 test files missing required //go:build tags — these violate the mandatory project convention and can silently include/exclude tests from wrong test runs
2. ⚠️ MEDIUM — Shared test helpers have ambiguous classification — test_helpers_shared_test.go (workflow) and test_helpers_git_test.go (cli) both self-describe as "shared between unit and integration tests" but naming convention routes them to //go:build !integration, making them inaccessible to integration tests
3. ⚠️ MEDIUM — pkg/constants/constants.go is 1,031 lines — significantly exceeds the 300-line refactoring guideline; may benefit from domain-based splitting
4. ℹ️ LOW — Two TODO comments in plugin_installation.go — validate CLI plugin install command syntax for Claude and Codex engines

Detailed Analysis

Missing Build Tags — File List:

File	Contains	Should Be
pkg/workflow/enable_api_proxy_test.go	Unit tests (TestEngineAWFEnableApiProxy)	//go:build !integration
pkg/workflow/docker_api_proxy_test.go	Unit tests (TestCollectDockerImages_APIProxyForEnginesWithLLMGateway)	//go:build !integration
pkg/workflow/test_helpers_shared_test.go	Helper boolPtr() — "shared by unit and integration"	Needs resolution (see Task 2)
pkg/cli/test_helpers_git_test.go	Helper initTestGitRepo() — "shared by unit and integration"	Needs resolution (see Task 2)
pkg/cli/codemod_activation_outputs_test.go	Unit tests (TestActivationOutputsCodemod)	//go:build !integration

Shared Helper Conflict:

The comments in test_helpers_shared_test.go and test_helpers_git_test.go both state they are "shared helper[s] used by both unit and integration tests." Under the current naming convention, they would receive //go:build !integration, which would exclude them from integration test builds — contradicting their stated intent. These files either need:

• A compound constraint: //go:build !integration || integration (no-op, equivalent to no tag — but satisfies the "must have a tag" rule visually), or
• Renaming to include integration in their filename (making them integration-only, but then unit tests can't use them), or
• Moving shared helpers to a non-test file in a testutil package

Large File Analysis (top 5):

File	Lines	Issue
pkg/constants/constants.go	1,031	Single file for all constants — splitting by domain would improve navigability
pkg/cli/trial_command.go	998	Complex command — may need decomposition
pkg/cli/gateway_logs.go	987	Log parsing logic — consider domain extraction
pkg/workflow/checkout_manager.go	983	Could benefit from step-level extraction
pkg/cli/logs_report.go	932	Report rendering — consider sub-renderer files

---

🤖 Tasks for Copilot Agent

NOTE TO PLANNER AGENT: The following tasks are designed for GitHub Copilot coding agent execution. Please split these into individual GitHub issues for Claude to process one at a time.

---

Improvement Tasks

Task 1: Add Missing Build Tags to Pure Unit Test Files

Priority: High
Estimated Effort: Small
Focus Area: Test Classification & Build Tag Hygiene

Description:
Three test files that contain actual test functions (not shared helpers) are missing the mandatory //go:build !integration tag required by the project convention. These files contain pure unit tests with no integration dependencies.

Files to fix:

• pkg/workflow/enable_api_proxy_test.go
• pkg/workflow/docker_api_proxy_test.go
• pkg/cli/codemod_activation_outputs_test.go

Acceptance Criteria:

• Each file starts with //go:build !integration followed by a blank line
• make test-unit still passes for all three files
• make lint passes with no new issues
• make agent-finish completes without errors

Code Region: pkg/workflow/enable_api_proxy_test.go, pkg/workflow/docker_api_proxy_test.go, pkg/cli/codemod_activation_outputs_test.go

Add the missing `//go:build !integration` build tag to three test files that are currently missing it. These files contain unit tests but were created without the required build tag.

For each file listed below, add `//go:build !integration` as the very first line, followed by a blank line, then the existing `package` declaration:

1. `pkg/workflow/enable_api_proxy_test.go` — add `//go:build !integration` as first line
2. `pkg/workflow/docker_api_proxy_test.go` — add `//go:build !integration` as first line
3. `pkg/cli/codemod_activation_outputs_test.go` — add `//go:build !integration` as first line

After adding the tags, run `make agent-finish` to verify no regressions. Do NOT add lock files to .gitignore.

---

Task 2: Resolve Shared Test Helper Build Tag Ambiguity

Priority: Medium
Estimated Effort: Medium
Focus Area: Test Classification & Build Tag Hygiene

Description:
Two test helper files describe themselves as "shared by both unit and integration tests" but lack any build tag. The current naming convention would assign //go:build !integration (since their names don't contain "integration"), which would exclude them from integration test builds — contradicting their purpose.

Files involved:

• pkg/workflow/test_helpers_shared_test.go — provides boolPtr() helper
• pkg/cli/test_helpers_git_test.go — provides initTestGitRepo() helper

Acceptance Criteria:

• Both files have a valid build tag that reflects their actual usage
• Unit tests that use these helpers still compile and pass under make test-unit
• Integration tests that use these helpers still compile and pass under make test
• The chosen approach is documented with a comment explaining the build constraint rationale
• make agent-finish completes without errors

Recommended approach:
If these helpers are genuinely needed by both unit and integration tests, use //go:build !integration || integration which satisfies the "must have a tag" convention while compiling under both modes. If they are only used by unit tests, use //go:build !integration.

Code Region: pkg/workflow/test_helpers_shared_test.go, pkg/cli/test_helpers_git_test.go

Resolve the build tag ambiguity in two shared test helper files. Both files currently have no build tag but their comments claim they are "used by both unit and integration tests".

Investigation steps:
1. Search for usages of `boolPtr(` in `pkg/workflow/` — are they called from integration test files?
2. Search for usages of `initTestGitRepo(` in `pkg/cli/` — are they called from integration test files?

Based on findings:
- If used only in unit tests: add `//go:build !integration` as first line of each file
- If genuinely used in both: add `//go:build !integration || integration` as first line of each file and update the comment to document this rationale

After making changes, run `make agent-finish` to verify no regressions.

---

Task 3: Split pkg/constants/constants.go by Domain

Priority: Medium
Estimated Effort: Large
Focus Area: Code Organization

Description:
pkg/constants/constants.go is 1,031 lines — more than 3× the 300-line refactoring guideline documented in AGENTS.md. All constants are in a single file, making navigation and comprehension difficult as the project grows. The file contains multiple distinct semantic type groups: LineLength, Version, FeatureFlag, URL, ModelName, JobName, StepID, CommandPrefix, WorkflowID, EngineName, and more.

Acceptance Criteria:

• constants.go is split into multiple domain-specific files within pkg/constants/
• Each new file is ≤300 lines
• All existing constant names and types remain unchanged (no breaking changes)
• All imports in other packages still compile correctly (no import changes needed — same package)
• make test-unit passes
• make lint passes
• make agent-finish completes without errors

Suggested split:

• constants_engine.go — EngineName type and engine constants
• constants_workflow.go — WorkflowID, JobName, StepID, CommandPrefix
• constants_versions.go — Version, ModelName, runtime version constants
• constants_features.go — FeatureFlag type and feature flag constants
• constants_network.go — URL type and URL/network constants
• constants_limits.go — LineLength and numeric limit constants

Code Region: pkg/constants/constants.go

Refactor `pkg/constants/constants.go` (currently 1,031 lines) by splitting it into domain-specific files within the same `pkg/constants/` package. The file contains multiple distinct constant groups that should be organized separately.

Steps:
1. Read the full contents of `pkg/constants/constants.go`
2. Group constants by semantic domain (engines, workflow identifiers, versions/models, feature flags, URLs, limits)
3. Create new files in `pkg/constants/` for each domain group (e.g., `constants_engine.go`, `constants_versions.go`, etc.)
4. Each new file should start with `//go:build !integration` build tag, then `package constants`
5. Move constants to appropriate files, keeping the original `constants.go` for any that don't fit a specific domain (or rename it `constants_misc.go`)
6. Run `make build` to verify compilation
7. Run `make agent-finish` to verify all tests and linting pass

Keep all type definitions with their associated constants (e.g., the `EngineName` type and `CopilotEngine`, `ClaudeEngine` etc. constants should stay together).

---

Task 4: Resolve TODO Comments in Plugin Installation Validation

Priority: Low
Estimated Effort: Small
Focus Area: Code Quality / Technical Debt

Description:
pkg/workflow/plugin_installation.go contains two TODO comments (lines 69 and 72) for validating the correct CLI plugin install command syntax for Claude and Codex engines. These represent incomplete validation logic that could allow incorrect plugin install configurations to pass validation silently.

Acceptance Criteria:

• TODO comments are resolved — either validation logic is implemented, or the comments are replaced with documented rationale for why validation is not needed/possible
• If validation is added: test cases cover both valid and invalid plugin install configurations
• make agent-finish completes without errors

Code Region: pkg/workflow/plugin_installation.go lines 69-72

Review and resolve two TODO comments in `pkg/workflow/plugin_installation.go` (around lines 69-72):

```
// TODO: validate the correct claude CLI plugin install command syntax
// TODO: validate the correct codex CLI plugin install command syntax
```

Steps:
1. Read `pkg/workflow/plugin_installation.go` to understand the current validation context
2. Determine what the correct plugin install command syntax is for Claude CLI and Codex CLI
3. Either:
   a. Implement syntax validation for each engine's plugin install command, OR
   b. If validation is genuinely not feasible (e.g., syntax varies by version), replace the TODO with a comment explaining why validation is skipped and what users should reference
4. If validation is added, add test cases in the corresponding test file
5. Run `make agent-finish` to verify

Do not add lock files to .gitignore.

---

📊 Historical Context

Previous Focus Areas

Date	Focus Area	Type	Custom	Key Outcomes
2026-03-16	Test Classification & Build Tag Hygiene	Custom	Yes	5 missing build tags identified; shared helper ambiguity documented

This is the first run. History will grow over subsequent analyses.

---

🎯 Recommendations

Immediate Actions (This Week)

1. Fix 3 pure unit test files missing //go:build !integration tags — Task 1 — Priority: High
2. Resolve shared helper ambiguity for test_helpers_shared_test.go and test_helpers_git_test.go — Task 2 — Priority: Medium

Short-term Actions (This Month)

1. Split pkg/constants/constants.go into domain-specific files — Task 3 — Priority: Medium
2. Resolve TODO comments in plugin_installation.go — Task 4 — Priority: Low

Long-term Actions (This Quarter)

1. Investigate large files (trial_command.go, gateway_logs.go, checkout_manager.go) for decomposition opportunities
2. Automate build tag compliance — add a CI lint step that fails if test files are missing build tags (prevent future regressions)

---

📈 Success Metrics

Track these metrics to measure improvement in Test Classification & Build Tag Hygiene:

• Build tag compliance: 99.5% (937/942) → 100% (942/942)
• Files with ambiguous shared helper classification: 2 → 0
• pkg/constants/constants.go line count: 1,031 → ≤300 per file
• Unresolved TODO comments in production code: 9 → ≤5

---

Next Steps

1. Review and prioritize the 4 tasks above
2. Assign Tasks 1 and 2 (quick wins) to the Copilot coding agent immediately
3. Schedule Task 3 (constants refactor) for a focused sprint
4. Track progress using the acceptance criteria checklists
5. Re-evaluate test hygiene in ~30 days after fixes are applied

---

References:
§23146230957

AI generated by Repository Quality Improvement Agent · history

• expires on Mar 17, 2026, 1:39 PM UTC

[github-actions[bot]]

This discussion has been marked as outdated by Repository Quality Improvement Agent.

A newer discussion is available at Discussion #21398.