docs(AGENTS.md): test commands don't work, assessor checklist omits weights, and the languages coupling is undocumented

[kami619]

AGENTS.md is the contributor onboarding contract for this repo (root CLAUDE.md is a single @AGENTS.md include). Four gaps in it cost real time and produced at least one shipped bug during work on #518. Each is verified below.

1. Every documented test command fails on a fresh checkout

AGENTS.md currently documents:

pytest                             # Run tests
pytest --cov=src/agentready        # With coverage

Dev dependencies are declared in [project.optional-dependencies] under the dev extra (pyproject.toml:35-43), so a plain uv sync does not install them. All three obvious invocations fail:

$ PYTHONPATH=src python -m pytest ...
E   ModuleNotFoundError: No module named 'lizard'

$ uv sync && uv run pytest ...
E   ModuleNotFoundError: No module named 'lizard'

$ .venv/bin/python -m pytest ...
No module named pytest

What actually works:

uv sync --all-extras
.venv/bin/python -m pytest

Suggested change:

  # Development
- pytest                             # Run tests
- pytest --cov=src/agentready        # With coverage
+ uv sync --all-extras               # Required first — pytest/black/ruff are in the `dev` extra
+ .venv/bin/python -m pytest         # Run tests (bare `pytest`/`uv run pytest` miss deps)
+ .venv/bin/python -m pytest --cov=src/agentready
  black . && isort . && ruff check . # Lint

2. The "Adding Assessors" checklist omits the weight entry

The checklist has four steps and never mentions src/agentready/data/default-weights.yaml. But Scorer loads that file (services/scorer.py:28) and derives weights from it, so an assessor added by faithfully following all four steps has no weight. The file must also keep summing to 1.0, which means a new attribute has to be funded by reducing others — a non-obvious constraint that isn't stated anywhere in AGENTS.md.

Suggested change:

  3. Register in `assessors/__init__.py:create_all_assessors()`
+ 4. Add weight to `data/default-weights.yaml`, reducing others so the file still sums to 1.0
- 4. Add tests in `tests/unit/test_assessors_*.py`
+ 5. Add tests in `tests/unit/test_assessors_*.py`

3. The repository.languages coupling is undocumented — and has already caused a bug

Assessors commonly gate on repository.languages. That dict's keys come only from LanguageDetector.EXTENSION_MAP (services/language_detector.py:20-59), and two properties of that map are easy to miss:

• detection is suffix-only (path.suffix, line 107), so there is no .tf entry and an extensionless Dockerfile can never match;
• a language needs ≥3 files to register (minimum_file_threshold, line 68).

Gating on a language name absent from that map produces silently dead code — no error, no warning, the branch simply never executes. This is exactly what happened in #518: the Terraform and Dockerfile suppression patterns are unreachable, while being advertised in docs/attributes.md and in the Attribute.description that ships in every generated report.

Because the coupling is cross-module, it is invisible while editing an assessor.

Suggested change — add to ## Key Patterns:

**Language gating**: `repository.languages` keys come *only* from `LanguageDetector.EXTENSION_MAP`
(`services/language_detector.py`). Detection is suffix-only — no `.tf`, no bare `Dockerfile` — and needs
≥3 files. Gating on a name absent from that map yields silently dead code.

4. Nothing warns against fabricated Repository fixtures

The natural way to unit-test an assessor is to hand-construct a Repository. Nothing cautions that this can encode states the production pipeline cannot reach, which turns a test suite into a rubber stamp.

Concretely in #518: tests/unit/test_assessors_code_quality.py:1324 and :1344 build languages={"Terraform": 1} and languages={"Dockerfile": 1}. Both language names are unproducible (see item 3), and the count 1 is below the detector's 3-file threshold. The tests pass; the feature is inert.

Suggested change — extend the Tests convention:

- **Tests**: All new assessors require unit tests. Maintain >80% coverage for new code.
+ **Tests**: All new assessors require unit tests. Maintain >80% coverage for new code.
+ Don't hand-build `Repository(languages=...)` fixtures the detector can't produce — it makes dead code look tested.

Optional follow-up

A cheap structural guard for item 3, worth considering alongside the doc change — any assessor with a language-keyed table gets a test asserting its keys are producible:

assert set(_SUPPRESSION_PATTERNS) <= set(LanguageDetector.EXTENSION_MAP.values())

That converts an invisible cross-module coupling into a CI failure.

Scope

Items 1–4 are edits to AGENTS.md only. The #518-specific defects referenced as evidence are tracked on that PR and are not in scope here.

---

_{Filed with AI assistance (Claude Opus 4.6 via Claude Code). All commands and line references above were executed/verified against main and branch 510.}