Enhance ATA consumer-drive profile pipeline, matching, and observability #552

Summary

Add a second iteration of the ATA consumer-drive profile system focused on maintainability, match quality, observability, and controlled expansion.

The current implementation ships a validated bundled catalog, applies ATA profile overrides to SMART status evaluation and replacement-risk scoring, exposes profile usage in the replacement-risk API, and gives users a global opt-out.

The next pass should improve the data pipeline, trust signals, and operator ergonomics without weakening the current fallback behavior.

Goals

• Make profile catalog growth safer and more repeatable.
• Improve match quality and explainability.
• Expose more debugging and provenance data.
• Preserve conservative fallback behavior for unknown or low-confidence drives.

Scope

1. Build a repeatable catalog import and lint pipeline

Create a workflow that can ingest curated source data, normalize it, apply validation rules, and emit the bundled JSON catalog.

Acceptance criteria:

• A documented import or generation path exists for the catalog.
• The generated output remains the embedded runtime source of truth.
• Validation catches bad families, conflicting aliases, invalid patterns, and confidence-rule violations.
• The workflow can be run locally before merging catalog changes.

2. Add explicit match confidence metadata

Differentiate between exact model_family hits, exact normalized model_name hits, and regex or pattern fallback hits.

Acceptance criteria:

• Matching returns a machine-readable match method.
• Matching returns enough metadata to distinguish stronger and weaker matches.
• The API can report this metadata for matched ATA drives.

3. Evaluate profile-aware weight overrides

Assess whether the evidence quality is strong enough to support per-family attribute weight overrides in replacement-risk scoring.

Acceptance criteria:

• If the data is not strong enough, document the decision and do not ship weight overrides.
• If the data is strong enough, add narrowly scoped support with tests and a clear fallback path.
• Do not add implied precision without defensible source quality.

4. Expose profile provenance in the API

Add profile provenance and versioning fields so operators can inspect why a drive was scored a certain way.

Acceptance criteria:

• Replacement-risk responses can include profile source, sample count, match method, and catalog version when applicable.
• Fields are omitted or null when generic ATA logic is used.
• OpenAPI docs are updated in the same change.

5. Add an admin or debug inspection surface

Provide a way to inspect the matched profile and the override path without requiring code tracing.

Acceptance criteria:

• Add either a debug endpoint, an admin inspection view, or both.
• The inspection output shows the matched family, match method, confidence gate result, and which overrides were applied.
• The output makes fallback reasons clear when no profile is used.

6. Improve model normalization and matching

Expand vendor-aware normalization so more real-world model strings match the intended family without overusing broad regex.

Acceptance criteria:

• Add targeted normalization for common vendor naming variation.
• Add regression tests for representative real model strings.
• Avoid broad matching rules that create obvious false positives.

7. Add advanced operator override controls

Give operators a way to disable or restrict profile usage beyond the current global toggle.

Acceptance criteria:

• Support at least one additional control surface beyond the global on or off switch.
• Candidate options: per-family denylist, per-family allowlist, or per-device force-generic override.
• The precedence rules are documented.

8. Add stronger catalog regression tests

Treat the catalog itself as a first-class artifact that needs regression protection.

Acceptance criteria:

• Add tests for duplicate aliases, overlapping patterns, shadowed families, confidence failures, and expected known matches.
• Add a fixture set that exercises common matched and unmatched ATA models.
• Fail fast when catalog edits change expected matches unintentionally.

Non-goals

• Do not add live runtime fetching of external reliability datasets.
• Do not change NVMe or SCSI scoring as part of this issue unless needed for shared plumbing.
• Do not remove the existing generic ATA fallback path.

Notes

• Keep the current conservative behavior: unknown or low-confidence ATA drives must still fall back cleanly.
• Any new API fields should ship with OpenAPI updates.
• If this turns out to be too large for one change, split it into follow-up issues after design settles.