Repository navigation
fix(smart): detect manufacturer SMART failures for ATA and NVMe and keep them through recalculation - #970
Merged
Conversation
smartctl JSON emits "now" and "past" for ata_smart_attributes when_failed, but the constants were FAILING_NOW and IN_THE_PAST, so no ATA attribute was ever flagged FailedSmart or given the past-failure warning. Switch the constants to the real values (comparison stays case-insensitive). The legacy strings are never produced by smartctl and nothing stored or tested relies on them, so they are not kept. Drives with a "past" attribute now show WarningScrutiny and drives with a "now" attribute show FailedSmart at attribute level. Stored data is not changed; status is recomputed on the next ingest from the persisted raw when_failed field.
A non-zero NVMe critical_warning now sets AttributeStatusFailedSmart as well as FailedScrutiny, and the device status gets DeviceStatusFailedSmart, so status_threshold=smart users and notifications see it. The status reason decodes the NVMe spec bits (spare below threshold, temperature, reliability degraded, read-only, volatile backup failed, PMR read-only) and flags unknown bits. percentage_used gains a warning band at 90 percent used. Failure stays at values above 100 (the existing threshold): the NVMe spec says 100 means rated endurance is consumed but is not itself a failure, and vendors may report up to 255. 90 is conservative enough to give months of notice on typical workloads without alerting on healthy mid-life drives.
…tion recalculateStatusFromSmart rebuilt device status from Passed and only propagated FailedScrutiny, so an override change dropped DeviceStatusFailedSmart. It now keeps FailedSmart from any non-ignored attribute with AttributeStatusFailedSmart, and keeps the device's existing bit (which carries the smartctl overall smart_status, not stored with the measurement) unless the only evidence for it was an attribute an override now ignores. persistRecalculatedDeviceStatus writes the exact recalculated bitset instead of only handling passed/failed transitions. ATA ingest now also sets DeviceStatusFailedSmart when a non-ignored attribute is FailedSmart (when_failed "now"), matching the NVMe path, so ingest and recalculation agree.
Ingest set DeviceStatusFailedSmart from an attribute outside the failures.transient.ata guard, so a transient attribute (default 195) with when_failed "now" failed the device. Gate it the same way as FailedScrutiny. recalculateStatusFromSmart now takes the config and skips transient ATA attributes for both failure kinds, so ingest and override recalculation agree. Also fix a stale field comment on the NVMe warning band, and add tests for persistRecalculatedDeviceStatus (clear one bit, unchanged, reset), transient and ignored ATA ingest, transient recalculation, and warn-level notification for a when_failed "past" attribute.
3 of 14 tasks
32 of 40 tasks
This was referenced Nov 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
WP-A (verdict correctness) for the 2.0.0 release. Manufacturer SMART failures that Scrutiny never flagged are now detected for ATA and NVMe, NVMe critical warnings are explained, and a device keeps its SMART-failure status when attribute overrides change.
Type of Change
Related Issues
Related to #965
Changes Made
when_failedcompared againstFAILING_NOW/IN_THE_PAST, but smartctl JSON emitsnow/past(seetestdata/smart-fail2.json:217), so attribute-level manufacturer threshold failures were never flagged. Constants now match smartctl, compared case-insensitively.nowsetsFailedSmart;pastsets a warning.critical_warning != 0now sets the attribute and deviceFailedSmartstatus, not onlyFailedScrutiny, in both NVMe status paths (respectingignoreoverrides). Status reasons decode bits 0-5 of the NVMe spec and report unknown bits.percentage_used: warning at 90 or above; above 100 stays a Scrutiny failure as before (the spec does not treat 100 itself as failure).FailedSmartwhen a non-ignored, non-transient attribute isFailedSmart, matching the NVMe path.failures.transient.ata(default[195]) is honored forFailedSmartat ingest and, for bothFailedScrutinyandFailedSmart, during override recalculation, so the two agree.recalculateStatusFromSmart(override changes) only propagatedFailedScrutiny, soFailedSmartwas dropped. It now keepsFailedSmartfrom non-ignored attributes and keeps the existing device bit (which carries smartctl's overallsmart_status, not stored with the measurement) unless the only evidence was an attribute an override now ignores.persistRecalculatedDeviceStatuswrites the exact recalculated bitset, so bits can be cleared as well as set.Testing
go build ./...,go vet ./...,go test ./...pass;GOOS=linux golangci-lint run --new-from-rev=origin/beta ./...(v2.14.0) 0 issues. New tests: real-fixture ATAwhen_failed(attribute and device status), NVMecritical_warningdecoding and device status,percentage_usedbands, notify filter withstatus_threshold=smart, status recalculation (keepsFailedSmart, ignored attribute drops it,smart_statusbit kept,FailedScrutiny-only unchanged, transient 195 skipped),persistRecalculatedDeviceStatusagainst in-memory SQLite (no-op, clear one bit, reset), ATA ingest with an ignored or transient failing attribute, and warn-level notification forwhen_failed: "past". Removing the transient guard makes the transient ingest test fail.Checklist
Notes
FailedSmartbit came fromsmart_statusand that also has an ignoredFailedSmartattribute and no other one loses the bit on recalculation, becausesmart_statusis not stored per measurement.when_failed: "past"is permanent in a drive's history and now yields a warning; with warn-level notifications and repeat notifications enabled, such a drive notifies on every run. NVMecritical_warningtemperature bit now counts as a SMART failure. Both belong in the 2.0 release notes.Review: not required (verdict logic on the
betaintegration branch; no production effect until a release, but an independent read-only review is being run per release coordination)