Skip to content

feat(dashboard): report per-drive filesystem free space - #1041

Merged
Starosdev merged 5 commits into
developfrom
feat/470-per-drive-filesystem-capacity
Oct 28, 2026
Merged

Starosdev merged 5 commits into
developfrom
feat/470-per-drive-filesystem-capacity

Conversation

@Starosdev

@Starosdev Starosdev commented Oct 28, 2026 •

Copy link
Copy Markdown
Owner

Summary

Scrutiny collected filesystem capacity per host, but nothing connected a filesystem to the drive underneath it, so neither the dashboard card nor the device detail page could report free space for a drive. This adds that link and surfaces it on both surfaces, which closes the remaining gap in #470 (self-test history and performance benchmarks already shipped).

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Refactoring (no functional changes)
  • CI/CD or infrastructure change

Related Issues

Closes #470

Filed separately during review, deliberately not fixed here: #1046 (collectors share an empty default host.id, so unconfigured hosts are treated as one). That collision predates this work and changing host identity needs its own migration story.

Changes Made

Resolving a mount to its drives

  • collector/pkg/blockdev walks the kernel holder tree under /sys/class/block from each mount source down to the whole disks beneath it. Partitions, LVM, md and LUKS resolve through the same walk. The walk reads sysfs rather than parsing the source device path on purpose: trimming digits off a partition name is wrong for every device whose name already ends in one (nvme0n1p2, md1p1, mmcblk0p1), and a device-mapper path such as /dev/mapper/vg-root names no disk at all.
  • An md hot spare holds none of the array's data, so each member's role is read from md/dev-<name>/state and a spare is skipped. A member whose state cannot be read is kept: over-attributing one drive is a smaller error than hiding a data-carrying drive's filesystems.
  • Loop, zram and ram devices are not reported. They are kernel-synthesized rather than drives, so Scrutiny never registers one and a link naming it could match nothing; a snap-based host would otherwise write one such row per snap package.
  • The filesystem collector uploads the resolved disk names with each snapshot, as backing_devices on FilesystemCapacity.

Storing and serving the link

  • New filesystem_devices table (migration m20261027000000), one row per (filesystem, disk) pair, because the relation is many-to-many: a disk carries several filesystems, and an LVM or md filesystem spans several disks. Links are replaced with the snapshot that owns them, so a moved mount leaves no stale attribution.
  • Purging a SMART host now deletes its filesystem capacity, host status and device link rows inside the same transaction as its devices. Left behind, they would attribute the purged host's free space to whichever drive next registered under the same device name and host id.
  • GET /api/device/:id/details carries filesystems and filesystem_status; the device summary used by GET /api/summary carries filesystems per device.
  • Per-drive reads collapse repeated mounts of one filesystem. A block device holds one filesystem, so two mounts reporting the same source are the same filesystem seen twice: a bind mount carries the underlying device as its source, and btrfs subvolume mounts repeat it. Only sources naming a block device are collapsed, since distinct mounts may legitimately share a source string such as none. The per-host capacity panel still lists every mount, which is the point there.

UI

  • The dashboard device card reports a single Free figure only when exactly one filesystem sits on the drive and that filesystem sits on no other drive. Every other case shows a mount count. Shared free space is never divided between drives, because the kernel reports no such split and any division would be invented.
  • The device detail page gains a Filesystems panel with per-mount used, free and total bytes, a usage bar carrying progressbar semantics, and a "shared with" note naming the other drives when a filesystem spans more than one. The panel renders only once a host status exists, so a deployment that never enabled the filesystem collector gains no empty card on every drive.
  • An unattributed filesystem stays unattributed. The panel explains the possible reasons (a collector older than the backing-device field, a host where sysfs cannot be read, a host id that differs between the filesystem and SMART collectors, or genuinely no mounted filesystem) instead of reporting zero bytes free.
  • The details response carries filesystem_collector_reporting, true when the filesystem collector has reported for any host. A per-host status cannot distinguish a collector running under another host id from one that was never enabled, and the two need different answers: the first gets the mismatch named, quoting the drive's host id, and the second gets no panel at all. This matters on the default configuration, where the filesystem collector maps an empty host.id to default and the SMART collector stores it unchanged, so nothing attributes until COLLECTOR_HOST_ID is set for both. The identity collision itself is [BUG] Collectors share an empty default host.id, so unconfigured hosts are treated as one #1046.
  • filesystemUsageClass moved to app/shared/filesystem-usage.utils.ts so the dashboard panel and the new detail panel cannot disagree about a filesystem's colour.

Docs

  • docs/FILESYSTEM_CAPACITY.md records the attribution model and its limits: Linux-only, md spares skipped, dm-cache and bcache cache devices still attributed the filesystem they front, btrfs multi-device attributed to one disk, repeated mounts collapsed per drive, and filesystem rows purged with their host.
  • docs/openapi.yaml documents the new fields. filesystem_status uses allOf because this document is 3.0.3, where sibling keys beside $ref are ignored.

Testing

  • I have tested this locally
  • I have added/updated tests that prove my fix/feature works
  • All existing tests pass

On this head CI has Test Frontend, Lint Backend (Go), Test Backend and every Linux, Windows and FreeBSD build green. See the CI note at the end of this body for the two checks that are not.

Locally: go build ./..., go vet ./webapp/backend/... ./collector/..., and golangci-lint run over every changed Go package are clean, with only findings that predate this branch remaining.

New tests:

  • collector/pkg/blockdev/backing_test.go builds a fake sysfs and /dev tree (two disks with partitions, an md array across both, an array with a hot spare, an LVM volume, an nvme namespace, a loop device, a mapper symlink) and covers whole disks, partitions, stacked devices, spare skipping, synthetic disks, non-block sources, unknown devices, a missing sysfs root and a cyclic holder tree.
  • collector/pkg/filesystem/detect_test.go asserts backing devices reach the snapshot and that a network mount is still reported with no attribution.
  • scrutiny_repository_filesystems_test.go covers grouping by drive, duplicate backing devices against the primary key, stale-link removal, source dedupe, and attachFilesystems (per-drive attribution, host isolation, a RAID passthrough device name matching nothing, and summaries left intact when the table is unreadable).
  • scrutiny_repository_hosts_test.go asserts a purged host's filesystem rows are gone and an unselected host's survive.
  • get_device_details_test.go covers the handler: only this drive's filesystems, no leakage from another host's drive of the same name, repeated mounts collapsed, filesystem_status omitted rather than null when a host never reported, and SMART data still served when the filesystem read fails.
  • dashboard-device.component.spec.ts covers the four card cases; detail.component.spec.ts covers the panel's helpers.

Each fix was neutered to confirm its test goes red: the partition-to-disk step, the spare filter, the stale-link delete, the source dedupe, and the purge cleanup.

Checklist

  • My code follows the project's code style
  • I have performed a self-review of my code
  • I have commented my code where necessary (particularly complex areas)
  • I have updated the documentation if needed
  • My changes generate no new warnings
  • No console.log, debug statements, or commented-out code

Additional Notes

Review: required (adds a database migration)

Rollout notes:

  • The migration is additive: it creates filesystem_devices and touches no existing table.
  • Per-drive attribution needs the filesystem collector, which stays disabled by default in omnibus deployments. Enabling it is unchanged (COLLECTOR_FILESYSTEM_CRON_SCHEDULE, COLLECTOR_FILESYSTEM_RUN_STARTUP), and this PR does not change that default.
  • A collector older than this change uploads snapshots with no backing devices. The host capacity panel keeps working and per-drive figures appear once the collector is upgraded, so server and collector may be rolled out in either order.
  • Attribution reads sysfs, so it produces results on Linux hosts only.

CI notes, neither caused by this change:

  • npm audit under Dependency Audit fails on five undici advisories inside npm's own bundled dependency tree, against a three-entry allowlist in .github/scripts/audit-release-tooling.mjs, which this PR does not touch. It currently fails on other open branches too, but the cause is advisory timing rather than anything branch-specific: develop's last Dependency Audit run passed on 2026-10-25, before these advisories were published.
  • Build darwin/amd64 and Build darwin/arm64 were cancelled on an earlier head after 15 minutes with no steps recorded, a macos-latest runner that never picked the job up rather than a build failure. Re-running them passed.
  • TestIsMountPointSuitable_RequiresHeadroomAboveTestFile in collector/pkg/performance failed on earlier heads of this branch, being environment-dependent on whether the runner's temp directory is tmpfs. fix(performance): cover the benchmark headroom rule without a filesystem #1040 has since merged and this branch now carries it, so the flake is gone. This PR touches neither that package nor its collector/pkg/mounts dependency.

develop has no required status checks, so none of these gate the merge button; they are recorded so a reviewer does not re-chase them.

Scrutiny collected filesystem capacity per host, but nothing connected a
filesystem to the drive underneath it, so neither the dashboard card nor
the device detail page could report free space for a drive.

The filesystem collector now walks the kernel holder tree under
/sys/class/block from each mount source down to the whole disks beneath
it, and uploads those disk names with the snapshot. Partitions, LVM, md
and LUKS all resolve through the same walk, which is why the walk reads
sysfs rather than parsing the source device path: trimming digits off a
partition name is wrong for every device whose name already ends in one,
and a device-mapper path names no disk at all.

The links are stored one row per (filesystem, disk) pair in a new
filesystem_devices table, because the relation is many-to-many: a disk
carries several filesystems, and an LVM or md filesystem spans several
disks. Both the device summary and the device detail response carry the
filesystems a drive backs, and the detail response also carries the
collector status for the drive's host.

A single free-space figure is reported for a drive only when exactly one
filesystem sits on it and that filesystem sits on no other drive. Every
other case shows a mount count on the card and a per-filesystem
breakdown on the detail page; shared free space is never divided between
drives, because the kernel reports no such split. A filesystem the
collector could not attribute stays attributed to nothing, and the detail
page explains which of the three possible reasons applies rather than
reporting zero bytes free.

Closes #470
Review of #1041 found two ways the per-drive figures could state something
false, and four smaller problems.

An md array lists every member under slaves, including a hot spare, so the
array's filesystem was attributed to a disk holding none of its data. The
walk now reads each member's role from md/dev-<name>/state and skips a
spare. A member whose state cannot be read is kept: over-attributing one
drive is a smaller error than hiding a data-carrying drive's filesystems.
A dm-cache or bcache cache device is still attributed the filesystem it
fronts, because the holder tree alone does not record which role a member
plays; that limit is now documented.

A block device holds one filesystem, so two mounts reporting the same
source device are one filesystem seen twice. A bind mount carries the
underlying device as its source, and btrfs subvolume mounts repeat it, so
the detail panel listed one filesystem's free space several times over and
the card fell back to a mount count for a drive with a single filesystem
on it. Per-drive reads now collapse repeated mounts of one source. The
per-host capacity panel keeps one row per mount, which is the point there,
so this is applied on the per-drive paths only. Only sources naming a block
device are collapsed: distinct mounts may legitimately share a source
string such as "none".

Also from the review:

- The Filesystems panel is rendered only once a host status exists, so a
  deployment that never enabled the filesystem collector no longer gains an
  empty card on every drive.
- The empty-list message named LVM, mdadm and LUKS but not a collector
  older than the backing-device field, which the rollout notes call the
  expected cause during a staged upgrade.
- docs/FILESYSTEM_CAPACITY.md claimed the device cards report free space,
  without the qualification that they do so only when attribution is
  unambiguous.
- The openapi document is 3.0.3, where sibling keys beside $ref are
  ignored, so filesystem_status lost its description. It is wrapped in
  allOf now.
- Moving the filesystem helpers below hasSelfTestSupport() returns that
  method's doc comment to it.
@Starosdev

Copy link
Copy Markdown
Owner Author

CI triage at head c7e27c0e: both red checks are unrelated to this PR

Recording this so nobody re-chases them.

Test Backend fails on exactly one test, TestIsMountPointSuitable_RequiresHeadroomAboveTestFile in collector/pkg/performance. The test asserts an "insufficient space" rejection, but the runner's temp directory is tmpfs, so the mount is rejected earlier with "filesystem type is tmpfs (device not mounted as a real filesystem)".

  • This PR touches neither collector/pkg/performance nor its dependency collector/pkg/mounts.
  • The same single test fails on unrelated branches (feat/494-backblaze-attribution, fix/1015-bucket-paging-offset).
  • fix(performance): cover the benchmark headroom rule without a filesystem #1040, "fix(test): skip the mount headroom test when the temp dir is tmpfs", is open against develop for exactly this, and its branch is the only recent CI run that passed.

npm audit (Dependency Audit) fails on undici advisories inside npm's own bundled dependency tree, which no change in this PR affects.

Everything this PR does touch is green on the same run:

Package Result
collector/pkg/blockdev ok, 89.4% coverage
collector/pkg/filesystem ok
webapp/backend/pkg/database ok, 59.2% coverage

Test Frontend, Lint Backend (Go), and all ten cross-platform builds are green, so the frontend build and Angular specs are verified by CI.

This stays in draft until #1040 merges and a develop merge here turns Test Backend green. The remaining gate after that is an independent review against the then-current head.

…ead paths

The remaining review findings on #1041.

Purging a SMART host left its filesystem capacity, host status and device
link rows behind, where they outlive every device they could describe.
Whichever drive next registered under the same device name on the same host
id inherited the purged host's free space. All three are now deleted inside
the same transaction as the host's devices, and the host-management test
fixture migrates those tables because they are part of a host's data.

Loop, zram and ram devices are no longer reported as backing devices. They
are kernel-synthesized rather than drives, so Scrutiny never registers one
and a link naming it could match nothing; a snap-based host was writing one
such row per snap package.

The read paths had no tests. Added coverage for the device details handler
(only this drive's filesystems, no leakage from another host's drive of the
same name, repeated mounts of one filesystem collapsed, the status key
omitted rather than null when a host never reported, and SMART data still
served when the filesystem read fails) and for attachFilesystems (per-drive
attribution, host isolation, a RAID passthrough device name matching
nothing, and summaries left intact when the table is unreadable). The
detail component's filesystem helpers are covered too.

Also from the review:

- The empty-list message now names a host id that differs between the
  filesystem and SMART collectors, which otherwise causes silent total
  non-attribution.
- The usage bar carries progressbar semantics instead of being an unlabelled
  div.
- The filesystems test file scopes its in-memory SQLite DSN by test name,
  matching the other test files in the package, so a later test using a
  second host id cannot break a neighbour.

Not taken: the collector's default empty host.id lets two hosts share an
identity, which cross-attributes filesystems between them. That predates
this work, already collides on the host capacity panel, and changing host
identity belongs in its own change.
Gating the Filesystems panel on a host status existing stopped a deployment
that never enabled the filesystem collector from gaining an empty card on
every drive, but it also silenced the case the panel most needs to report.

The filesystem collector maps an empty host.id to "default" while the SMART
collector stores it unchanged, so an omnibus deployment that enables the
collector without setting COLLECTOR_HOST_ID attributes nothing: devices
carry one host id and filesystems another. No host status exists for the
device's host id, so the gate was false, so the message written for exactly
that mismatch never rendered. The feature was inert and unexplained on the
default configuration.

The device details response now carries filesystem_collector_reporting,
true when the collector has reported for any host. That is what separates a
collector running under another host id from a collector that was never
enabled, which a per-host status cannot distinguish. The panel renders when
it is true and names the mismatch, quoting the drive's host id; with nothing
reported anywhere the panel still does not render, because there is nothing
drive-specific to say.

The underlying identity collision stays in #1046, which is where changing
host identity belongs. This only stops the UI from going quiet about it.
…-filesystem-capacity

# Conflicts:
#	webapp/backend/pkg/web/handler/get_device_details.go
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant