Skip to content

Remote scanning via a helper binary built from disktree-core (follow-up to #101) #110

Open

Description

@A-N-uraag

Summary

A proposal for a second way to scan another machine: a small helper binary
built from disktree-core that runs on the remote host and is driven over
ssh (stdin/stdout). It would sit alongside the find-over-ssh approach in
#101, not replace it.

I'm opening this as an issue first to check the direction before writing
anything.

Existing work

What a helper binary adds

Where #101's approach stops, as I read remote.rs:

  • Hardlinks. Local scans de-duplicate through Node.inode. The find
    listing doesn't include inode or device, so remote nodes have no identity
    and a hardlinked file is counted once per link. (This could also be fixed
    inside Scan another machine over SSH, read-only, from the path field #101 by adding %D/%i to the format; I haven't tested that.)
  • Hosts without GNU find. macOS, the BSDs and Windows hosts can't be
    scanned today. disktree-core already has backends for macOS and Windows.
  • Same accounting as a local scan. Using the real scanner means sizes,
    filters, hidden-entry rules and depth limits behave identically, rather than
    being reimplemented over find output.
  • Scan speed. The core's parallel walk (rayon) runs on the remote
    instead of a single find stream.

Trade-offs

  • The helper has to be installed on the remote (or copied there). Scan another machine over SSH, read-only, from the path field #101 needs
    nothing. This is the main cost, which is why I'd treat this as an optional
    second backend.
  • It needs a small wire protocol and a version handshake, since the local UI
    and the remote helper can differ in version.
  • Node has no serde support today, so serializing it means either a feature
    flag on disktree-core or a compact format of its own.

Rough design

  • A new crate (working name disktree-agent) with a serve mode that speaks
    length-prefixed messages over stdin/stdout.
  • The local side spawns ssh <host> disktree-agent serve, so no ports are
    opened and ~/.ssh/config keeps working.
  • A first set of requests: hello (version), scan, and space. Whether the
    tree is shipped whole or fetched one level at a time (as in Add disktree-web: browser dashboard for remote/headless machines #16) is an open
    question; I'd measure on a large tree before choosing.
  • It plugs in where remote::scan does, behind the same ScanHandle.
  • Read-only to begin with, matching Scan another machine over SSH, read-only, from the path field #101. Remote deletion would be a separate,
    later discussion, because the guards in removal.rs assume a local
    filesystem.

Suggested split

  1. Protocol and agent crate, with tests and no UI changes.
  2. Hook it up as a backend, once Scan another machine over SSH, read-only, from the path field #101's seam has landed.
  3. Anything beyond read-only, if wanted.

Questions

  1. Is a helper binary something you'd want in the project, or should remote
    scanning stay on the zero-install path?
  2. If yes, is a new crate in the workspace acceptable, and would an optional
    serde feature on disktree-core be okay?
  3. Should this wait for Scan another machine over SSH, read-only, from the path field #101 (and Turn the trail into a path field to type or paste where to go #90, which it builds on) to land first?

I'm happy to start with step 1 if the direction sounds right.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions