Skip to content

Retire stale source paths when watched files move between directories - #1238

Merged
mmguero merged 1 commit into
cisagov:contributor-prsfrom
sylvesterkaczmarek:fix/forget-moved-watcher-source-paths-20261015
Oct 15, 2026
Merged

mmguero merged 1 commit into
cisagov:contributor-prsfrom
sylvesterkaczmarek:fix/forget-moved-watcher-source-paths-20261015

Conversation

@sylvesterkaczmarek

Copy link
Copy Markdown
Contributor

Description
A FileMovedEvent removes the tracked source path only when source and destination have the same parent directory. With recursive watching or multiple watched directories, moving a queued file between subdirectories leaves its old name in the processing queue or staging history. The worker can then invoke the processor on a nonexistent source path as well as the destination.

Retire any tracked source from both queues before handling the destination, irrespective of the parent directory. Moves from an untracked source remain harmless because removal uses pop with a default. The staging-event fix already present on contributor-prs is preserved.

Related issue(s)
New reproducible regression found on current contributor-prs. No existing tracking issue or overlapping open PR was found. This branch is based directly on contributor-prs, including its previously merged watcher fixes.

Type of change

  • Bug fix

How has this been tested?
python -m pytest -q shared/tests/test_watch_moved_paths.py

12 cases pass on macOS, Python 3.12 and watchdog 6.0.0. Eight cases fail against unchanged contributor-prs. Tests import the actual watcher and worker, move real files inside temporary directories, deliver real watchdog event objects, and check both pending and staged queues. Coverage includes same-directory renames, cross-directory moves after created/modified/closed events, unrelated queued files, and worker callbacks in polling and native-event modes. Worker scheduling is controlled in the tests; no full Malcolm container deployment or live observer was started.

Black (120-column, skip string normalization), Python compilation and git diff --check pass. Ruff correctness checks pass with the existing unused defaultdict import left unchanged.

Documentation

  • No documentation update is needed; this restores the intended move-handling behavior without changing configuration.

Was AI used in writing the code, tests, or description for this pull request?

  • Yes

OpenAI ChatGPT generated the patch, tests and this description. Validation included inspecting the diff against current contributor-prs, checking open and previously merged contributions for overlap, reproducing the regression before the fix, and running the actual test suite after it. Maintainer review is still required.

@mmguero mmguero self-assigned this Oct 15, 2026
@mmguero mmguero added the upload Relating to PCAP and/or Zeek log ingestion label Oct 15, 2026
@mmguero mmguero added this to the v26.10.0 milestone Oct 15, 2026
@mmguero mmguero added this to Malcolm Oct 15, 2026
@mmguero mmguero moved this to Review in Malcolm Oct 15, 2026
@mmguero
mmguero merged commit 03a0496 into cisagov:contributor-prs Oct 15, 2026
@github-project-automation github-project-automation Bot moved this from Review to Done in Malcolm Oct 15, 2026
@mmguero mmguero mentioned this pull request Oct 29, 2026
@mmguero mmguero moved this from Done to Released in Malcolm Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

upload Relating to PCAP and/or Zeek log ingestion

Projects

Status: Released

Development

Successfully merging this pull request may close these issues.

2 participants