manifest.json uses absolute paths as keys — --update always triggers full rebuild on CI or different clone path

[varuna-zededa]

Summary

manifest.json and cache/stat-index.json store absolute file paths as keys. This means --update (incremental re-extraction) always falls back to a full rebuild when the repo is cloned to a different path — including CI environments (GitHub Actions, etc.).

Reproduction

from graphify.detect import detect_incremental
from pathlib import Path

# After a full build at /original/path, copy the repo to /new/path
# then run detect_incremental from /new/path:
result = detect_incremental(Path('.'))
print(result['new_total'])   # reports ALL files as new
print(len(result['deleted_files']))  # reports ALL original files as deleted

Confirmed by copying a 193-file corpus repo from /Users/.../device-managing-portal to /private/tmp/dmp-test:

• 193 files "deleted" (old absolute paths gone from manifest)
• 618 files "new" (new absolute paths not in manifest)
• Result: full re-extraction triggered instead of incremental

Root cause

manifest.json keys are absolute paths:

{
  "/Users/Varuna_1/git/device-managing-portal/backend/apps/__init__.py": {
    "mtime": 1783744253.177,
    "ast_hash": "d41d8cd98f00b204e9800998ecf8427e",
    "semantic_hash": "d41d8cd98f00b204e9800998ecf8427e"
  }
}

cache/stat-index.json has the same structure — absolute path as key, content hash as value — but the hash is never used as a fallback lookup when the path key doesn't match.

Impact

• CI/CD broken: GitHub Actions checks out repos at /home/runner/work/<repo>/ — never matching a developer's local path. Every CI run does a full rebuild.
• Multi-machine teams broken: developer A builds the graph, commits graphify-out/, developer B clones and runs --update — full rebuild every time.
• graph.json itself is fine: all source_file values are already relative paths. Only the incremental bookkeeping files are affected.

Expected behavior

detect_incremental() should match files by relative path from the repo root (or by content hash as a fallback), not by absolute path. This would make graphify-out/manifest.json portable across machines and clone paths.

Suggested fix

Option A — store relative paths as manifest keys:

# Instead of: str(abs_path)  →  "/Users/.../repo/backend/foo.py"
# Use:         str(abs_path.relative_to(scan_root))  →  "backend/foo.py"

Option B — hash-first lookup: when an absolute path key misses, fall back to matching by ast_hash/semantic_hash on the file content. This is more robust (survives renames too) but requires an inverted index.

Option A is simpler and fixes the CI use case completely.

Environment

• graphifyy version: pip show graphifyy → installed 2026-07-17
• Python 3.12
• macOS 25.5.0

[oleksii-tumanov]

Thanks for the detailed report.

What version does graphify --version report? Could you reproduce with a fresh graphify-out?

This appears to overlap #777 / #1125 and fixed since v0.8.31.

[HerenderKumar]

Tried to reproduce this against current v8 (post-0.9.19) to help answer the version question above — results support the overlap-with-#777/#1125 theory:

Fresh manifest, current code: built at path A (save_manifest(..., root=target), the shape every current caller uses — cli extract, watch, and the current skill runbook), keys come out relative (src/main.py). Copied the whole clone (including graphify-out/) to a different path and ran detect_incremental: 0 new, 0 deleted, all unchanged. Portable as designed.

Legacy absolute-keyed manifest (what pre-0.8.31 — or an outdated installed skill runbook that never passed root= — wrote) evaluated at a new clone path: every file reported new, and on a machine where the old absolute paths don't exist they'd all additionally report deleted — which matches the "193 deleted + 618 new" signature exactly.

So the manifest itself looks healthy on current code, and the symptom points to a graphify-out/ whose manifest predates the relativization (upgrading the package doesn't rewrite it until the next successful save — save_manifest does migrate seeded legacy rows to relative keys on its next run with root=).

One part of the report that IS still true on v8: cache/stat-index.json keys are absolute (abs_key = str(resolved) in cache.py), so a clone move silently loses the stat fastpath (full re-hash, though not re-extraction) and dead rows for the old path accumulate. That might be worth splitting into its own focused issue.