detect_incremental reports live files as deleted when a file leaves the scan scope, and --update prunes them from the graph

[AlternateLogic]

Summary

detect_incremental derives its deleted set as "manifest key not in the scanned file set", while
save_manifest retains a manifest entry whenever "the file exists on disk". The two predicates
disagree. A file that leaves the scan scope — because .graphifyignore grew, for instance — but stays on
disk therefore satisfies the retain test and fails the deleted test, so it is reported deleted on every
subsequent run, permanently.

That set flows straight into build_merge(prune_sources=...), so a completed --update evicts live
sources from the graph. The failure is silent and looks like a normal incremental.

Mechanism

Two lines, in graphify/detect.py (0.8.33):

save_manifest, in the seed-from-existing block (detect.py:1190-1205) — retains on existence:

for f, entry in existing.items():
    ...
    if Path(f).exists():
        manifest[f] = normalised

detect_incremental (detect.py:1322) — deletes on scan membership:

current_files = {f for flist in full["files"].values() for f in flist}
deleted_files = [f for f in manifest if f not in current_files]

detect() honours .graphifyignore / .gitignore (detect.py:707-711), so scan membership can shrink
while the file is still on disk. Nothing ever removes the orphaned entry, because the only prune test is
existence.

The sink is graphify/__main__.py:4183:

prune_sources=deleted_files or None,

and the same call appears in the --update skill flow
(graphify/skills/*/references/update.md, the build_merge step).

Reproduction

1. Build a graph over a repo. The manifest now holds every scanned file.
2. Add a directory to .graphifyignore that the manifest already covers, and leave the files on disk.
3. Run the --update detect step.

deleted_files now holds every file under that prefix. Every one exists.

Observed on a repository where .graphifyignore gained three prefixes while their files stayed on disk.
The pre-run manifest held 3,808 entries; entries under the three newly-ignored prefixes numbered
620 + 612 + 6 = 1,238, which is exactly the deleted count --update reported. After 1,220 of those
files were later deleted from disk, the count fell to 18 on its own, with no fix applied — which is
the mechanism confirming itself.

A useful diagnostic:

from pathlib import Path
from graphify.detect import detect, load_manifest, _MANIFEST_PATH
cur = {f for v in detect(Path('.'))['files'].values() for f in v}
man = load_manifest(_MANIFEST_PATH, root=Path('.'))
d = [f for f in man if f not in cur]
print('deleted:', len(d), 'of which still on disk:', sum(1 for f in d if Path(f).exists()))

Any non-zero second number is a set of phantom deletions.

Two things this is NOT

Both were proposed and both are wrong, so they may save you a wrong turn:

1. Not a path-format mismatch between writers. Every in-product save_manifest call passes root=
   (watch.py:603/641/712, __main__.py:4152/4241), so the on-disk manifest is uniformly relative, and
   detect_incremental re-anchors it on read at detect.py:1268. The key sets are format-compatible.
   The arithmetic also rules it out: zero key overlap would make every scanned file new, whereas the
   failing run reported 766 changed against a 3,808-entry manifest — 67 percent overlap.
2. Not a missing relativize inside detect_incremental. It already passes root to
   load_manifest.

Suggested fix

Restrict the deleted set to entries whose file is actually gone:

deleted_files = [
    f for f in manifest
    if f not in current_files and not Path(f).exists()
]

prune_sources exists to evict nodes for files that no longer exist. A file that exists but left the
scan scope has no nodes to evict, because the graph was never built from it.

Optionally, and separately: have save_manifest also drop entries outside the current scan scope, so the
manifest stops accumulating orphans. The reader change above is the safety half; that is the cleanup
half.

Two lower-severity observations from the same investigation

1. --update's merge step calls save_manifest(incremental['files']) with no root=
   (graphify/skills/*/references/update.md), so it writes the legacy absolute-keyed manifest while
   watch.py writes the relative form. It self-heals on the next watch run and it is not the cause
   of the bug above, but the two writers disagreeing on format is confusing to anyone debugging this
   area — it sent our first two analyses down the wrong path.
2. The to_json node-count guard and _check_shrink both fired correctly and are what prevented
   real damage here. Worth keeping. Neither sits inside --update itself, so a prune arriving through
   build_merge is not covered by them.

Environment

• graphifyy 0.8.33, installed via uv tool install, Python 3.13.7
• Windows 11
• .graphifyignore present at the scan root