Summary
import never refreshes an existing database row. When a source file is replaced by
different content at the same path, the import itself is correct, but the row keeps the
old hash forever, so the file is re-imported on every subsequent run indefinitely.
index handles this case correctly. import does not.
Source references are permalinks against
83f5101.
Mechanism
ImportTask records the import at
ImportTask.cs#L342-L358
with a single call:
await database
.InsertAsync(
new FileRecord(file, sha256, sha1, sourceInfo.Length, sourceInfo.LastWriteTimeUtc.Ticks, false),
cancellationToken
)
and
Database.InsertAsync
is INSERT OR IGNORE on the path primary key:
INSERT OR IGNORE INTO files (path, sha256, sha1, file_size, mtime_ticks, is_processed)
VALUES (@path, @sha256, @sha1, @fileSize, @mtimeTicks, @isProcessed)
If a row already exists for that path, the write is silently discarded.
UpdateHashesAsync
and
UpdateMetadataAsync
exist on Database but are called only from
IndexTask.cs#L55
and
IndexTask.cs#L71.
ImportTask calls neither, so there is no path by which an existing row is ever corrected
during an import.
For contrast,
IndexTask.IndexFileAsync
covers all three cases explicitly: insert when absent, UpdateHashesAsync when the SHA-256
moved, UpdateMetadataAsync when only size or mtime moved.
Note that the skip decision is not affected.
Database.ResolveHashesAsync
only trusts the cached hash while size and mtime both still match, so a replaced file is
re-hashed, Sha256ExistsAsync correctly returns false, and the new content is imported.
The defect is purely in what gets recorded afterwards. ImportTask already holds cached
from the GetByPathAsync at
ImportTask.cs#L153-L155.
Reproduction
Constructed case, two distinct media files:
mkdir -p /tmp/pc-src /tmp/pc-dst
cp first.jpg /tmp/pc-src/photo.jpg
PhotoCleaner import --path=/tmp/pc-src --outpath=/tmp/pc-dst --db=/tmp/pc-dst/Import.db
sqlite3 /tmp/pc-dst/Import.db "select path, sha256 from files;"
# one row, sha256 = hash(first.jpg)
# replace the source with different content at the same path
cp second.jpg /tmp/pc-src/photo.jpg
PhotoCleaner import --path=/tmp/pc-src --outpath=/tmp/pc-dst --db=/tmp/pc-dst/Import.db
# imports second.jpg correctly
sqlite3 /tmp/pc-dst/Import.db "select path, sha256 from files;"
# still sha256 = hash(first.jpg) <-- the defect
PhotoCleaner import --path=/tmp/pc-src --outpath=/tmp/pc-dst --db=/tmp/pc-dst/Import.db
# imports second.jpg AGAIN, and will on every run from here on
Impact
Every run after the replacement re-copies the file into /Processed under a fresh
_N suffix and re-uploads it. Immich rejects the upload as a duplicate and
immich-cli --delete-duplicates cleans it up, so nothing is lost, but the work repeats
forever and the duplicate churn is invisible unless the log is read closely.
This matters most in exactly the situation where a source tree is repaired in place: a
file re-downloaded from an upstream source to its original path is permanently
un-recordable, so a repaired collection never settles.
Suggested fix
Give import the same three-way handling index already has: when the resolved hash
differs from the cached row, call UpdateHashesAsync instead of relying on an insert
that cannot land. The cached record is already in scope, so the comparison needs no
extra read.
An alternative is to make InsertAsync an upsert, but that changes behaviour for
index as well and loses the deliberate distinction between the insert and update paths,
so extending ImportTask looks like the smaller change.
Summary
importnever refreshes an existing database row. When a source file is replaced bydifferent content at the same path, the import itself is correct, but the row keeps the
old hash forever, so the file is re-imported on every subsequent run indefinitely.
indexhandles this case correctly.importdoes not.Source references are permalinks against
83f5101.Mechanism
ImportTaskrecords the import atImportTask.cs#L342-L358with a single call:
and
Database.InsertAsyncis
INSERT OR IGNOREon thepathprimary key:If a row already exists for that path, the write is silently discarded.
UpdateHashesAsyncand
UpdateMetadataAsyncexist on
Databasebut are called only fromIndexTask.cs#L55and
IndexTask.cs#L71.ImportTaskcalls neither, so there is no path by which an existing row is ever correctedduring an import.
For contrast,
IndexTask.IndexFileAsynccovers all three cases explicitly: insert when absent,
UpdateHashesAsyncwhen the SHA-256moved,
UpdateMetadataAsyncwhen only size or mtime moved.Note that the skip decision is not affected.
Database.ResolveHashesAsynconly trusts the cached hash while size and mtime both still match, so a replaced file is
re-hashed,
Sha256ExistsAsynccorrectly returns false, and the new content is imported.The defect is purely in what gets recorded afterwards.
ImportTaskalready holdscachedfrom the
GetByPathAsyncatImportTask.cs#L153-L155.Reproduction
Constructed case, two distinct media files:
Impact
Every run after the replacement re-copies the file into
/Processedunder a fresh_Nsuffix and re-uploads it. Immich rejects the upload as a duplicate andimmich-cli --delete-duplicatescleans it up, so nothing is lost, but the work repeatsforever and the duplicate churn is invisible unless the log is read closely.
This matters most in exactly the situation where a source tree is repaired in place: a
file re-downloaded from an upstream source to its original path is permanently
un-recordable, so a repaired collection never settles.
Suggested fix
Give
importthe same three-way handlingindexalready has: when the resolved hashdiffers from the cached row, call
UpdateHashesAsyncinstead of relying on an insertthat cannot land. The
cachedrecord is already in scope, so the comparison needs noextra read.
An alternative is to make
InsertAsyncan upsert, but that changes behaviour forindexas well and loses the deliberate distinction between the insert and update paths,so extending
ImportTasklooks like the smaller change.