Skip to content

An import that re-states identical content reports it as modified #373

Description

@DutchJaFO

Description

An import does not account for every item that arrives in it. Two failure modes with one cause:

  • A quote already stored is reported as modified. ImportActionPlanner computes
    effectiveChanged — the fields whose resolved value differs from the stored one, compared through
    FieldMergeResolver.ValuesEqual — and records a Modify even when that set is empty and nothing
    would be written.
  • Every other entity type is not reported at all. The planner emits actions for "the Quote itself
    and any not-yet-existing Source/Character/Person it references", and every not-yet-existing
    StageDirection/SoundCue/Conversation. An entity that already exists produces no action, so it appears
    nowhere in the report.

The counts reach the seed log, the /import and /admin/database/seed/preview responses, and #302's
per-file notification. An operator whose reseed changed nothing is told their whole dataset was
rewritten, and told nothing whatsoever about six of the seven entity types involved.

Two different nothings have to be distinguishable (developer, 2026-09-02): "we need to know if 'no
changes' happened because there was no content at all or because content was identical to what already
existed."

Every incoming item is accounted for, of every type (developer, 2026-09-02): "if cold start says
'7 characters, 13 quotes, etc added' then the reseed should say '7 characters unchanged, 13 quotes
unchanged etc.' … hiding such details would have us chase non-existent bugs."

This is what makes a reseed safe when the sources change (developer, 2026-09-02). Bundled content
is refreshed from upstream, and after a refresh the operator needs to see exactly which items the new
version added or altered, distinguished from the ones it left alone. Without a row per incoming entity
there is nothing to distinguish them by, and the operator has to guess which of their data a reseed
touched — which is the reason to run one at all.

Not reachable before #372. A reseed truncated first, so every row was always an Add. #372 makes a
reseed an ordinary import against existing content, which is what exercises this path.

Reproduction steps

dotnet script scripts/testing/test-env.csx -- create --name qt-373 --port 19523 `
  --image quotinator:local --env Quotinator__AdminApiKey=t2-373
$headers = @{ "X-Api-Key" = "t2-373" }
while ((Invoke-RestMethod "http://localhost:19523/api/v1/quotes?page=1&pageSize=1").totalCount -lt 1) { Start-Sleep 2 }

# The cold start has already imported every file. Reseed the same, unchanged files and read the report.
Invoke-RestMethod -Method Post -Headers $headers `
  "http://localhost:19523/api/v1/admin/database/reseed" |
  Select-Object -ExpandProperty reports | ConvertTo-Json -Depth 5

Requires #372 (a reseed that does not truncate first). Against a build without it the reseed empties
the database, and every row is legitimately an Add.

Expected behaviour

Every incoming item produces an accounted-for outcome, of every entity type. A row whose
effectiveChanged set is empty is Unchanged, not Modify; an already-existing Source, Character,
Person, Series, Universe, StageDirection, SoundCue or Conversation is Unchanged rather than absent.
The report states how many items arrived per entity type alongside what became of them, so a reseed of
files matching what is stored reads 7 characters incoming, 7 unchanged; 13 quotes incoming, 13
unchanged
— and after a source refresh, exactly which items are new or altered.

Actual behaviour

Measured 2026-09-02, cold start followed by one reseed of the same three bundled files. Quotes are
miscounted; every other type disappears.

File Cold start Reseed
quotinator-curated.json Character +7, Conversation +4, Person +3, Quote +13, SoundCue +1, Source +7, StageDirection +2 Quote +0 ~13 — and nothing else
NikhilNamal17_popular-movie-quotes.json Quote +687 ~45, Source +389 ~46 Quote +0 ~732
vilaboim_movie-quotes.json Quote +99, Source +86 Quote +0 ~99

Nothing changed between the two runs. The second says all 844 quotes were updated, and says nothing at
all about the seven Characters, three People and seven Sources that arrived and were already correct.

Each file also produces a second, differently-shaped #302 confirmation, because the breakdown is part
of a confirmation's identity — so the notification list grows on every reseed.

Failing tests

All need to be written; none exists today. The full row-by-row table, including the control each
positive assertion needs, is in
the plan doc.

Test class Test method Status before fix
DatabaseInitializerOwnershipTests CHECK-constraint and structural drift tests, extended with Unchanged ❌
ImportActionPlannerTests ReimportingIdenticalContent_ReportsUnchangedNotModified ❌
ImportActionPlannerTests ChangedContent_StillReportsModified — the control; a planner calling everything unchanged passes without it ❌
ImportActionPlannerTests AbsentContent_ReportsNewNotUnchanged — the other of the two nothings ❌
ImportActionPlannerTests ExistingReferencedEntities_AreReportedUnchanged — Source, Character, Person ❌
ImportActionPlannerTests ExistingCompositeEntities_AreReportedUnchanged — StageDirection, SoundCue, Conversation ❌
ImportActionPlannerTests AbsentReferencedEntities_AreStillAdded — the control against breaking creation and insertion order ❌
ImportActionReportBuilderTests Incoming_EqualsTheSumOfEveryOutcome ❌
DatabaseInitializerTests Reseed_ReportsEveryEntityTypeTheColdStartDid ❌
DatabaseInitializerTests Reseed_AgainstCurrentContent_WritesOneConfirmationPerFile — not two ❌
ImportEndpointTests Import_OfIdenticalContent_ReportsUnchanged — same planner, same misreport ❌
NotificationTableTests A payload without the new fields still renders, reading them as 0 ❌
automated-testing/import-and-staged-actions/21-reseed-preserves-existing-data.md A reseed against an up-to-date database accounts for every incoming item ❌

Scope

ImportActionKind gains Unchanged, so ADR 008 applies in full: Import_Action.ActionType carries
CHECK (ActionType IN ('Add', 'Modify')) in the baseline and three migration files, SQLite cannot widen
a CHECK in place, so this is a table-rebuild migration with the baseline updated in the same commit and
both drift tests extended. An unchanged action's status is Applied — terminal, nothing to decide.

EntityTypeActionCounts gains Incoming and Unchanged (one construction site). Because the kind is
persisted, ImportActionReportBuilder needs one new arm and no signature change. Incoming equals the
sum of every outcome bucket, and asserting that identity is what exposes the builder's two
_ => counts fall-throughs, which today discard an unmatched action rather than counting it.

The seed log line, docs/api-endpoints.md (twice) and the endpoint [Description] attributes each
enumerate the counts by hand and gain the new ones.

A reseed will write roughly one action per incoming entity rather than only per new one — hundreds
where there were none. That is the feature rather than its cost: it is what makes an import auditable
after the fact, and what lets an operator see which items a refreshed source actually changed. #249's
auto-purge clears a batch's actions once it applies fully.

#302's notification reports it in its body, not in a new column. ReseedEntityCountDto gains the
fields; the rendered table is unchanged, because the payload already carries structured detail and the
body is where the summary belongs. Its Added > 0 || Modified > 0 filter must admit an unchanged-only
breakdown, or a no-op reseed's confirmation vanishes exactly when this issue exists to report it.

Definition of done

  • Failing test(s) listed above are red before the fix is written
  • Fix implemented
  • All listed tests pass (green)
  • No regression in related tests
  • Findings summarised in a closing comment

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions