Skip to content

Bulk-decide a staged import batch via file export/import, CSV and JSON (Phase 1 of #153) #163

Description

@DutchJaFO

Background

Phase 1 of a two-phase plan toward #153 ("Declarative conflict-resolution file for recurring
third-party source conflicts"). Today, resolving a staged batch's ambiguous actions means calling
POST /api/v1/import/actions/{id}/decide once per action — fine for one or two conflicts, tedious
for a batch with many. This issue adds a file-based bulk alternative: export a batch's actions,
edit decisions in the file, re-import to apply them all at once. Phase 2 (#153) will generalize the
per-action decisions this issue produces into reusable, persistent per-source rules.

Depends on #162 (Source field decidability), #149, #154 (decide/undo/apply machinery, staging
engine).

Correction (2026-07-24, re-verified before implementation): this issue was filed when only Quote
and Source were decidable. Since then, #171 (StageDirection), #172 (SoundCue), #173 (Person), #175
(Character), and #176 (Conversation) have all shipped their own Modify/decidability paths, and #206
merged Quotinator.Engine into Quotinator.Core. The scope below has grown accordingly by explicit
developer decision — see the amended items and the new items at the end.

Both CSV and JSON are supported — not JSON-only. The row shape is one row per (action, field)
pair
, not one row per action:

ActionId,EntityId,EntityType,Field,ExistingValue,IncomingValue,Decision,CustomValue,MarkCompletenessAs
<guid>,<quoteId>,Quote,quoteText,"old text","new text",,,
<guid>,<quoteId>,Quote,character,"","Neo",Replace,,
<guid>,<sourceId>,Source,date,"1994","1994-06",Custom,"1994-06-15",
<guid>,<characterId>,Character,name,"Blocked Row","New Name",Replace,,Complete
  • Decision is Keep/Replace/Custom (blank = not yet decided); CustomValue only matters when
    Decision=Custom.
  • MarkCompletenessAs is one value per ActionId group (repeated on every row of that group, or
    left blank) — used to resolve a Blocked action (see below).
  • List-valued fields (e.g. Quote's genres) use a delimited value in the cell (drama;comedy) — a
    standard CSV convention, no nesting needed.
  • This shape is entity-agnostic and serializes identically as JSON (an array of these flat objects).

What needs to be done

  1. New GET /api/v1/import/actions/export?batchId=&format=csv|json endpoint (default json) returns
    every decidable field across the batch's actions in the flat shape above. Pending, Decided,
    and Blocked actions are all included — the idea is that bulk-decide is exactly how an operator
    resolves what to do with a Blocked batch, not something scoped out of it. No X-Api-Key
    required, matching GET /import/actions's existing public-read precedent.
  2. New POST /api/v1/import/actions/bulk-decide?batchId=&format=csv|json endpoint accepts the edited
    file back, groups rows by ActionId, and applies each action's decision via the existing
    IImportActionService.DecideAsync — reuses existing per-field validation, no new validation logic
    invented. Requires X-Api-Key, matching every other staged-action write endpoint.
  3. CSV parsing/writing uses a proper CSV library (quoting, embedded commas/quotes in ExistingValue/
    IncomingValue handled correctly) — not naive string splitting. The existing CsvLineParser
    (Quotinator.Converters.Csv, currently internal) is extracted into a shared, non-internal
    location and reused here, with a matching writer added — no duplicated CSV logic, no new NuGet
    dependency (DRY).
  4. Data-domain validation: Decision must be a recognised FieldResolutionChoice member
    (Keep/Replace/Custom); an unrecognised value is a clear per-row error, not silently defaulted.
  5. Consumer-domain validation: EntityType must be one of ImportActionEntityTypes.All; Field must
    be a valid, currently-decidable field name for that EntityType. After this issue, every entity
    type is decidable
    — Series and Universe (the last two without a Modify/decide path) get one as
    part of this issue too (see item 9 below). The reverse (EntityType, Field)ConflictDecisionRequest
    mapping this requires must cover all nine types explicitly, including Conversation — Conversation's
    description field has no dedicated To*DecisionMap helper today (it's built inline in
    DecideAsync, unlike every other entity's own named helper), so the mapping needs its own explicit
    case for it rather than being derived from a helper that doesn't exist.
  6. A row-level error (unknown ActionId, action not part of the requested batchId, invalid
    EntityType, invalid Decision value, unknown Field name, action already applied) reports which
    row failed without aborting the rest of the file — matches POST /import's existing "one bad row
    never aborts the rest" model.
  7. GET .../export's output (either format) round-trips through POST .../bulk-decide unmodified
    with zero errors — baseline correctness check before any real edits are made.
  8. README.md/addon/DOCS.md endpoint tables updated in the same commit.
  9. New: Series/Universe gain Modify/decidability. Both currently match/create purely by Name
    (natural key only, no explicit id, no Correction shape, no Modify path — unlike every other
    entity). They get the same id-present-Correction / id-absent-Creation two-shape treatment Source: explicit file-carried id, decoupling matching from Title/Type/Date content #162
    gave Source and Person: explicit id, Modify/decidability, wire up dateOfBirth/dateOfDeath #173 gave Person: series[]/universe[] schema entries gain an optional id;
    Sql.Series/Sql.Universe gain SelectExistingById/UpdateFieldsById/SelectCompletenessById/
    UpdateCompletenessById; PlanSeriesAsync/PlanUniverseAsync gain an id-match diff flow ahead of
    the existing natural-key path; DecideAsync/ComputeAmbiguousFields/ReverseAppliedActionsAsync/
    ApplyResolvedActionAsync/ClearStaleAddTargetsAsync all gain Series/Universe branches;
    ConflictDecisionRequest gains SeriesName/SeriesUniverseId/UniverseName.
  10. New: an already-Decided action's export shows its actual original per-field choice, not a
    guess.
    Previously, DecideAsync only ever persisted the resolved value (MergedFields) — the
    caller's original Keep/Replace/Custom choice was discarded once the merge ran. This meant
    export could only infer what was chosen (lossy, and ambiguous whenever existing and incoming
    already agreed). System_ImportActions (owned by Quotinator.Data) gains a new additive
    OriginalDecision column; DecideAsync now also writes the caller's actual per-field decision
    there, so a bulk-decide "revise a past decision" is a genuine round-trip, not an approximation.
    This was never a problem before this issue because every previous decide flow was one-shot and
    forward-only (the caller always supplied a fresh decision; the only way to "change your mind" was
    undo, which discards the old decision and starts over) — this is the first feature that needs the
    system to remember and redisplay a prior choice.

Expected tests

Test class Test method Starts
Quotinator.Api.Tests ExportImportActions_Json_ReturnsFlatFieldRows
Quotinator.Api.Tests ExportImportActions_Csv_ReturnsFlatFieldRows
Quotinator.Core.Tests DecideAsync_PersistsOriginalDecision_ExportReflectsExactPriorChoice
Quotinator.Api.Tests BulkDecide_JsonRoundTrip_NoErrors
Quotinator.Api.Tests BulkDecide_CsvRoundTrip_NoErrors
Quotinator.Api.Tests BulkDecide_ValidFile_DecidesEveryAction
Quotinator.Api.Tests BulkDecide_AllNineEntityTypes_DecidesEachCorrectly
Quotinator.Api.Tests BulkDecide_ConversationRow_DecidesDescriptionViaInlineMapping
Quotinator.Api.Tests BulkDecide_BlockedAction_ResolvesWithMarkCompletenessAs
Quotinator.Api.Tests BulkDecide_UnknownActionId_ReportsRowErrorWithoutAbortingOthers
Quotinator.Api.Tests BulkDecide_InvalidEntityTypeValue_Returns422WithClearMessage
Quotinator.Api.Tests BulkDecide_InvalidDecisionValue_Returns422WithClearMessage
Quotinator.Api.Tests BulkDecide_UnknownFieldName_Returns422WithClearMessage
Quotinator.Core.Tests PlanSeriesAsync_IdMatchFound_FieldsDiffer_StagesModifyAction
Quotinator.Core.Tests PlanUniverseAsync_IdMatchFound_NameDiffers_StagesModifyAction
Quotinator.Core.Tests DecideAsync_SeriesModify_ResolvesFieldDecisions
Quotinator.Core.Tests DecideAsync_UniverseModify_ResolvesFieldDecisions

Definition of done

  • All expected tests listed above start red before implementation
  • All requirements implemented
  • All expected tests pass (green)
  • No regression in related tests
  • Findings summarised in a closing comment

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions