Lead Governance: Multi-Campaign Deduplication, Cross-Campaign Audience Ingestion & Provenance Merging

[sumamakhan761]

1. Problem Statement & Current Behavior

When marketing managers manage multiple campaigns (e.g., Campaign 1, Campaign 2, and Campaign 3) or upload CSV audiences across recurring broadcasts, duplicate contact occurrences frequently emerge:

1. Cross-Campaign Recipient Overlap in Multi-Select:
   • The same prospective buyer (e.g., identical email prospect@example.com or phone number) may exist as an audience recipient across Campaign 1, Campaign 2, and Campaign 3.
   • When selecting multiple campaigns on the Email Overview page and clicking "Export Leads (CSV)", the exported dataset contains duplicate rows for the same lead (one row per campaign dispatch) without an option to export unique prospects with consolidated engagement totals.
2. Provenance Collisions & Multi-Touch Blindspots:
   • When executing "Assign Leads to Pre-Sales Manager", the backend deduplicates new Lead creation by email and phone to avoid creating duplicate CRM rows. However, only the first resolved campaign title is stamped onto the lead's sourceId (e.g., Email: Campaign 1). Subsequent campaign touches (Campaign 2, Campaign 3) do not append multi-touch attribution or log activity timeline events.
   • Pre-sales executives and managers are left unaware that a prospect interacted with multiple distinct marketing broadcasts.
3. Pre-Assignment Duplicate Visibility:
   • The UI bulk action bar currently displays total audience rows rather than distinct unique contacts, giving managers an inflated impression of how many new prospects are entering the pipeline before confirmation.

2. Expected Behavior & User Experience

1. Deduplication Mode in Multi-Campaign Export:
   • The CSV Export workflow must offer two selectable modes:
     • Flattened Dispatch Records: 1 row per recipient dispatch across each campaign (useful for deliverability audits).
     • Deduplicated Unique Leads (Recommended): Deduplicates by normalized email / phone, aggregating performance across campaigns (e.g., Total Dispatches: 3, Total Opens: 2, Total Clicks: 1, Campaigns Involved: "Festive Launch, Tower B VIP, Weekend Open House").
2. Pre-Assignment Summary & Deduplication Preview Dialog:
   • Clicking "Assign Leads to Pre-Sales Manager" should open an informative preview modal showing a granular breakdown before triggering writes:
     • Total Records Processed: e.g., 2,400
     • Unique Potential Leads: e.g., 1,850
     • Cross-Campaign Duplicate Mentions: e.g., 550
     • Already Active in CRM: e.g., 320
     • Net New Leads to be Ingested: e.g., 1,530
3. Multi-Touch Lead Activity & Timeline Logging:
   • When a prospect already active in the CRM is referenced in an assigned campaign, or touches multiple selected campaigns:
     • The system preserves primary sourceId while creating an ActivityTimeline record:
       "[Email Marketing] Re-engaged via campaign: <Campaign Title> (Status: OPENED, Clicks: 2)"
     • Allows pre-sales executives in the lead profile drawer to view the complete omnichannel campaign history of the prospect.

---

3. Technical Scope & Affected Packages

• Audience Service: apps/api/src/marketing/email/services/email-audience.service.ts
  • Enhance getExportLeadsData to support a deduplicate: boolean query flag that groups and aggregates metrics by normalized email and phone.
  • Enhance bulkAssignRecipientsToCrm to insert ActivityTimeline entries for recurring cross-campaign touches instead of silently skipping them.
  • Add previewBulkAssignStats(dto: BulkAssignLeadsDto) endpoint returning unique count vs duplicate count breakdown before execution.
• Email DTOs: apps/api/src/marketing/email/dto/email.dto.ts
  • Add deduplicate?: boolean and strategy?: 'LATEST' | 'MERGE' to ExportLeadsDto and BulkAssignLeadsDto.
• Campaign List Component: apps/web/features/marketing/email/components/EmailCampaignListTable.tsx
  • Integrate confirmation modal rendering distinct lead count vs gross recipient count.
  • Add Export Options dropdown (Flattened vs Unique Prospects).
• Recipient Table Component: apps/web/features/marketing/email/components/EmailRecipientTable.tsx
  • Client-side deduplication badge or counter when viewing filtered lists.
• Shared Contracts: packages/types/src/email.ts
  • Define ExportLeadsRow and BulkAssignPreviewSummary interfaces.

---

4. UI/UX Specifications

• Bulk Assignment Confirmation Modal:
  • Clean dialog using the design system (Dialog.tsx) featuring an interactive visual breakdown card:
    • 🟢 New Leads: Green badge with count.
    • 🟡 Cross-Campaign Duplicates: Amber badge indicating merged provenance.
    • 🔵 Existing CRM Contacts: Blue badge indicating updated activity timelines.
• Export Options:
  • Split button or modal toggle:
    • “Export Unique Contacts (1 row per lead)”
    • “Export All Event Dispatches (1 row per send)”
• Notifications:
  • Toast notification includes a clear breakdown:
    "Assigned 1,530 new leads and updated timeline for 320 existing contacts across 3 campaigns."

---

5. Definition of Done (Acceptance Criteria)

• Multi-campaign CSV export supports both deduplicated unique leads and raw flattened dispatch rows.
• Assigning leads across multiple overlapping campaigns records ActivityTimeline entries for each campaign touchpoint.
• Pre-assignment preview modal displays exact counts for net new leads, cross-campaign duplicates, and existing CRM records before executing the assignment.
• No duplicate Lead records created in PostgreSQL when processing multiple campaigns with overlapping email/phone data.
• Full TypeScript compilation passes cleanly (pnpm --filter @brokeros/api exec tsc --noEmit and pnpm --filter @brokeros/web exec tsc --noEmit).