feat(workflowbuilder): broken workflow refs and cross-module delete guards

.github/workflows/build-and-test.yml

@@ -225,21 +225,7 @@ jobs:
       - name: buf breaking (vs base branch)
         env:
           BASE_REF: origin/${{ github.event.pull_request.base.ref || 'main' }}
-        run: |
-          set -euo pipefail
-          if ! git rev-parse --verify "${BASE_REF}" >/dev/null 2>&1; then
-            echo "::error::Missing ${BASE_REF} — ensure checkout fetch-depth is 0"
-            exit 1
-          fi
-          if git rev-parse "${BASE_REF}:buf.yaml" >/dev/null 2>&1; then
-            buf breaking --against ".git#ref=${BASE_REF}"
-          else
-            echo "Base has no buf.yaml yet — per-module breaking against ${BASE_REF}"
-            awk '/^[[:space:]]*- path: / { print $3 }' buf.yaml | while read -r dir; do
-              echo "buf breaking ${dir}"
-              buf breaking "${dir}" --against ".git#ref=${BASE_REF},subdir=${dir}"
-            done
-          fi
+        run: ./scripts/buf-breaking-against-base.sh
 
   link-check:
     name: Markdown link check

buf.yaml

@@ -8,6 +8,7 @@ version: v2
 modules:
   - path: src/Modules/Identity/Axis.Identity.Contracts/Protos
   - path: src/Modules/FormBuilder/Axis.FormBuilder.Contracts/Protos
+  - path: src/Modules/WorkflowBuilder/Axis.WorkflowBuilder.Contracts/Protos
 lint:
   use:
     - STANDARD

docs/PROGRESS.md

@@ -13,7 +13,7 @@ Foundation phases (each a sequence of small PRs):
 |---|---|---|
 | **Phase 0 — Foundation decisions** | ✅ done (PR #59) | Rewrote ADR-001/002/009; added ADR-010..023; updated `ARCHITECTURE.md` + `CLAUDE.md` + `patterns.md`. |
 | **Phase 1 — Infrastructure foundation** | ✅ done | PR #83–#90: Kafka/RabbitMQ (ADR-017), per-module DBs (ADR-011), Wolverine enroll (ADR-012), migrations (ADR-023), OpenTelemetry (ADR-018), **Avro + Schema Registry + CloudEvents** for WorkflowBuilder lifecycle events (ADR-019). |
-| **Phase 2 — Per-module HTTP/gRPC boundary** | ⚠️ in progress | Identity complete: `Axis.Identity.Contracts` + `IdentityService` gRPC (`GetUserPermissions`) + Avro lifecycle events (`OrganizationVerifiedEvent`, `UserDeactivatedEvent`, `UserReactivatedEvent`, `RoleAssignedEvent`, `RoleRemovedEvent`) + per-module `OrganizationVerifiedHandler` (provisions each module's tenant schema from Kafka) + gateway `IdentityServiceClient` gRPC wiring. Remaining: DataModeling/FormBuilder/WorkflowBuilder/WorkflowEngine `Axis.{Module}.Contracts` projects with their own events. |
+| **Phase 2 — Per-module HTTP/gRPC boundary** | ⚠️ in progress | All MVP modules have `Axis.{Module}.Contracts` + Kafka Avro events. Identity + FormBuilder gRPC services wired. WorkflowBuilder `WorkflowFormReferenceService` gRPC + `FormDeletedEvent` Kafka. **Deferred:** DataModeling gRPC. |
 | **Phase 3 — Per-module EF migrations** | ⚠️ in progress | Identity, DataModeling, FormBuilder, WorkflowBuilder, WorkflowEngine have migrations; tests use `MigrateAsync`. PageBuilder pending (module not started). |
 | **Phase 4 — Deployment readiness** | ⏳ pending | Per-module Dockerfile; `docker-compose.dev.yml` runs each module as a separate container; CI builds per-module artifacts; K8s manifests; per-module Vault policies. |
 
@@ -39,27 +39,27 @@ Full auth, user, role, invitation, and session management. OpenIddict 5.x OIDC s
 
 ## DataModeling — E03-data-modeling
 
-**Domain ✅ | Application ✅ | Infrastructure ✅ | API ✅ | Frontend ⏳ · Service-boundary retrofit ⏳**
+**Domain ✅ | Application ✅ | Infrastructure ✅ | API ✅ | Frontend ⏳ · Service-boundary retrofit ✅**
 
 Custom model, field, data class, and record CRUD. Full-text search, per-field JSONB filters, sort-by-column, bulk delete, CSV export. All endpoints covered by integration tests.
 
-> ⏳ **Retrofit:** add `Axis.DataModeling.Contracts` (gRPC + Avro events for `ModelCreated`/`FieldAdded`/...); move to `axis_datamodeling` database; generate initial EF migration; switch tests to migrations.
+> ✅ **Contracts + Kafka:** `Axis.DataModeling.Contracts` publishes lifecycle events; FormBuilder + WorkflowBuilder consume `ModelDeletedEvent` for broken refs. **Deferred:** relation fields on other models flagged broken when target model deleted.
 
 ## WorkflowBuilder — E04-workflow-builder
 
-**Domain ✅ | Application ✅ | Infrastructure ✅ | API ✅ | Frontend ⏳ · Service-boundary retrofit ⏳**
+**Domain ✅ | Application ✅ | Infrastructure ✅ | API ✅ | Frontend ⏳ · Service-boundary retrofit ⚠️**
 
 Workflow definitions with steps, transitions, triggers, cycle detection, publish/archive lifecycle. Import/export (JSON + ZIP). All endpoints covered by integration tests.
 
-> ⏳ **Retrofit:** add `Axis.WorkflowBuilder.Contracts`; existing cross-module events (`FormStepAdded`, `WorkflowPublished`, ...) become Kafka-published with Avro schemas; move to `axis_workflowbuilder` database; existing EF migrations stay but tests switch from `EnsureCreated` to `MigrateAsync`.
+> ✅ **Broken refs:** `workflow_form_references` + `workflow_model_references` read models; sync on step/trigger changes; `ModelDeletedHandler` + `FormDeletedHandler` (Kafka); publish blocked when broken; `GetWorkflow` exposes `isBroken`; `WorkflowFormReferenceService` gRPC for form delete guard (draft + active workflows).
 
 ## FormBuilder — E05-form-builder
 
 **Domain ✅ | Application ✅ | Infrastructure ✅ | API ✅ | Frontend ⏳ · Service-boundary retrofit ⚠️**
 
 Form definitions + F04 form tasks (`FormSubmission`, token submit, my tasks, expiry job). Submission user resolved via `ICurrentUser` in Application.
 
-> ✅ **Phase 2 Contracts:** Avro form-task events; `form_model_references` + `ModelDeletedHandler` (DataModeling Kafka) for Relation Picker broken refs; delete-model blocked when active form references exist. **Deferred:** gRPC service.
+> ✅ **Phase 2 Contracts:** Avro form-task + `FormDeletedEvent`; `form_model_references` + `ModelDeletedHandler`; delete-model guard via FormBuilder gRPC; delete-form guard via WorkflowBuilder gRPC (`WorkflowFormReferenceService`).
 
 ## WorkflowEngine — E06-workflow-engine
 

docs/epics/E03-data-modeling/README.md

@@ -83,7 +83,7 @@ A concrete instance of a Model. Records are stored in the tenant's schema using
 |---|---|---|
 | Domain | ✅ Done | `DataModel`, `Field`, `DataRecord` aggregates; all field types and domain events |
 | Application | ✅ Done | All command/query handlers; `RecordFieldValidator`; `BulkDeleteRecordsHandler`; `ExportRecordsCsvHandler` |
-| Infrastructure | ✅ Done | EF Core mappings, repositories, JSONB field converters; `GetPagedAsync` with filter/sort; `BulkDeleteAsync`; `GetAllForExportAsync`. Database `axis_datamodeling` with initial migration `InitialCreate` ([ADR-011](../../TECH_STACK.md#adr-011-per-module-database-with-schema-per-tenant-inside), [ADR-023](../../TECH_STACK.md#adr-023-per-module-ef-core-migrations-only)). DbContext + UnitOfWork inlined per ADR-017. `OrganizationVerifiedHandler` subscribes to Identity's Kafka event and provisions this module's tenant schema on its own (PR #93 — no central provisioner). `Axis.DataModeling.Contracts` + `DataModelingEventMapper` publish 9 Avro lifecycle/field events via Wolverine outbox → Kafka ([ADR-019](../../TECH_STACK.md#adr-019-avro-and-schema-registry-for-event-payloads-with-cloudevents-envelope), [ADR-025](../../TECH_STACK.md#adr-025-transport-selection-rule-by-message-name-suffix)) (PR #101). **Deferred:** Kafka consumers in FormBuilder/WorkflowBuilder for model/field delete refs. |
+| Infrastructure | ✅ Done | EF Core mappings, repositories, JSONB field converters; `GetPagedAsync` with filter/sort; `BulkDeleteAsync`; `GetAllForExportAsync`. Database `axis_datamodeling` with initial migration `InitialCreate` ([ADR-011](../../TECH_STACK.md#adr-011-per-module-database-with-schema-per-tenant-inside), [ADR-023](../../TECH_STACK.md#adr-023-per-module-ef-core-migrations-only)). DbContext + UnitOfWork inlined per ADR-017. `OrganizationVerifiedHandler` subscribes to Identity's Kafka event and provisions this module's tenant schema on its own (PR #93 — no central provisioner). `Axis.DataModeling.Contracts` + `DataModelingEventMapper` publish 9 Avro lifecycle/field events via Wolverine outbox → Kafka ([ADR-019](../../TECH_STACK.md#adr-019-avro-and-schema-registry-for-event-payloads-with-cloudevents-envelope), [ADR-025](../../TECH_STACK.md#adr-025-transport-selection-rule-by-message-name-suffix)) (PR #101). **Deferred:** Kafka consumer in WorkflowBuilder for field delete refs; relation fields on other models flagged broken when target model deleted. **Done:** `ModelDeletedHandler` in FormBuilder + WorkflowBuilder. |
 | API | ✅ Done | 7 record endpoints (CRUD + bulk-delete + CSV export); filter/sort params; HTTP 422 `ValidationProblemDetails` on create/update |
 | Frontend | ⏳ Pending | — |
 

docs/epics/E03-data-modeling/features/F01-model-definition.md

@@ -124,4 +124,4 @@ Users can create custom data models within their organization. A model defines t
 
 > **Implementation status** — Domain + Application: ✅ | Infrastructure: ✅ | API: ✅ | Frontend: ⏳
 > Gaps vs spec: workflow reference check pending E04; form Relation Picker refs blocked/flagged via FormBuilder `ModelDeletedEvent` consumer (US-033 partial); 30-day purge background job pending.
-> **Deferred:** WorkflowBuilder `record.*` trigger broken flags; DataModeling relation fields on other models flagged broken when target model deleted.
+> **Deferred:** DataModeling relation fields on other models flagged broken when target model deleted. WorkflowBuilder `record.*` trigger broken flags shipped via `ModelDeletedHandler` (Kafka).

docs/epics/E04-workflow-builder/README.md

@@ -76,7 +76,7 @@ The workflow builder is the heart of the platform. It is what differentiates Axi
 |---|---|---|
 | Domain | ✅ Done | `WorkflowDefinition`, `Step`, `Trigger` aggregates; all step types and domain events; ConfigureStep method; AddTrigger duplicate-type guard |
 | Application | ✅ Done | All 15 handlers: CreateWorkflow, PublishWorkflow, ArchiveWorkflow, UnarchiveWorkflow, UpdateWorkflow, DuplicateWorkflow, AddStep, RemoveStep, ConfigureStep, AddTransition, RemoveTransition, AddTrigger, RemoveTrigger, ImportWorkflow, BulkExportWorkflows; GetWorkflows, GetWorkflow, ExportWorkflow queries |
-| Infrastructure | ✅ Done | WorkflowBuilderDbContext, EF Core configuration (WorkflowDefinition with steps/transitions/triggers as JSONB), WorkflowRepository, 7 integration tests (Testcontainers). Schema managed via EF Core migrations (e.g. `AddUniqueConstraintOnWorkflowName`). DbContext + UnitOfWork inlined per ADR-017. `OrganizationVerifiedHandler` subscribes to Identity's Kafka event and provisions this module's tenant schema on its own (PR #93 — no central provisioner). Cross-module lifecycle events publish via `Axis.WorkflowBuilder.Contracts` Avro + Kafka ([ADR-019](../../TECH_STACK.md#adr-019-avro-and-schema-registry-for-event-payloads-with-cloudevents-envelope)). |
+| Infrastructure | ✅ Done | WorkflowBuilderDbContext, EF Core configuration, WorkflowRepository. `workflow_form_references` + `workflow_model_references` read models with `IWorkflowReferenceSync`; `ModelDeletedHandler` + `FormDeletedHandler` (Kafka); `WorkflowFormReferenceService` gRPC. Migration `AddWorkflowReferenceReadModels`. `OrganizationVerifiedHandler` + Avro lifecycle publish ([ADR-019](../../TECH_STACK.md#adr-019-avro-and-schema-registry-for-event-payloads-with-cloudevents-envelope)). Publish blocked when broken refs; `GetWorkflow` returns `isBroken` on steps/triggers. |
 | API | ✅ Done | 18 endpoints: workflow CRUD + publish/archive/unarchive/duplicate, step/transition/trigger management, JSON export, JSON import, ZIP bulk export |
 | Frontend | ⏳ Pending | — |
 

docs/epics/E05-form-builder/README.md

@@ -80,7 +80,7 @@ Workflow reaches Form step
 |---|---|---|
 | Domain | ✅ Done | `FormDefinition`, `FormField`, `FormSubmission` aggregates; field types and form-task domain events |
 | Application | ⚠️ Partial | Form definition CRUD + fields; F04: `SubmitFormByToken`, `GetFormTaskByToken`, `GetMyFormTasks`, `ExpireFormSubmissionHandler`. Notifications and role-based assignee resolution pending |
-| Infrastructure | ✅ Done | `form_model_references` read model + `ModelDeletedHandler` (DataModeling Kafka) flags broken Relation Picker fields; delete-model guard via `IFormModelReferenceRepository`. Database `axis_formbuilder` ([ADR-011](../../TECH_STACK.md#adr-011-per-module-database-with-schema-per-tenant-inside)); EF migrations including `AddFormSubmissions`; tests/fixtures use `MigrateAsync` ([ADR-023](../../TECH_STACK.md#adr-023-per-module-ef-core-migrations-only)). `FormSubmission` + expiry scheduling via Wolverine. DbContext + UnitOfWork inlined per ADR-017. `FormBuilderEventMapper` translates domain events to Avro at `SaveChangesAsync` and publishes via outbox → Kafka ([ADR-019](../../TECH_STACK.md#adr-019-avro-and-schema-registry-for-event-payloads-with-cloudevents-envelope)). `OrganizationVerifiedHandler` subscribes to Identity's Kafka event and provisions this module's tenant schema on its own (PR #93 — no central provisioner). `FormStepReachedHandler` consumes WorkflowEngine's `FormStepReachedEvent` from Kafka (Contracts only — no Domain reference). Consumes WorkflowBuilder lifecycle events from `Axis.WorkflowBuilder.Contracts`. |
+| Infrastructure | ✅ Done | `form_model_references` read model + `ModelDeletedHandler` (DataModeling Kafka) flags broken Relation Picker fields; delete-model guard via `IFormModelReferenceRepository`. Delete-form guard: `FormWorkflowDeletionGuard` calls WorkflowBuilder gRPC `CountBlockingFormReferences`; `FormDeletedEvent` Avro published on delete for WorkflowBuilder `FormDeletedHandler`. Database `axis_formbuilder` ([ADR-011](../../TECH_STACK.md#adr-011-per-module-database-with-schema-per-tenant-inside)); EF migrations including `AddFormSubmissions`; tests/fixtures use `MigrateAsync` ([ADR-023](../../TECH_STACK.md#adr-023-per-module-ef-core-migrations-only)). `FormSubmission` + expiry scheduling via Wolverine. DbContext + UnitOfWork inlined per ADR-017. `FormBuilderEventMapper` translates domain events to Avro at `SaveChangesAsync` and publishes via outbox → Kafka ([ADR-019](../../TECH_STACK.md#adr-019-avro-and-schema-registry-for-event-payloads-with-cloudevents-envelope)). `OrganizationVerifiedHandler` subscribes to Identity's Kafka event and provisions this module's tenant schema on its own (PR #93 — no central provisioner). `FormStepReachedHandler` consumes WorkflowEngine's `FormStepReachedEvent` from Kafka (Contracts only — no Domain reference). Consumes WorkflowBuilder lifecycle events from `Axis.WorkflowBuilder.Contracts`. |
 | Contracts | ✅ Done | `Axis.FormBuilder.Contracts` — Avro schemas `FormTaskSubmittedEvent` + `FormTaskExpiredEvent` (the form-task lifecycle events WorkflowEngine reacts to). Hand-written `ISpecificRecord` generated code + `FormBuilderKafkaTopics` + `FormBuilderEventExtensions` (typed GUID accessors + `SubmittedData()` JSON round-trip helper since Avro lacks a native any-type). |
 | API | ✅ Done | `FormEndpoints` (definitions) + `FormTaskEndpoints` (token submit, my tasks). `submittedBy` resolved via `ICurrentUser` in Application |
 | Frontend | ⏳ Pending | — |

docs/epics/README.md

@@ -42,6 +42,6 @@ Platform-wide infrastructure tracked in [PROGRESS.md](../PROGRESS.md) (not owned
 |---|---|---|
 | Phase 0 — ADRs & architecture | ✅ | ADR-010..023 |
 | Phase 1 — Infrastructure | ✅ | Per-module DBs, Kafka/RabbitMQ, OpenTelemetry ([ADR-018](../TECH_STACK.md#adr-018-opentelemetry-sdk-with-grafana-stack-for-observability)), Avro/Schema Registry pilot on WorkflowBuilder events ([ADR-019](../TECH_STACK.md#adr-019-avro-and-schema-registry-for-event-payloads-with-cloudevents-envelope)) |
-| Phase 2 — Module boundaries | ⚠️ | Identity, WorkflowBuilder, FormBuilder, WorkflowEngine, and DataModeling `Axis.{Module}.Contracts` + Kafka Avro events. **Deferred:** cross-module consumers (e.g. model delete → form/workflow refs); DataModeling gRPC. |
+| Phase 2 — Module boundaries | ⚠️ | Identity, WorkflowBuilder, FormBuilder, WorkflowEngine, and DataModeling `Axis.{Module}.Contracts` + Kafka Avro events. **Done:** model/form delete → broken refs (FormBuilder + WorkflowBuilder Kafka handlers, WorkflowBuilder gRPC for form delete guard). **Deferred:** DataModeling gRPC; field-delete consumers. |
 
 Implementation patterns: [OpenTelemetry observability](../playbooks/patterns.md#opentelemetry-observability).

docs/playbooks/agent-checklist.md

@@ -35,7 +35,7 @@ Docs touched: docs/epics/…
 | **0** | AC map + docs touched (when `src/`, `tests/`, or `frontend/` change) |
 | **1** | Full .NET + frontend verification (table below) |
 | **2** | Doc walk-through (rows below) |
-| **3** | Retrospective (seven questions) |
+| **3** | Retrospective (questions below) |
 
 **CI-only gates** (run automatically on PR, no local action required):
 
@@ -97,10 +97,25 @@ Gate 2:
 
 **Deferred follow-ups (mandatory when leaving work open):** do not wait for the user. Any skipped review item, thin-endpoint refactor, or partial layer needs a named `**Deferred (...):**` line — full rules in [process.md § Deferred follow-up](process.md). Remove the line when fixed.
 
+### Review feedback (CodeRabbit / human)
+
+Apply **before** resolving review threads (no user reminder required). Bots are **signal**, not authority — validate against [patterns.md](./patterns.md) and [CLAUDE.md](../../CLAUDE.md).
+
+Do **not** ship the first diff that only makes CI green or closes the thread. For each comment, ask:
+
+1. **Is this already best practice** for this codebase (patterns, layer boundaries, siblings in the same module)?
+2. **Can I improve or enhance** beyond what the reviewer suggested (clearer ownership, fewer round-trips, one transaction boundary, consistent error handling)?
+3. If a better design is feasible but skipped, is that a deliberate **minimal diff** (user asked) or should it be **`**Deferred (...):**`**?
+
+**Default:** prefer the design you would defend in review. **Exception:** user explicitly requests the smallest change — say so in the PR Summary.
+
+**PR Summary (one line when review fixes are non-trivial):** `Review fixes: improved — <what>` or `Review fixes: minimal — <why>`.
+
+**Examples** (illustrative — not an exhaustive list): splitting an invariant across “mutate then query”; external calls with catch-only patches; duplicating logic to silence a linter. In those cases, look for a single owner of the invariant or parity with existing guards/handlers in the repo.
 
 ### Gate 3 — retrospective
 
-Answer **Yes** or **No** on **each line** (same `-` bullet style as Gate 2 — do not collapse to `1–7 No`). If **Yes**, name the doc updated in this PR.
+Answer **Yes** or **No** on **each line** (same `-` bullet style as Gate 2 — do not collapse to a single `No`). If **Yes**, name the doc updated in this PR.
 
 ```text
 Gate 3:
@@ -109,6 +124,7 @@ Gate 3:
 - Infrastructure footgun? → No
 - Non-obvious test setup? → No
 - Changed direction mid-task? → No
+- Review-driven change left as a shortcut when a better design was feasible? → No
 - Spec gap discovered? → No
 - Incident-level detail in rule text? → No
 ```

scripts/buf-breaking-against-base.sh

@@ -0,0 +1,36 @@
+#!/usr/bin/env bash
+# Per-module buf breaking vs BASE_REF. Skips proto roots that did not exist on the base
+# branch (avoids workspace image-count mismatch when adding a new modules: entry).
+set -euo pipefail
+
+ROOT="$(cd "$(dirname "$0")/.." && pwd)"
+cd "$ROOT"
+
+BASE_REF="${BASE_REF:?Set BASE_REF (e.g. origin/main)}"
+
+if ! git rev-parse --verify "${BASE_REF}" >/dev/null 2>&1; then
+  echo "buf-breaking-against-base FAIL: missing ${BASE_REF}" >&2
+  exit 1
+fi
+
+if ! command -v buf >/dev/null 2>&1; then
+  echo "buf-breaking-against-base FAIL: buf CLI not on PATH" >&2
+  exit 1
+fi
+
+if [ ! -f buf.yaml ]; then
+  echo "buf-breaking-against-base: no buf.yaml — skip"
+  exit 0
+fi
+
+while read -r dir; do
+  [ -z "${dir}" ] && continue
+  if git ls-tree -r --name-only "${BASE_REF}" "${dir}" 2>/dev/null | grep -qE '\.proto$'; then
+    echo "buf breaking ${dir} (vs ${BASE_REF})"
+    buf breaking "${dir}" --against ".git#ref=${BASE_REF},subdir=${dir}"
+  else
+    echo "buf breaking: skip ${dir} (new on this branch — no baseline on ${BASE_REF})"
+  fi
+done < <(awk '/^[[:space:]]*- path: / { print $3 }' buf.yaml)
+
+echo "buf-breaking-against-base: OK"