Skip to content

Top-level Space create is not atomic — churn leaves content-less, grant-less ghost roots #638

Closed

Description

@rbuergi

Creates during pod churn landed the node row without content or the creator's Admin grant. The partitions were then un-writable, un-deletable (30s delete timeouts) and writes into them HANG on hub activation instead of failing fast. Create must be all-or-nothing; writes into a broken partition must fail fast with a speaking error.

From the 2026-07-24 plugin-install retrospective.

🤖 Generated with Claude Code

Activity

  1. rbuergi commented on Aug 8, 2026

    @rbuergi
    ContributorAuthor

    Status audit against current main (2026-08-08) — PARTIAL; re-scoping to what remains.

    Fixed — the hang half: writes into a confirmed-absent partition fail fast with a speaking error (PartitionWriteGuardValidator.cs:250-260 — 'no partition, no write'); PathResolutionService.cs:357-380 refuses to synthesize a placeholder root for an absent partition (404 instead of activating a grain onto a missing schema, pinned by PgOnlyProdShapeTests); grain activation is bounded and always terminal (MessageHubGrain FirstNodeResolutionTimeout 30s, 'no path leaves _hubReadyRaw pending forever', real cause surfaced via GrainActivationFailureRegistry, pinned by PartitionRootActivationTest). #909 additionally made ghost-root repair writes actually land (version floor vs MonotonicWriteGuard, MeshExtensions.cs:2781-2870, pinned by GhostPartitionRootRecoveryTest).

    Remaining (this issue now tracks exactly these):

    1. Create is still not atomic: order is provision → write row → post-creation grant, and a failed grant returns Fail while the row STAYS ('Node persisted but post-creation handler failed', MeshExtensions.cs:693-718) — no compensating delete. The Top-level Space create is not atomic — churn leaves content-less, grant-less ghost roots #638 residue (row present, grant absent) is still producible; only its reporting improved.
    2. Self-heal gaps for exactly that residue: EnsurePartitionBootstrap treats a content-less ghost row as 'root exists' (rootExists = root is not null, :1192) so it never repairs it, and the heal is gated on CheckPermission(Create) (:1208) — a grant-less partition denies precisely the user whose grant went missing, so it self-heals only under System/platform-admin.
    3. A provisioned-but-grant-less partition still surfaces as generic RLS 'access denied' (fast, but not the speaking partition-broken error asked for).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions