Webhook endpoint answers after post-creation handlers: GitHub org-hook delivery recorded 500 (10 s timeout) although the event was stored (roll window) #6039

Moved from the fleet's private triage inbox (MeshWeaver.Feedback#47, filed 2026-09-26). Core owns it: memex/Memex.Portal.Shared/Api/WebhookInboxEndpoints.cs is core.

Owner: core Memex.Portal.Shared/Api/WebhookInboxEndpoints.cs (+ MeshExtensions CreateNode post-creation path). Measured 2026-09-26 on memex.systemorph.com. GitHub org hook 686037663 delivery 3844954004041777152 (guid 8914f260-b9d5-11f1-9f1c-5957880239ef, issues/edited, Systemorph/Memex#116) is recorded status_code 500, duration 10, response headers null, status text verbatim: 'POST https://memex.systemorph.com/api/hooks/Hosting/PlatformBuilds giving up after 1 attempt(s): ... context deadline exceeded (Client.Timeout exceeded while awaiting headers)'. So the 500 is GitHub's own label for NO RESPONSE in 10 s, not a server 500. The portal side: pod memex-portal-deployment-85f748bdd7-qvqf7 (new ReplicaSet of the Roll to 3.0.0-ci.9414, Ops/Actions/selfupdate-roll-memex-3-0-0-ci-9414-63633d50, whose watcher reports an observer pod gone at 18:10:20Z) logged 'Node created at Hosting/PlatformBuilds/_Inbox/54cf68225ef94775a0482c11d17d9f26 by system-security' at 18:10:31.369Z, i.e. the store succeeded ~200 ms after GitHub sent; the event was later processed and is gone from the inbox, so nothing was lost (and an 'edited' action is ignored by triage anyway; the endpoint never parses the body, so the action plays no role). What stalled is the path from 'Node created' to the HTTP answer: MeshExtensions CreateNode runs the post-creation handlers and posts the response after that log line, and the endpoint awaits CreateNode(...).Take(1) before answering. The last 100 deliveries: p50 0.46 s, p90 2.38 s, max 10 s; the 13 after 18:13Z (still in the roll) took 1.0-4.9 s. Not established: which stage (post-creation handlers, response routing to the root hub while the old ReplicaSet's silo was leaving, or Kestrel) held the answer; 'Webhook stored at' (Information) and the [CreateNode] step lines (Debug) are not shipped at this instance's level (a control query for 'Webhook stored at|ignoring non-build event' over 35 min returned 0 lines for ALL deliveries), so the logs cannot say. Risk: GitHub does not redeliver; any delivery that stalls past 10 s is shown red and a sender that treats non-2xx as failure (node-repo-ci-failure, publish-bake) may retry or fail although the event was stored. Candidate direction (not a bound increase): answer once the store has committed rather than after post-creation handlers and the response hop.

Context (added by triage):

• Category: bug (verified defect report, agent session, Claude Code)
• Where: no specific Space/portal node named — observed during the memex.systemorph.com self-update roll to 3.0.0-ci.9414; affects the portal webhook inbox path generally.
• Evidence trails: gh api orgs/Systemorph/hooks/686037663/deliveries/3844954004041777152; Ops/Logs/memex-1790446231369780045-48bdd7-qvqf7 (Node created); Ops/Actions/logs-memex-20260926-inbox-54cf6822 (1 line only); Ops/Actions/logs-memex-20260926-webhook-stored-vs-ignored (0 lines, control arm); Ops/Actions/logs-memex-20260926-fail-lines-1810 (health-check failures every 10 s on old pod 5f99d7f5fb-hk7t7 18:09:58-18:11:38, fail: Polly on qvqf7 18:10:50).
• Origin: https://memex.systemorph.com/rbuergi/Feedback/org-webhook-timeout-during-roll-20260926

filed by the triage agent on behalf of rbuergi (agent session, Claude Code) via systemorph-com