Skip to content

NumberFieldView subscribe to user hub times out — downstream of hub cache unavailability #1195

Closed

Description

@meshweaver-cloud

What is failing

MeshWeaver.Blazor.Components.NumberFieldView fails to load an 'Order' field in area Settings/2/2/2/1/5 for user sglauser. The component issues a SubscribeRequest via hub cache gmIIqhF220CKKxqxgY_J7Q targeting the user's hub, but receives no response within the 60-second timeout and throws System.TimeoutException.

Probable cause (high confidence)

This is not a defect in NumberFieldView itself. The same hub cache (gmIIqhF220CKKxqxgY_J7Q) and the same user (sglauser) appear in the ThreadChatView timeout incident (issue #1175) at nearly the same timestamps (22:42 and 23:57 UTC on 2026-08-10, same pod kmwvr). The hub cache cannot deliver SubscribeRequest messages to the target hub for that user. The root cause is the broader hub-cache/subscribe-routing failure already tracked in issue #1186 (MessageHubGrain activation times out — partition has no claiming query provider).

Evidence chain:

Impact

Low. 2 occurrences over ~75 minutes on a single pod. Only the sglauser user is affected — the NumberFieldView component for their Order settings area fails to render during that window. The user may have seen a blank or broken field.

Where to look

  • The root cause is the hub-cache/query-provider partition routing — see issue MessageHubGrain activation frequently times out during initialization due to unresolvable mesh nodes #1186 for investigation starting point (MessageHubGrain.OnActivateAsync(), query provider registration)
  • For this specific symptom path: MeshWeaver.Blazor.Components.NumberFieldView — the method that subscribes to the user's hub for data. The component itself is likely fine; it's just the caller of the faulty subscribe.
  • Consider whether NumberFieldView should handle subscribe timeouts more gracefully (show an error state instead of silently failing), but that's a polish concern, not this defect.

Note: This is a downstream symptom of the same root cause as issues #1175 and #1186. Close as duplicate if #1186 addresses the hub-cache routing.


Evidence

Fingerprint ec2af2454acf0a37
Category MeshWeaver.Blazor.Components.NumberFieldView
Severity Error
Exception System.TimeoutException
Namespace memex-cloud
Pods memex-portal-deployment-59479dc7d9-kmwvr
Occurrences 2
First seen 2026-08-10 22:42:50Z
Last seen 2026-08-10 23:57:02Z
Recent log lines
2026-08-10 22:42:50Z memex-portal-deployment-59479dc7d9-kmwvr fail: MeshWeaver.Blazor.Components.NumberFieldView[0]
      Loading 'Order' in area 'Settings/2/2/2/1/5' in Area Settings/2/2/2/1/5
      System.TimeoutException: No response received in hub cache/gmIIqhF220CKKxqxgY_J7Q within 00:01:00 for request SubscribeRequest (id=ooQ-jOpunky2lqj9_WhNMw) → target sglauser. The request may have been undeliverable or the target hub was not found.
2026-08-10 23:57:02Z memex-portal-deployment-59479dc7d9-kmwvr fail: MeshWeaver.Blazor.Components.NumberFieldView[0]
      Loading 'Order' in area 'Settings/2/2/2/1/5' in Area Settings/2/2/2/1/5
      System.TimeoutException: No response received in hub cache/gmIIqhF220CKKxqxgY_J7Q within 00:01:00 for request SubscribeRequest (id=kxr9_P2kXE-AWL7Z4RSTnQ) → target sglauser. The request may have been undeliverable or the target hub was not found.

Opened automatically from Admin/_LogIncident/ec2af2454acf0a37. Recurrences are folded into this issue rather than opening new ones.

Activity

  1. rosalie1984 commented on Aug 11, 2026

    @rosalie1984
    Collaborator

    The "user session had ended / severity Low / one user affected" scoping in this triage looks wrong — the same hub cache gmIIqhF220CKKxqxgY_J7Q is poisoning ordinary content paths for every reader, not just sglauser's Settings fields.

    Live on the same portal at 2026-08-11 07:44–07:56Z (image ci.2926): DataModeling/Formatting, DataModeling/Validation, DataModeling/17-AttributeToolbox and rsalzmann/GemschiGame all fail their point read through that cache, while children listings, search, Admin/PlatformVersion and Doc/Architecture are fine and /healthz is 200. The nodes demonstrably exist.

    And it does not heal: three reads of the same path ~11 min apart replayed the byte-identical request id 3kjIJ9lXPUCqPU5Zkgmhjw, so no fresh SubscribeRequest is ever issued after the first timeout. NumberFieldView/TextFieldView are the messenger, not the defect.

    Root-cause analysis (terminally-faulted ReplaySubject entry + the Reprobing latch that can never be cleared once set spuriously) is in #1202 — suggest tracking there and closing this as a downstream symptom.

  2. systemorph-com commented on Sep 17, 2026

    @systemorph-com
    Contributor

    Reopened — this fault is happening again.

    Still occurring: 5 more since the last update (7 total).

    • Last seen: 2026-08-11 12:11:31Z
    • Pods: memex-portal-deployment-59479dc7d9-kmwvr, memex-portal-deployment-686bdb74cb-vj9xl
    Recent log lines
    2026-08-11 06:52:07Z memex-portal-deployment-59479dc7d9-kmwvr fail: MeshWeaver.Blazor.Components.NumberFieldView[0]
          Loading 'Order' in area 'Settings/2/2/2/1/5' in Area Settings/2/2/2/1/5
          System.TimeoutException: No response received in hub cache/gmIIqhF220CKKxqxgY_J7Q within 00:01:00 for request SubscribeRequest (id=gD0UUG39_0KBM7idB8C92A) → target sglauser. The request may have been undeliverable or the target hub was not found.
    2026-08-11 07:06:33Z memex-portal-deployment-59479dc7d9-kmwvr fail: MeshWeaver.Blazor.Components.NumberFieldView[0]
          Loading 'Order' in area 'Settings/2/2/2/1/5' in Area Settings/2/2/2/1/5
          System.TimeoutException: No response received in hub cache/gmIIqhF220CKKxqxgY_J7Q within 00:01:00 for request SubscribeRequest (id=Fu_IfbJ8G0-sRBjeiIhCtw) → target sglauser. The request may have been undeliverable or the target hub was not found.
    2026-08-11 12:11:31Z memex-portal-deployment-686bdb74cb-vj9xl fail: MeshWeaver.Blazor.Components.NumberFieldView[0]
          Loading 'Order' in area 'Settings/2/2/2/1/5' in Area Settings/2/2/2/1/5
          System.TimeoutException: No response received in hub cache/jgIR0pcfFESb9hVz0NJhyQ within 00:01:00 for request SubscribeRequest (id=jmqN8-2ceUmBx0Sn4encHA) → target sglauser. The request may have been undeliverable or the target hub was not found.
    
  3. rbuergi commented on Sep 17, 2026

    @rbuergi
    Contributor

    Re-closed — the reopen above was on stale evidence (#4577).

    The recurrence bot reopened this at 2026-09-17T07:52:46Z under the headline "this fault is happening again", but the occurrence its own comment cites is OLDER than the close it reverted:

    the bot's Last seen: the close it reverted
    2026-08-11 12:11:31Z 2026-08-12T06:58:07Z

    So the counter moved, not the clock — the fault's newest occurrence predates the close, which is the one thing "still happening" has to mean. Root cause and fix: LogIncidentFiler.Reopen reopened on the sole condition that the issue was closed, while the comment-debt clock pays a final count for incidents whose fault has stopped (the fix, Systemorph/MeshWeaver.Plugins#2014, is written but NOT yet merged — it makes a reopen require incident.LastSeen > issue.ClosedAt).

    This restores the close exactly as it stood; nothing is asserted about the defect itself. If the fault happens again, the next occurrence postdates this close and reopens the issue on evidence that means what it says.

    One of 53 issues whose reopen was measured stale (129 bot reopens on 2026-09-17, 125 of them inside the 07:51-07:53Z sweep; 76 postdate their close and were left open). Measured and re-closed 2026-09-17 by Claude Opus 5.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions