Skip to content

Message senders and targets are addressing the mesh router hub directly instead of using the correct portal/import hubs #1113

Closed

Description

@meshweaver-cloud

What is failing

The MeshWeaver.Messaging.MessageHub (the mesh router) is receiving messages that address it as either the sender or the target — meaning work is being dispatched directly to or from the router hub. The router hub must only route messages, not execute work. Work should be dispatched through dedicated session portal hubs (for REST/Blazor/MCP) or import hubs (for bulk imports).

Probable cause

Confidence: high — the log message itself is a built-in guard that explicitly describes the issue.

Message producers (portal hubs, API token handlers, etc.) are either:

  • Sending messages with mesh/... as the sender address instead of their own portal hub address
  • Targeting mesh/... directly with work messages instead of routing through the correct functional hub

This appears across multiple hub instances and message types (RawJson, PatchDataChangeRequest, ValidateTokenRequest/Response), affecting the LossModelling, BinaryClicker, Claims, Feedback, and other model/work hubs. The guard reports once per role+type combination and then self-suppresses for that hub, so the 801 occurrences represent 801 distinct message-type+role combinations that have been detected since deployment.

Impact

  • 801 misrouted message types detected in ~8 hours across 6 memex-portal pods
  • Each misaddressed message was detected and logged but the router likely still attempted to forward it (or dropped it) — potential functional impact on the targeted model/work hubs depending on whether the message eventually reached the correct handler
  • This is an infrastructure/architecture issue rather than a code bug: the messaging topology needs to be corrected so that portal hubs use their own identity as sender and target work through the correct hub hierarchy

Where to look

  • MeshWeaver.Messaging.MessageHub — the guard that logs this when it detects the mesh hub being used as a work participant
  • Hub initialization/address assignment for portal hubs (how does a portal hub get its identity?)
  • Message dispatch paths that may be resolving the router address instead of the caller's hub address

Evidence

Fingerprint b6dc7a15e653fc41
Category MeshWeaver.Messaging.MessageHub
Severity Error
Namespace memex
Pods memex-portal-deployment-7f65668566-qdk6j, memex-portal-deployment-84855f7679-z4rsp, memex-portal-deployment-6956d54bb5-l6mxv, memex-portal-deployment-666657df96-jq49m, memex-portal-deployment-8476467cb6-nkbhh, memex-portal-deployment-575dc48bfb-wq7s8
Occurrences 840
First seen 2026-08-10 01:08:41Z
Last seen 2026-08-10 08:59:17Z
Recent log lines
2026-08-10 07:12:03Z memex-portal-deployment-575dc48bfb-wq7s8 fail: MeshWeaver.Messaging.MessageHub[0]
      ROUTER_TRAFFIC: RawJson has the mesh hub as sender (sender: mesh/l_cWjTNjd0Gx9zIk1SoW_A, target: UWDeepfield/UwAttention). The mesh hub is the ROUTER and must not execute work — it belongs on a hub of its own (session portal hub for REST/Blazor/MCP, import hub for bulk imports). Reported once per role+type for this hub.
2026-08-10 07:12:06Z memex-portal-deployment-575dc48bfb-wq7s8 fail: MeshWeaver.Messaging.MessageHub[0]
      ROUTER_TRAFFIC: RawJson has the mesh hub as sender (sender: mesh/l_cWjTNjd0Gx9zIk1SoW_A, target: Store/Coupon). The mesh hub is the ROUTER and must not execute work — it belongs on a hub of its own (session portal hub for REST/Blazor/MCP, import hub for bulk imports). Reported once per role+type for this hub.
2026-08-10 07:14:20Z memex-portal-deployment-575dc48bfb-wq7s8 fail: MeshWeaver.Messaging.MessageHub[0]
      ROUTER_TRAFFIC: RawJson has the mesh hub as sender (sender: mesh/l_cWjTNjd0Gx9zIk1SoW_A, target: Admin/_LogIncident/b6dc7a15e653fc41/_Thread/triage-the-log-incident-at-admin-loginci-9db8). The mesh hub is the ROUTER and must not execute work — it belongs on a hub of its own (session portal hub for REST/Blazor/MCP, import hub for bulk imports). Reported once per role+type for this hub.
2026-08-10 07:20:13Z memex-portal-deployment-575dc48bfb-wq7s8 fail: MeshWeaver.Messaging.MessageHub[0]
      ROUTER_TRAFFIC: ClickedEvent has the mesh hub as target (sender: portal/-D8cD8EIpmGWtuqjqumwotTOUL6f1rbP3nM2Tfhpi-Q, target: mesh/l_cWjTNjd0Gx9zIk1SoW_A). The mesh hub is the ROUTER and must not execute work — it belongs on a hub of its own (session portal hub for REST/Blazor/MCP, import hub for bulk imports). Reported once per role+type for this hub.
2026-08-10 07:21:02Z memex-portal-deployment-575dc48bfb-wq7s8 fail: MeshWeaver.Messaging.MessageHub[0]
      ROUTER_TRAFFIC: UnsubscribeRequest has the mesh hub as target (sender: cache/l_cWjTNjd0Gx9zIk1SoW_A, target: mesh/l_cWjTNjd0Gx9zIk1SoW_A). The mesh hub is the ROUTER and must not execute work — it belongs on a hub of its own (session portal hub for REST/Blazor/MCP, import hub for bulk imports). Reported once per role+type for this hub.
2026-08-10 07:34:22Z memex-portal-deployment-575dc48bfb-wq7s8 fail: MeshWeaver.Messaging.MessageHub[0]
      ROUTER_TRAFFIC: RawJson has the mesh hub as sender (sender: mesh/l_cWjTNjd0Gx9zIk1SoW_A, target: Instances/_Notification/83nZe9gZYEShvHp58hDNlg). The mesh hub is the ROUTER and must not execute work — it belongs on a hub of its own (session portal hub for REST/Blazor/MCP, import hub for bulk imports). Reported once per role+type for this hub.
2026-08-10 07:34:22Z memex-portal-deployment-575dc48bfb-wq7s8 fail: MeshWeaver.Messaging.MessageHub[0]
      ROUTER_TRAFFIC: RawJson has the mesh hub as sender (sender: mesh/l_cWjTNjd0Gx9zIk1SoW_A, target: Instances/Deployment/_Activity/compile-202608100109308946cd9c50d826c47349f494db88ce96282). The mesh hub is the ROUTER and must not execute work — it belongs on a hub of its own (session portal hub for REST/Blazor/MCP, import hub for bulk imports). Reported once per role+type for this hub.
2026-08-10 07:34:22Z memex-portal-deployment-575dc48bfb-wq7s8 fail: MeshWeaver.Messaging.MessageHub[0]
      ROUTER_TRAFFIC: RawJson has the mesh hub as sender (sender: mesh/l_cWjTNjd0Gx9zIk1SoW_A, target: Instances/_Activity/import-a62727f11a4703b8-20260809175155703-4f4c6329). The mesh hub is the ROUTER and must not execute work — it belongs on a hub of its own (session portal hub for REST/Blazor/MCP, import hub for bulk imports). Reported once per role+type for this hub.
2026-08-10 07:34:22Z memex-portal-deployment-575dc48bfb-wq7s8 fail: MeshWeaver.Messaging.MessageHub[0]
      ROUTER_TRAFFIC: RawJson has the mesh hub as sender (sender: mesh/l_cWjTNjd0Gx9zIk1SoW_A, target: Instances/Skill/deployment-activity). The mesh hub is the ROUTER and must not execute work — it belongs on a hub of its own (session portal hub for REST/Blazor/MCP, import hub for bulk imports). Reported once per role+type for this hub.
2026-08-10 08:41:24Z memex-portal-deployment-575dc48bfb-wq7s8 fail: MeshWeaver.Messaging.MessageHub[0]
      ROUTER_TRAFFIC: RawJson has the mesh hub as sender (sender: mesh/l_cWjTNjd0Gx9zIk1SoW_A, target: Store/Catalog). The mesh hub is the ROUTER and must not execute work — it belongs on a hub of its own (session portal hub for REST/Blazor/MCP, import hub for bulk imports). Reported once per role+type for this hub.

Opened automatically from Admin/_LogIncident/b6dc7a15e653fc41. Recurrences are folded into this issue rather than opening new ones.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions