Skip to content

ExecuteScript hangs for the full 60s and logs a dispatch failure when the target node's hub is not found, instead of failing fast #2918

Closed

Description

@meshweaver-cloud

What is failing

The ExecuteScript MCP tool dispatches its ExecuteScriptRequest to the target Code node's hub through the caller's per-caller MCP session hub. When the target hub cannot be resolved, the request simply gets no response and expires after the 60-second default — the caller receives a timeout instead of a clear "target not found / not executable" error.

Probable cause

Medium confidence: the target rbuergi/LokiProbe0831 does not exist anywhere in the mesh (no node by that id matches under the user's namespace), so routing could not deliver the request and it expired undelivered. This is the same hang-until-timeout failure class as the portal-session fix documented in More reliable REST, MCP and gRPC access (2026-07-29) — but on the delivery side (unresolvable target hub) rather than the session side.

Caveat: with a single sample and no application stack frame, the evidence cannot distinguish between the request being dropped by routing (hub not found) and being delivered to a hub that never responded — the exception text names both. What is certain from the message is that no response arrived within the window.

Impact

Minimal by the numbers: a single occurrence on one pod, in the caller's own namespace, against a probe-named target. The immediate effect is one wasted minute for one MCP caller. The concern is systemic rather than acute: every call against a nonexistent or non-executable target pays a full 60s timeout, which agents can easily multiply (retry loops), and the timeout path costs a server-side error log per call.

Where to look

  • MeshWeaver.AI.MeshOperations — the ExecuteScript tool handler and its dispatch of ExecuteScriptRequest to the target path via the per-caller MCP session hub (portal/mcp-rbuergi-*). The tool contract is documented in Executing Scripts via MCP, including the Timeout status.
  • The failure mode to fix: resolve the target node (existence and isExecutable) before dispatching, and/or have routing return an explicit "target hub not found" response the tool can surface, rather than letting the request/response layer time out.
  • Related but not duplicate: ContentRoute: System.TimeoutException (incident ade144cc74e8454b) shows the same request/response timeout class in a different category — worth checking whether the undeliverable-request path is shared.

No existing incident or issue covers this category/component; this does not appear to be a duplicate.


Evidence

Fingerprint 6094e6b21967f47b
Category MeshWeaver.AI.MeshOperations
Severity Error
Exception System.TimeoutException
Namespace memex-cloud
Pods memex-portal-deployment-9bf79b774-b8dl6
Occurrences 1
First seen 2026-08-31 18:38:59Z
Last seen 2026-08-31 18:38:59Z
Recent log lines
2026-08-31 18:38:59Z memex-portal-deployment-9bf79b774-b8dl6 fail: MeshWeaver.AI.MeshOperations[0]
      ExecuteScript dispatch failed for rbuergi/LokiProbe0831
      System.TimeoutException: No response received in hub portal/mcp-rbuergi-efe3e320 within 00:01:00 for request ExecuteScriptRequest (id=Jit9QIu2Zk2AtiiKKLLbJA) → target rbuergi/LokiProbe0831. The request may have been undeliverable or the target hub was not found.

Opened automatically from Admin/_LogIncident/6094e6b21967f47b. Recurrences are folded into this issue rather than opening new ones.

Activity

  1. added a commit that references this issue on Sep 1, 2026
  2. systemorph-com commented on Sep 17, 2026

    @systemorph-com
    Contributor

    Reopened — this fault is happening again.

    Still occurring: 1 more since the last update (2 total).

    • Last seen: 2026-08-31 18:40:12Z
    • Pods: memex-portal-deployment-9bf79b774-b8dl6, memex-portal-deployment-9bf79b774-nvnzz
    Recent log lines
    2026-08-31 18:38:59Z memex-portal-deployment-9bf79b774-b8dl6 fail: MeshWeaver.AI.MeshOperations[0]
          ExecuteScript dispatch failed for rbuergi/LokiProbe0831
          System.TimeoutException: No response received in hub portal/mcp-rbuergi-efe3e320 within 00:01:00 for request ExecuteScriptRequest (id=Jit9QIu2Zk2AtiiKKLLbJA) → target rbuergi/LokiProbe0831. The request may have been undeliverable or the target hub was not found.
    2026-08-31 18:40:12Z memex-portal-deployment-9bf79b774-nvnzz fail: MeshWeaver.AI.MeshOperations[0]
          ExecuteScript dispatch failed for rbuergi/LokiProbe0831
          System.TimeoutException: No response received in hub portal/mcp-rbuergi-2d8f0991 within 00:01:00 for request ExecuteScriptRequest (id=PVusiCAPokOVmkqk1NM0gg) → target rbuergi/LokiProbe0831. The request may have been undeliverable or the target hub was not found.
    
  3. rbuergi commented on Sep 17, 2026

    @rbuergi
    Contributor

    Re-closed — the reopen above was on stale evidence (#4577).

    The recurrence bot reopened this at 2026-09-17T07:52:27Z under the headline "this fault is happening again", but the occurrence its own comment cites is OLDER than the close it reverted:

    the bot's Last seen: the close it reverted
    2026-08-31 18:40:12Z 2026-09-01T14:37:19Z

    So the counter moved, not the clock — the fault's newest occurrence predates the close, which is the one thing "still happening" has to mean. Root cause and fix: LogIncidentFiler.Reopen reopened on the sole condition that the issue was closed, while the comment-debt clock pays a final count for incidents whose fault has stopped (the fix, Systemorph/MeshWeaver.Plugins#2014, is written but NOT yet merged — it makes a reopen require incident.LastSeen > issue.ClosedAt).

    This restores the close exactly as it stood; nothing is asserted about the defect itself. If the fault happens again, the next occurrence postdates this close and reopens the issue on evidence that means what it says.

    One of 53 issues whose reopen was measured stale (129 bot reopens on 2026-09-17, 125 of them inside the 07:51-07:53Z sweep; 76 postdate their close and were left open). Measured and re-closed 2026-09-17 by Claude Opus 5.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions