Repository navigation
nodeops hub posts GetDataRequest messages without AccessContext in PostPipeline #1253
Description
What is failing
The nodeops hub is dispatching GetDataRequest messages to FutuRe/GroupAnalysis/Source/* nodes through the post pipeline without an AccessContext. The pipeline guard in MeshWeaver.AccessContext.PostPipeline rejects these messages — no identity is attached, so downstream receivers see a null user context and the message is effectively dropped.
Probable cause
The nodeops hub (or the caller that initiates the batch of GetDataRequest messages against all the sources under FutuRe/GroupAnalysis/Source/) does not set an AccessContext on the post before sending. The log's own guidance points to two likely root causes:
- A batch/background operation running on the hub without first calling
AccessService.ImpersonateAsSystemor usingPostOptions.ImpersonateAsHub. - A code path that posts messages outside the scope of an authenticated HTTP/signalR circuit, so no
CircuitContext(and therefore noAccessContext) is available.
All 22 occurrences share the same millisecond timestamp (05:53:33.743) on the same pod, which strongly suggests a single bulk operation triggered the burst — e.g., a node (re)load or refresh loop in nodeops that iterates source nodes and posts a GetDataRequest for each.
Confidence: Moderate. The log explicitly names the missing piece and the symptom is clear, but without the source code of the nodeops hub's batch logic I cannot pinpoint the exact line.
Impact
Low severity. 22 occurrences in a single burst on one pod, and there is no exception — the pipeline guard logs and presumably skips the post. Affected FutuRe/GroupAnalysis data sources (FutuReDataCube, AnalysisContent, ProfitabilityLayoutAreas, FutuReDataLoader, ExternalDependencies) would not receive the data requests they needed. Users who rely on these sources may see stale or missing data until a retry with proper context succeeds.
Where to look
- Hub:
portal/nodeops-*— thenodeopshub that orchestrates operations on data sources. - Pipeline:
MeshWeaver.AccessContext.PostPipeline— the guard that logs this message. The callers ofPostorSendAsyncon this hub that loop overFutuRe/GroupAnalysis/Source/*targets. - Reference:
AccessContextPropagation.md/ the internal guidance referenced in the log (feedback_access_context_always_set).
Evidence
| Fingerprint | d67b48fe47ffcb8d |
| Category | MeshWeaver.AccessContext |
| Severity | Error |
| Namespace | memex-cloud |
| Pods | memex-portal-deployment-78bb6bfd5-fpvxk |
| Occurrences | 22 |
| First seen | 2026-08-12 05:53:33Z |
| Last seen | 2026-08-12 05:53:33Z |
Recent log lines
2026-08-12 05:53:33Z memex-portal-deployment-78bb6bfd5-fpvxk fail: MeshWeaver.AccessContext[0]
PostPipeline: AccessContext must never be null for an application post — no identity, no delivery. hub=portal/nodeops-gB4Xuk0tdUCT1waPIjtwZg, message=GetDataRequest, target=FutuRe/GroupAnalysis/Source/FutuReDataCube was posted with no AccessContext (no Context, no CircuitContext). The post lost the user identity. Wire its source: user-context from the circuit/HTTP user; infrastructure via AccessService.ImpersonateAsSystem / PostOptions.ImpersonateAsHub; threads/activities via AccessContextScope.FromNode. See AccessContextPropagation.md / feedback_access_context_always_set.
2026-08-12 05:53:33Z memex-portal-deployment-78bb6bfd5-fpvxk fail: MeshWeaver.AccessContext[0]
PostPipeline: AccessContext must never be null for an application post — no identity, no delivery. hub=portal/nodeops-gB4Xuk0tdUCT1waPIjtwZg, message=GetDataRequest, target=FutuRe/GroupAnalysis/Source/AnalysisContent was posted with no AccessContext (no Context, no CircuitContext). The post lost the user identity. Wire its source: user-context from the circuit/HTTP user; infrastructure via AccessService.ImpersonateAsSystem / PostOptions.ImpersonateAsHub; threads/activities via AccessContextScope.FromNode. See AccessContextPropagation.md / feedback_access_context_always_set.
2026-08-12 05:53:33Z memex-portal-deployment-78bb6bfd5-fpvxk fail: MeshWeaver.AccessContext[0]
PostPipeline: AccessContext must never be null for an application post — no identity, no delivery. hub=portal/nodeops-gB4Xuk0tdUCT1waPIjtwZg, message=GetDataRequest, target=FutuRe/GroupAnalysis/Source/ProfitabilityLayoutAreas was posted with no AccessContext (no Context, no CircuitContext). The post lost the user identity. Wire its source: user-context from the circuit/HTTP user; infrastructure via AccessService.ImpersonateAsSystem / PostOptions.ImpersonateAsHub; threads/activities via AccessContextScope.FromNode. See AccessContextPropagation.md / feedback_access_context_always_set.
2026-08-12 05:53:33Z memex-portal-deployment-78bb6bfd5-fpvxk fail: MeshWeaver.AccessContext[0]
PostPipeline: AccessContext must never be null for an application post — no identity, no delivery. hub=portal/nodeops-gB4Xuk0tdUCT1waPIjtwZg, message=GetDataRequest, target=FutuRe/GroupAnalysis/Source/FutuReDataLoader was posted with no AccessContext (no Context, no CircuitContext). The post lost the user identity. Wire its source: user-context from the circuit/HTTP user; infrastructure via AccessService.ImpersonateAsSystem / PostOptions.ImpersonateAsHub; threads/activities via AccessContextScope.FromNode. See AccessContextPropagation.md / feedback_access_context_always_set.
2026-08-12 05:53:33Z memex-portal-deployment-78bb6bfd5-fpvxk fail: MeshWeaver.AccessContext[0]
PostPipeline: AccessContext must never be null for an application post — no identity, no delivery. hub=portal/nodeops-gB4Xuk0tdUCT1waPIjtwZg, message=GetDataRequest, target=FutuRe/GroupAnalysis/Source/ExternalDependencies was posted with no AccessContext (no Context, no CircuitContext). The post lost the user identity. Wire its source: user-context from the circuit/HTTP user; infrastructure via AccessService.ImpersonateAsSystem / PostOptions.ImpersonateAsHub; threads/activities via AccessContextScope.FromNode. See AccessContextPropagation.md / feedback_access_context_always_set.
Opened automatically from Admin/_LogIncident/d67b48fe47ffcb8d. Recurrences are folded into this issue rather than opening new ones.
Activity
systemorph-com commentedon Sep 17, 2026
Reopened — this fault is happening again.
Still occurring: 2106 more since the last update (2128 total).
memex-portal-deployment-78bb6bfd5-fpvxk,memex-portal-deployment-7d4b5684bb-9lrxl,memex-portal-deployment-764d5fbf-nn6nj,memex-portal-deployment-98785f547-zg442,memex-portal-deployment-78db689895-jpqs2,memex-portal-deployment-cb69ff6-vjbfr,memex-portal-deployment-6f68bcbc76-qx8sr,memex-portal-deployment-7bcb749b9f-7fqd5,memex-portal-deployment-7db656d9d9-nbfzk,memex-portal-deployment-6d58c699f8-9t675,memex-portal-deployment-76cbbdd5f-9mtkj,memex-portal-deployment-76cbbdd5f-82mvq,memex-portal-deployment-6f64bf4b56-2mzrf,memex-portal-deployment-6c46b84848-hnh9v,memex-portal-deployment-7d8fd6f84b-47cmq,memex-portal-deployment-7d8fd6f84b-mjp87,memex-portal-deployment-7d8fd6f84b-s7sr8,memex-portal-deployment-7d8fd6f84b-slmzp,memex-portal-deployment-856d4474c-hr5rd,memex-portal-deployment-66cfbf759-gmgl4,memex-portal-deployment-799c674567-p5lr8,memex-portal-deployment-7767bf7cd7-ffrhj,memex-portal-deployment-7767bf7cd7-rwhxm,memex-portal-deployment-75597864c8-wwfnf,memex-portal-deployment-6dd4fbd566-wv9p7,memex-portal-deployment-9bf79b774-tk88b,memex-portal-deployment-9bf79b774-jwrng,memex-portal-deployment-9bf79b774-nvnzz,memex-portal-deployment-9bf79b774-b8dl6,memex-portal-deployment-85859db964-q5zcm,memex-portal-deployment-85859db964-xk5gw,memex-portal-deployment-77c854cfcf-clz9l,memex-portal-deployment-77c854cfcf-ns2r5,memex-portal-deployment-77cfc55bfc-767mq,memex-portal-deployment-5b9cbf46dd-fxbhd,memex-portal-deployment-5b9cbf46dd-zn7nfRecent log lines
rbuergi commentedon Sep 18, 2026
Re-closing — the defect is fixed at a named call site in current
origin/mainLast seen
2026-09-06 05:54:48Z(theLast seen:in the bot's own reopen comment, 2026-09-17T07:53:23Z) — quiet 12.0 days as of 2026-09-18T06:05Z, while ingest is live (incidents were still being updated 2026-09-18T05:42Z).The fix is PR #1279 ("fix(compile): read
@@includes as System so a refused post cannot park a NodeType"), merged 2026-08-12T12:22:44Z — two seconds before this issue was originally closed. It is in currentorigin/main, and the code cites this issue by number:MeshNodeCompilationService.cs:217quotes this incident verbatim — "Issue #1253 (memex-cloud 2026-08-12: 22 refused reads in one millisecond, the five@@targets ofFutuRe/LocalAnalysis/Source/ExternalDependenciesin file order)" — and the read it documents is nowReadCompileSourceNode, which wraps every compile-path node read inaccessService.RunAsSystem(...). That is exactly the producer this incident named: aGetDataRequestissued fromportal/nodeops-*atFutuRe/GroupAnalysis/Source/*, refused by the never-nullPostPipelineguard because the ambientAsyncLocalAccessContext did not survive the continuation hop.On the residual counter (
2106 more since the last update). That number is not 2,106 more of this producer. The incident's fingerprint masks the identifying fields:LogLineParser.Normalize(MeshWeaver.Plugins,src/MeshWeaver.Observability.Contract/LogLineParser.cs:254-265) replaces any multi-segment slash path with{path}, sohub=portal/nodeops-…andtarget=FutuRe/GroupAnalysis/Source/…both collapse and every null-AccessContextGetDataRequestpost, from any hub to any target, folds onto fingerprintd67b48fe47ffcb8d. The counter is a shared bucket, not this call site.The 2026-09-17T07:51–07:53Z reopen was part of a bulk wave whose predicate read an accumulated counter rather than "occurred since the close" — analysed in Reading a Recurrence Reopen.
Closing a dormant incident is safe by design: a recurrence re-opens the issue automatically with fresh evidence, so nothing is lost by closing while quiet — and if a different producer starts posting without an AccessContext, that reopen will carry the pod and timestamp to find it.
systemorph-com commentedon Sep 22, 2026
Reopened — this fault is happening again.
Still occurring: 2 more since the last update (2130 total).
memex-portal-deployment-5bb495d47b-8bdwp,memex-portal-deployment-78bb6bfd5-fpvxk,memex-portal-deployment-7d4b5684bb-9lrxl,memex-portal-deployment-764d5fbf-nn6nj,memex-portal-deployment-98785f547-zg442,memex-portal-deployment-78db689895-jpqs2,memex-portal-deployment-cb69ff6-vjbfr,memex-portal-deployment-6f68bcbc76-qx8sr,memex-portal-deployment-7bcb749b9f-7fqd5,memex-portal-deployment-7db656d9d9-nbfzk,memex-portal-deployment-6d58c699f8-9t675,memex-portal-deployment-76cbbdd5f-9mtkj,memex-portal-deployment-76cbbdd5f-82mvq,memex-portal-deployment-6f64bf4b56-2mzrf,memex-portal-deployment-6c46b84848-hnh9v,memex-portal-deployment-7d8fd6f84b-47cmq,memex-portal-deployment-7d8fd6f84b-mjp87,memex-portal-deployment-7d8fd6f84b-s7sr8,memex-portal-deployment-7d8fd6f84b-slmzp,memex-portal-deployment-856d4474c-hr5rd,memex-portal-deployment-66cfbf759-gmgl4,memex-portal-deployment-799c674567-p5lr8,memex-portal-deployment-7767bf7cd7-ffrhj,memex-portal-deployment-7767bf7cd7-rwhxm,memex-portal-deployment-75597864c8-wwfnf,memex-portal-deployment-6dd4fbd566-wv9p7,memex-portal-deployment-9bf79b774-tk88b,memex-portal-deployment-9bf79b774-jwrng,memex-portal-deployment-9bf79b774-nvnzz,memex-portal-deployment-9bf79b774-b8dl6,memex-portal-deployment-85859db964-q5zcm,memex-portal-deployment-85859db964-xk5gw,memex-portal-deployment-77c854cfcf-clz9l,memex-portal-deployment-77c854cfcf-ns2r5,memex-portal-deployment-77cfc55bfc-767mq,memex-portal-deployment-5b9cbf46dd-fxbhd,memex-portal-deployment-5b9cbf46dd-zn7nfRecent log lines
Re-addressed by the current identity function: this incident inherited
d67b48fe47ffcb8d(Systemorph/MeshWeaver.Plugins#1796). Those nodes are superseded and will not fold, file or comment again.systemorph-com commentedon Sep 22, 2026
Still occurring: 2 more since the last update (2132 total).
memex-portal-deployment-5bb495d47b-8bdwp,memex-portal-deployment-78bb6bfd5-fpvxk,memex-portal-deployment-7d4b5684bb-9lrxl,memex-portal-deployment-764d5fbf-nn6nj,memex-portal-deployment-98785f547-zg442,memex-portal-deployment-78db689895-jpqs2,memex-portal-deployment-cb69ff6-vjbfr,memex-portal-deployment-6f68bcbc76-qx8sr,memex-portal-deployment-7bcb749b9f-7fqd5,memex-portal-deployment-7db656d9d9-nbfzk,memex-portal-deployment-6d58c699f8-9t675,memex-portal-deployment-76cbbdd5f-9mtkj,memex-portal-deployment-76cbbdd5f-82mvq,memex-portal-deployment-6f64bf4b56-2mzrf,memex-portal-deployment-6c46b84848-hnh9v,memex-portal-deployment-7d8fd6f84b-47cmq,memex-portal-deployment-7d8fd6f84b-mjp87,memex-portal-deployment-7d8fd6f84b-s7sr8,memex-portal-deployment-7d8fd6f84b-slmzp,memex-portal-deployment-856d4474c-hr5rd,memex-portal-deployment-66cfbf759-gmgl4,memex-portal-deployment-799c674567-p5lr8,memex-portal-deployment-7767bf7cd7-ffrhj,memex-portal-deployment-7767bf7cd7-rwhxm,memex-portal-deployment-75597864c8-wwfnf,memex-portal-deployment-6dd4fbd566-wv9p7,memex-portal-deployment-9bf79b774-tk88b,memex-portal-deployment-9bf79b774-jwrng,memex-portal-deployment-9bf79b774-nvnzz,memex-portal-deployment-9bf79b774-b8dl6,memex-portal-deployment-85859db964-q5zcm,memex-portal-deployment-85859db964-xk5gw,memex-portal-deployment-77c854cfcf-clz9l,memex-portal-deployment-77c854cfcf-ns2r5,memex-portal-deployment-77cfc55bfc-767mq,memex-portal-deployment-5b9cbf46dd-fxbhd,memex-portal-deployment-5b9cbf46dd-zn7nf,memex-portal-deployment-86dbb4d55c-jj7gcRecent log lines
Re-addressed by the current identity function: this incident inherited
d67b48fe47ffcb8d(Systemorph/MeshWeaver.Plugins#1796). Those nodes are superseded and will not fold, file or comment again.systemorph-com commentedon Sep 23, 2026
Still occurring: 5 more since the last update (2137 total).
memex-portal-deployment-5bb495d47b-8bdwp,memex-portal-deployment-78bb6bfd5-fpvxk,memex-portal-deployment-7d4b5684bb-9lrxl,memex-portal-deployment-764d5fbf-nn6nj,memex-portal-deployment-98785f547-zg442,memex-portal-deployment-78db689895-jpqs2,memex-portal-deployment-cb69ff6-vjbfr,memex-portal-deployment-6f68bcbc76-qx8sr,memex-portal-deployment-7bcb749b9f-7fqd5,memex-portal-deployment-7db656d9d9-nbfzk,memex-portal-deployment-6d58c699f8-9t675,memex-portal-deployment-76cbbdd5f-9mtkj,memex-portal-deployment-76cbbdd5f-82mvq,memex-portal-deployment-6f64bf4b56-2mzrf,memex-portal-deployment-6c46b84848-hnh9v,memex-portal-deployment-7d8fd6f84b-47cmq,memex-portal-deployment-7d8fd6f84b-mjp87,memex-portal-deployment-7d8fd6f84b-s7sr8,memex-portal-deployment-7d8fd6f84b-slmzp,memex-portal-deployment-856d4474c-hr5rd,memex-portal-deployment-66cfbf759-gmgl4,memex-portal-deployment-799c674567-p5lr8,memex-portal-deployment-7767bf7cd7-ffrhj,memex-portal-deployment-7767bf7cd7-rwhxm,memex-portal-deployment-75597864c8-wwfnf,memex-portal-deployment-6dd4fbd566-wv9p7,memex-portal-deployment-9bf79b774-tk88b,memex-portal-deployment-9bf79b774-jwrng,memex-portal-deployment-9bf79b774-nvnzz,memex-portal-deployment-9bf79b774-b8dl6,memex-portal-deployment-85859db964-q5zcm,memex-portal-deployment-85859db964-xk5gw,memex-portal-deployment-77c854cfcf-clz9l,memex-portal-deployment-77c854cfcf-ns2r5,memex-portal-deployment-77cfc55bfc-767mq,memex-portal-deployment-5b9cbf46dd-fxbhd,memex-portal-deployment-5b9cbf46dd-zn7nf,memex-portal-deployment-86dbb4d55c-jj7gc,memex-portal-deployment-c54d5847c-bf27p,memex-portal-deployment-c54d5847c-t7cm2,memex-portal-deployment-c54d5847c-m9crdRecent log lines
Re-addressed by the current identity function: this incident inherited
d67b48fe47ffcb8d(Systemorph/MeshWeaver.Plugins#1796). Those nodes are superseded and will not fold, file or comment again.systemorph-com commentedon Sep 23, 2026
Reopened — this fault is happening again.
Still occurring: 7 more since the last update (2144 total).
memex-portal-deployment-5bb495d47b-8bdwp,memex-portal-deployment-78bb6bfd5-fpvxk,memex-portal-deployment-7d4b5684bb-9lrxl,memex-portal-deployment-764d5fbf-nn6nj,memex-portal-deployment-98785f547-zg442,memex-portal-deployment-78db689895-jpqs2,memex-portal-deployment-cb69ff6-vjbfr,memex-portal-deployment-6f68bcbc76-qx8sr,memex-portal-deployment-7bcb749b9f-7fqd5,memex-portal-deployment-7db656d9d9-nbfzk,memex-portal-deployment-6d58c699f8-9t675,memex-portal-deployment-76cbbdd5f-9mtkj,memex-portal-deployment-76cbbdd5f-82mvq,memex-portal-deployment-6f64bf4b56-2mzrf,memex-portal-deployment-6c46b84848-hnh9v,memex-portal-deployment-7d8fd6f84b-47cmq,memex-portal-deployment-7d8fd6f84b-mjp87,memex-portal-deployment-7d8fd6f84b-s7sr8,memex-portal-deployment-7d8fd6f84b-slmzp,memex-portal-deployment-856d4474c-hr5rd,memex-portal-deployment-66cfbf759-gmgl4,memex-portal-deployment-799c674567-p5lr8,memex-portal-deployment-7767bf7cd7-ffrhj,memex-portal-deployment-7767bf7cd7-rwhxm,memex-portal-deployment-75597864c8-wwfnf,memex-portal-deployment-6dd4fbd566-wv9p7,memex-portal-deployment-9bf79b774-tk88b,memex-portal-deployment-9bf79b774-jwrng,memex-portal-deployment-9bf79b774-nvnzz,memex-portal-deployment-9bf79b774-b8dl6,memex-portal-deployment-85859db964-q5zcm,memex-portal-deployment-85859db964-xk5gw,memex-portal-deployment-77c854cfcf-clz9l,memex-portal-deployment-77c854cfcf-ns2r5,memex-portal-deployment-77cfc55bfc-767mq,memex-portal-deployment-5b9cbf46dd-fxbhd,memex-portal-deployment-5b9cbf46dd-zn7nf,memex-portal-deployment-86dbb4d55c-jj7gc,memex-portal-deployment-c54d5847c-bf27p,memex-portal-deployment-c54d5847c-t7cm2,memex-portal-deployment-c54d5847c-m9crdRecent log lines
Re-addressed by the current identity function: this incident inherited
d67b48fe47ffcb8d(Systemorph/MeshWeaver.Plugins#1796). Those nodes are superseded and will not fold, file or comment again.rbuergi commentedon Sep 23, 2026
Duplicate of #5227 for everything since the 2026-09-22 reopen.
portal/nodeops-*readingFutuRe/GroupAnalysis/Source/*during compiles. fix(compile): read @@ includes as System so a refused post cannot park a NodeType #1279 fixed it (see the 2026-09-18 comment above).hub=portal/reads-…, message=GetDataRequest, target=<page path>:Store,Doc/AI,Home,ThreeBody/Orbits,Crm/Guide,AgenticOffice/01-Willkommen/Exercise. That is Portal posts GetDataRequest to Home with no AccessContext — post is rejected at the identity gate #5227's line. It lands here only because the superseded fingerprintd67b48fe47ffcb8dmasks the paths.ApplicationPage's pre-rendered-body read, fixed in Systemorph/MeshWeaver.Plugins#2333 with a negative control that reproduces the line verbatim.Tracking continues on #5227: #5227 (comment)
systemorph-com commentedon Sep 26, 2026
Left closed — this ticket was closed as NOT PLANNED, which is a decision rather than a claim that the fault stopped. The occurrences below are recorded here; if the decision should change, that is a person's call and not a recurrence's.
Still occurring: 1 more since the last update (2145 total).
memex-portal-deployment-5bb495d47b-8bdwp,memex-portal-deployment-78bb6bfd5-fpvxk,memex-portal-deployment-7d4b5684bb-9lrxl,memex-portal-deployment-764d5fbf-nn6nj,memex-portal-deployment-98785f547-zg442,memex-portal-deployment-78db689895-jpqs2,memex-portal-deployment-cb69ff6-vjbfr,memex-portal-deployment-6f68bcbc76-qx8sr,memex-portal-deployment-7bcb749b9f-7fqd5,memex-portal-deployment-7db656d9d9-nbfzk,memex-portal-deployment-6d58c699f8-9t675,memex-portal-deployment-76cbbdd5f-9mtkj,memex-portal-deployment-76cbbdd5f-82mvq,memex-portal-deployment-6f64bf4b56-2mzrf,memex-portal-deployment-6c46b84848-hnh9v,memex-portal-deployment-7d8fd6f84b-47cmq,memex-portal-deployment-7d8fd6f84b-mjp87,memex-portal-deployment-7d8fd6f84b-s7sr8,memex-portal-deployment-7d8fd6f84b-slmzp,memex-portal-deployment-856d4474c-hr5rd,memex-portal-deployment-66cfbf759-gmgl4,memex-portal-deployment-799c674567-p5lr8,memex-portal-deployment-7767bf7cd7-ffrhj,memex-portal-deployment-7767bf7cd7-rwhxm,memex-portal-deployment-75597864c8-wwfnf,memex-portal-deployment-6dd4fbd566-wv9p7,memex-portal-deployment-9bf79b774-tk88b,memex-portal-deployment-9bf79b774-jwrng,memex-portal-deployment-9bf79b774-nvnzz,memex-portal-deployment-9bf79b774-b8dl6,memex-portal-deployment-85859db964-q5zcm,memex-portal-deployment-85859db964-xk5gw,memex-portal-deployment-77c854cfcf-clz9l,memex-portal-deployment-77c854cfcf-ns2r5,memex-portal-deployment-77cfc55bfc-767mq,memex-portal-deployment-5b9cbf46dd-fxbhd,memex-portal-deployment-5b9cbf46dd-zn7nf,memex-portal-deployment-86dbb4d55c-jj7gc,memex-portal-deployment-c54d5847c-bf27p,memex-portal-deployment-c54d5847c-t7cm2,memex-portal-deployment-c54d5847c-m9crd,memex-portal-deployment-589ff8f895-c8qvjRecent log lines
Re-addressed by the current identity function: this incident inherited
d67b48fe47ffcb8d(Systemorph/MeshWeaver.Plugins#1796). Those nodes are superseded and will not fold, file or comment again.systemorph-com commentedon Sep 26, 2026
Left closed — this ticket was closed as NOT PLANNED, which is a decision rather than a claim that the fault stopped. The occurrences below are recorded here; if the decision should change, that is a person's call and not a recurrence's.
Still occurring: 2 more since the last update (2147 total).
memex-portal-deployment-5bb495d47b-8bdwp,memex-portal-deployment-78bb6bfd5-fpvxk,memex-portal-deployment-7d4b5684bb-9lrxl,memex-portal-deployment-764d5fbf-nn6nj,memex-portal-deployment-98785f547-zg442,memex-portal-deployment-78db689895-jpqs2,memex-portal-deployment-cb69ff6-vjbfr,memex-portal-deployment-6f68bcbc76-qx8sr,memex-portal-deployment-7bcb749b9f-7fqd5,memex-portal-deployment-7db656d9d9-nbfzk,memex-portal-deployment-6d58c699f8-9t675,memex-portal-deployment-76cbbdd5f-9mtkj,memex-portal-deployment-76cbbdd5f-82mvq,memex-portal-deployment-6f64bf4b56-2mzrf,memex-portal-deployment-6c46b84848-hnh9v,memex-portal-deployment-7d8fd6f84b-47cmq,memex-portal-deployment-7d8fd6f84b-mjp87,memex-portal-deployment-7d8fd6f84b-s7sr8,memex-portal-deployment-7d8fd6f84b-slmzp,memex-portal-deployment-856d4474c-hr5rd,memex-portal-deployment-66cfbf759-gmgl4,memex-portal-deployment-799c674567-p5lr8,memex-portal-deployment-7767bf7cd7-ffrhj,memex-portal-deployment-7767bf7cd7-rwhxm,memex-portal-deployment-75597864c8-wwfnf,memex-portal-deployment-6dd4fbd566-wv9p7,memex-portal-deployment-9bf79b774-tk88b,memex-portal-deployment-9bf79b774-jwrng,memex-portal-deployment-9bf79b774-nvnzz,memex-portal-deployment-9bf79b774-b8dl6,memex-portal-deployment-85859db964-q5zcm,memex-portal-deployment-85859db964-xk5gw,memex-portal-deployment-77c854cfcf-clz9l,memex-portal-deployment-77c854cfcf-ns2r5,memex-portal-deployment-77cfc55bfc-767mq,memex-portal-deployment-5b9cbf46dd-fxbhd,memex-portal-deployment-5b9cbf46dd-zn7nf,memex-portal-deployment-86dbb4d55c-jj7gc,memex-portal-deployment-c54d5847c-bf27p,memex-portal-deployment-c54d5847c-t7cm2,memex-portal-deployment-c54d5847c-m9crd,memex-portal-deployment-589ff8f895-c8qvjRecent log lines
Re-addressed by the current identity function: this incident inherited
d67b48fe47ffcb8d(Systemorph/MeshWeaver.Plugins#1796). Those nodes are superseded and will not fold, file or comment again.systemorph-com commentedon Sep 26, 2026
Left closed — this ticket was closed as NOT PLANNED, which is a decision rather than a claim that the fault stopped. The occurrences below are recorded here; if the decision should change, that is a person's call and not a recurrence's.
Still occurring: 2 more since the last update (2149 total).
memex-portal-deployment-5bb495d47b-8bdwp,memex-portal-deployment-78bb6bfd5-fpvxk,memex-portal-deployment-7d4b5684bb-9lrxl,memex-portal-deployment-764d5fbf-nn6nj,memex-portal-deployment-98785f547-zg442,memex-portal-deployment-78db689895-jpqs2,memex-portal-deployment-cb69ff6-vjbfr,memex-portal-deployment-6f68bcbc76-qx8sr,memex-portal-deployment-7bcb749b9f-7fqd5,memex-portal-deployment-7db656d9d9-nbfzk,memex-portal-deployment-6d58c699f8-9t675,memex-portal-deployment-76cbbdd5f-9mtkj,memex-portal-deployment-76cbbdd5f-82mvq,memex-portal-deployment-6f64bf4b56-2mzrf,memex-portal-deployment-6c46b84848-hnh9v,memex-portal-deployment-7d8fd6f84b-47cmq,memex-portal-deployment-7d8fd6f84b-mjp87,memex-portal-deployment-7d8fd6f84b-s7sr8,memex-portal-deployment-7d8fd6f84b-slmzp,memex-portal-deployment-856d4474c-hr5rd,memex-portal-deployment-66cfbf759-gmgl4,memex-portal-deployment-799c674567-p5lr8,memex-portal-deployment-7767bf7cd7-ffrhj,memex-portal-deployment-7767bf7cd7-rwhxm,memex-portal-deployment-75597864c8-wwfnf,memex-portal-deployment-6dd4fbd566-wv9p7,memex-portal-deployment-9bf79b774-tk88b,memex-portal-deployment-9bf79b774-jwrng,memex-portal-deployment-9bf79b774-nvnzz,memex-portal-deployment-9bf79b774-b8dl6,memex-portal-deployment-85859db964-q5zcm,memex-portal-deployment-85859db964-xk5gw,memex-portal-deployment-77c854cfcf-clz9l,memex-portal-deployment-77c854cfcf-ns2r5,memex-portal-deployment-77cfc55bfc-767mq,memex-portal-deployment-5b9cbf46dd-fxbhd,memex-portal-deployment-5b9cbf46dd-zn7nf,memex-portal-deployment-86dbb4d55c-jj7gc,memex-portal-deployment-c54d5847c-bf27p,memex-portal-deployment-c54d5847c-t7cm2,memex-portal-deployment-c54d5847c-m9crd,memex-portal-deployment-589ff8f895-c8qvj,memex-portal-deployment-589ff8f895-vw5jj,memex-portal-deployment-589ff8f895-ghgjtRecent log lines
Re-addressed by the current identity function: this incident inherited
d67b48fe47ffcb8d(Systemorph/MeshWeaver.Plugins#1796). Those nodes are superseded and will not fold, file or comment again.systemorph-com commentedon Sep 26, 2026
Left closed — this ticket was closed as NOT PLANNED, which is a decision rather than a claim that the fault stopped. The occurrences below are recorded here; if the decision should change, that is a person's call and not a recurrence's.
Still occurring: 1 more since the last update (2150 total).
memex-portal-deployment-5bb495d47b-8bdwp,memex-portal-deployment-78bb6bfd5-fpvxk,memex-portal-deployment-7d4b5684bb-9lrxl,memex-portal-deployment-764d5fbf-nn6nj,memex-portal-deployment-98785f547-zg442,memex-portal-deployment-78db689895-jpqs2,memex-portal-deployment-cb69ff6-vjbfr,memex-portal-deployment-6f68bcbc76-qx8sr,memex-portal-deployment-7bcb749b9f-7fqd5,memex-portal-deployment-7db656d9d9-nbfzk,memex-portal-deployment-6d58c699f8-9t675,memex-portal-deployment-76cbbdd5f-9mtkj,memex-portal-deployment-76cbbdd5f-82mvq,memex-portal-deployment-6f64bf4b56-2mzrf,memex-portal-deployment-6c46b84848-hnh9v,memex-portal-deployment-7d8fd6f84b-47cmq,memex-portal-deployment-7d8fd6f84b-mjp87,memex-portal-deployment-7d8fd6f84b-s7sr8,memex-portal-deployment-7d8fd6f84b-slmzp,memex-portal-deployment-856d4474c-hr5rd,memex-portal-deployment-66cfbf759-gmgl4,memex-portal-deployment-799c674567-p5lr8,memex-portal-deployment-7767bf7cd7-ffrhj,memex-portal-deployment-7767bf7cd7-rwhxm,memex-portal-deployment-75597864c8-wwfnf,memex-portal-deployment-6dd4fbd566-wv9p7,memex-portal-deployment-9bf79b774-tk88b,memex-portal-deployment-9bf79b774-jwrng,memex-portal-deployment-9bf79b774-nvnzz,memex-portal-deployment-9bf79b774-b8dl6,memex-portal-deployment-85859db964-q5zcm,memex-portal-deployment-85859db964-xk5gw,memex-portal-deployment-77c854cfcf-clz9l,memex-portal-deployment-77c854cfcf-ns2r5,memex-portal-deployment-77cfc55bfc-767mq,memex-portal-deployment-5b9cbf46dd-fxbhd,memex-portal-deployment-5b9cbf46dd-zn7nf,memex-portal-deployment-86dbb4d55c-jj7gc,memex-portal-deployment-c54d5847c-bf27p,memex-portal-deployment-c54d5847c-t7cm2,memex-portal-deployment-c54d5847c-m9crd,memex-portal-deployment-589ff8f895-c8qvj,memex-portal-deployment-589ff8f895-vw5jj,memex-portal-deployment-589ff8f895-ghgjt,memex-portal-deployment-56fbdcd48f-cbms9Recent log lines
Re-addressed by the current identity function: this incident inherited
d67b48fe47ffcb8d(Systemorph/MeshWeaver.Plugins#1796). Those nodes are superseded and will not fold, file or comment again.systemorph-com commentedon Sep 26, 2026
Left closed — this ticket was closed as NOT PLANNED, which is a decision rather than a claim that the fault stopped. The occurrences below are recorded here; if the decision should change, that is a person's call and not a recurrence's.
Still occurring: 1 more since the last update (2150 total).
memex-portal-deployment-5bb495d47b-8bdwp,memex-portal-deployment-78bb6bfd5-fpvxk,memex-portal-deployment-7d4b5684bb-9lrxl,memex-portal-deployment-764d5fbf-nn6nj,memex-portal-deployment-98785f547-zg442,memex-portal-deployment-78db689895-jpqs2,memex-portal-deployment-cb69ff6-vjbfr,memex-portal-deployment-6f68bcbc76-qx8sr,memex-portal-deployment-7bcb749b9f-7fqd5,memex-portal-deployment-7db656d9d9-nbfzk,memex-portal-deployment-6d58c699f8-9t675,memex-portal-deployment-76cbbdd5f-9mtkj,memex-portal-deployment-76cbbdd5f-82mvq,memex-portal-deployment-6f64bf4b56-2mzrf,memex-portal-deployment-6c46b84848-hnh9v,memex-portal-deployment-7d8fd6f84b-47cmq,memex-portal-deployment-7d8fd6f84b-mjp87,memex-portal-deployment-7d8fd6f84b-s7sr8,memex-portal-deployment-7d8fd6f84b-slmzp,memex-portal-deployment-856d4474c-hr5rd,memex-portal-deployment-66cfbf759-gmgl4,memex-portal-deployment-799c674567-p5lr8,memex-portal-deployment-7767bf7cd7-ffrhj,memex-portal-deployment-7767bf7cd7-rwhxm,memex-portal-deployment-75597864c8-wwfnf,memex-portal-deployment-6dd4fbd566-wv9p7,memex-portal-deployment-9bf79b774-tk88b,memex-portal-deployment-9bf79b774-jwrng,memex-portal-deployment-9bf79b774-nvnzz,memex-portal-deployment-9bf79b774-b8dl6,memex-portal-deployment-85859db964-q5zcm,memex-portal-deployment-85859db964-xk5gw,memex-portal-deployment-77c854cfcf-clz9l,memex-portal-deployment-77c854cfcf-ns2r5,memex-portal-deployment-77cfc55bfc-767mq,memex-portal-deployment-5b9cbf46dd-fxbhd,memex-portal-deployment-5b9cbf46dd-zn7nf,memex-portal-deployment-86dbb4d55c-jj7gc,memex-portal-deployment-c54d5847c-bf27p,memex-portal-deployment-c54d5847c-t7cm2,memex-portal-deployment-c54d5847c-m9crd,memex-portal-deployment-589ff8f895-c8qvj,memex-portal-deployment-589ff8f895-vw5jj,memex-portal-deployment-589ff8f895-ghgjt,memex-portal-deployment-56fbdcd48f-cbms9Recent log lines
Re-addressed by the current identity function: this incident inherited
d67b48fe47ffcb8d(Systemorph/MeshWeaver.Plugins#1796). Those nodes are superseded and will not fold, file or comment again.systemorph-com commentedon Oct 4, 2026
Left closed — this ticket was closed as NOT PLANNED, which is a decision rather than a claim that the fault stopped. The occurrences below are recorded here; if the decision should change, that is a person's call and not a recurrence's.
Still occurring: 1 more since the last update (2151 total).
memex-portal-deployment-5bb495d47b-8bdwp,memex-portal-deployment-78bb6bfd5-fpvxk,memex-portal-deployment-7d4b5684bb-9lrxl,memex-portal-deployment-764d5fbf-nn6nj,memex-portal-deployment-98785f547-zg442,memex-portal-deployment-78db689895-jpqs2,memex-portal-deployment-cb69ff6-vjbfr,memex-portal-deployment-6f68bcbc76-qx8sr,memex-portal-deployment-7bcb749b9f-7fqd5,memex-portal-deployment-7db656d9d9-nbfzk,memex-portal-deployment-6d58c699f8-9t675,memex-portal-deployment-76cbbdd5f-9mtkj,memex-portal-deployment-76cbbdd5f-82mvq,memex-portal-deployment-6f64bf4b56-2mzrf,memex-portal-deployment-6c46b84848-hnh9v,memex-portal-deployment-7d8fd6f84b-47cmq,memex-portal-deployment-7d8fd6f84b-mjp87,memex-portal-deployment-7d8fd6f84b-s7sr8,memex-portal-deployment-7d8fd6f84b-slmzp,memex-portal-deployment-856d4474c-hr5rd,memex-portal-deployment-66cfbf759-gmgl4,memex-portal-deployment-799c674567-p5lr8,memex-portal-deployment-7767bf7cd7-ffrhj,memex-portal-deployment-7767bf7cd7-rwhxm,memex-portal-deployment-75597864c8-wwfnf,memex-portal-deployment-6dd4fbd566-wv9p7,memex-portal-deployment-9bf79b774-tk88b,memex-portal-deployment-9bf79b774-jwrng,memex-portal-deployment-9bf79b774-nvnzz,memex-portal-deployment-9bf79b774-b8dl6,memex-portal-deployment-85859db964-q5zcm,memex-portal-deployment-85859db964-xk5gw,memex-portal-deployment-77c854cfcf-clz9l,memex-portal-deployment-77c854cfcf-ns2r5,memex-portal-deployment-77cfc55bfc-767mq,memex-portal-deployment-5b9cbf46dd-fxbhd,memex-portal-deployment-5b9cbf46dd-zn7nf,memex-portal-deployment-86dbb4d55c-jj7gc,memex-portal-deployment-c54d5847c-bf27p,memex-portal-deployment-c54d5847c-t7cm2,memex-portal-deployment-c54d5847c-m9crd,memex-portal-deployment-589ff8f895-c8qvj,memex-portal-deployment-589ff8f895-vw5jj,memex-portal-deployment-589ff8f895-ghgjt,memex-portal-deployment-56fbdcd48f-cbms9,memex-portal-deployment-75f7dd4cfb-mk985Recent log lines
Re-addressed by the current identity function: this incident inherited
d67b48fe47ffcb8d(Systemorph/MeshWeaver.Plugins#1796). Those nodes are superseded and will not fold, file or comment again.systemorph-com commentedon Oct 4, 2026
Left closed — this ticket was closed as NOT PLANNED, which is a decision rather than a claim that the fault stopped. The occurrences below are recorded here; if the decision should change, that is a person's call and not a recurrence's.
Still occurring: 1 more since the last update (2151 total).
memex-portal-deployment-5bb495d47b-8bdwp,memex-portal-deployment-78bb6bfd5-fpvxk,memex-portal-deployment-7d4b5684bb-9lrxl,memex-portal-deployment-764d5fbf-nn6nj,memex-portal-deployment-98785f547-zg442,memex-portal-deployment-78db689895-jpqs2,memex-portal-deployment-cb69ff6-vjbfr,memex-portal-deployment-6f68bcbc76-qx8sr,memex-portal-deployment-7bcb749b9f-7fqd5,memex-portal-deployment-7db656d9d9-nbfzk,memex-portal-deployment-6d58c699f8-9t675,memex-portal-deployment-76cbbdd5f-9mtkj,memex-portal-deployment-76cbbdd5f-82mvq,memex-portal-deployment-6f64bf4b56-2mzrf,memex-portal-deployment-6c46b84848-hnh9v,memex-portal-deployment-7d8fd6f84b-47cmq,memex-portal-deployment-7d8fd6f84b-mjp87,memex-portal-deployment-7d8fd6f84b-s7sr8,memex-portal-deployment-7d8fd6f84b-slmzp,memex-portal-deployment-856d4474c-hr5rd,memex-portal-deployment-66cfbf759-gmgl4,memex-portal-deployment-799c674567-p5lr8,memex-portal-deployment-7767bf7cd7-ffrhj,memex-portal-deployment-7767bf7cd7-rwhxm,memex-portal-deployment-75597864c8-wwfnf,memex-portal-deployment-6dd4fbd566-wv9p7,memex-portal-deployment-9bf79b774-tk88b,memex-portal-deployment-9bf79b774-jwrng,memex-portal-deployment-9bf79b774-nvnzz,memex-portal-deployment-9bf79b774-b8dl6,memex-portal-deployment-85859db964-q5zcm,memex-portal-deployment-85859db964-xk5gw,memex-portal-deployment-77c854cfcf-clz9l,memex-portal-deployment-77c854cfcf-ns2r5,memex-portal-deployment-77cfc55bfc-767mq,memex-portal-deployment-5b9cbf46dd-fxbhd,memex-portal-deployment-5b9cbf46dd-zn7nf,memex-portal-deployment-86dbb4d55c-jj7gc,memex-portal-deployment-c54d5847c-bf27p,memex-portal-deployment-c54d5847c-t7cm2,memex-portal-deployment-c54d5847c-m9crd,memex-portal-deployment-589ff8f895-c8qvj,memex-portal-deployment-589ff8f895-vw5jj,memex-portal-deployment-589ff8f895-ghgjt,memex-portal-deployment-56fbdcd48f-cbms9,memex-portal-deployment-75f7dd4cfb-mk985Recent log lines
Re-addressed by the current identity function: this incident inherited
d67b48fe47ffcb8d(Systemorph/MeshWeaver.Plugins#1796). Those nodes are superseded and will not fold, file or comment again.systemorph-com commentedon Oct 4, 2026
Left closed — this ticket was closed as NOT PLANNED, which is a decision rather than a claim that the fault stopped. The occurrences below are recorded here; if the decision should change, that is a person's call and not a recurrence's.
Still occurring: 2 more since the last update (2153 total).
memex-portal-deployment-5bb495d47b-8bdwp,memex-portal-deployment-78bb6bfd5-fpvxk,memex-portal-deployment-7d4b5684bb-9lrxl,memex-portal-deployment-764d5fbf-nn6nj,memex-portal-deployment-98785f547-zg442,memex-portal-deployment-78db689895-jpqs2,memex-portal-deployment-cb69ff6-vjbfr,memex-portal-deployment-6f68bcbc76-qx8sr,memex-portal-deployment-7bcb749b9f-7fqd5,memex-portal-deployment-7db656d9d9-nbfzk,memex-portal-deployment-6d58c699f8-9t675,memex-portal-deployment-76cbbdd5f-9mtkj,memex-portal-deployment-76cbbdd5f-82mvq,memex-portal-deployment-6f64bf4b56-2mzrf,memex-portal-deployment-6c46b84848-hnh9v,memex-portal-deployment-7d8fd6f84b-47cmq,memex-portal-deployment-7d8fd6f84b-mjp87,memex-portal-deployment-7d8fd6f84b-s7sr8,memex-portal-deployment-7d8fd6f84b-slmzp,memex-portal-deployment-856d4474c-hr5rd,memex-portal-deployment-66cfbf759-gmgl4,memex-portal-deployment-799c674567-p5lr8,memex-portal-deployment-7767bf7cd7-ffrhj,memex-portal-deployment-7767bf7cd7-rwhxm,memex-portal-deployment-75597864c8-wwfnf,memex-portal-deployment-6dd4fbd566-wv9p7,memex-portal-deployment-9bf79b774-tk88b,memex-portal-deployment-9bf79b774-jwrng,memex-portal-deployment-9bf79b774-nvnzz,memex-portal-deployment-9bf79b774-b8dl6,memex-portal-deployment-85859db964-q5zcm,memex-portal-deployment-85859db964-xk5gw,memex-portal-deployment-77c854cfcf-clz9l,memex-portal-deployment-77c854cfcf-ns2r5,memex-portal-deployment-77cfc55bfc-767mq,memex-portal-deployment-5b9cbf46dd-fxbhd,memex-portal-deployment-5b9cbf46dd-zn7nf,memex-portal-deployment-86dbb4d55c-jj7gc,memex-portal-deployment-c54d5847c-bf27p,memex-portal-deployment-c54d5847c-t7cm2,memex-portal-deployment-c54d5847c-m9crd,memex-portal-deployment-589ff8f895-c8qvj,memex-portal-deployment-589ff8f895-vw5jj,memex-portal-deployment-589ff8f895-ghgjt,memex-portal-deployment-56fbdcd48f-cbms9,memex-portal-deployment-75f7dd4cfb-mk985,memex-portal-deployment-75f7dd4cfb-99zs9,memex-portal-deployment-75f7dd4cfb-lp6fqRecent log lines
Re-addressed by the current identity function: this incident inherited
d67b48fe47ffcb8d(Systemorph/MeshWeaver.Plugins#1796). Those nodes are superseded and will not fold, file or comment again.