Skip to content

The log-incident issue commenter cannot post ('Resource not accessible by integration') — an incident that keeps firing goes silently quiet on its own thread #4022

Closed

Description

@rbuergi

Summary

The log-incident → GitHub-issue commenter cannot post, and the failure is recorded only inside
the incident node. An incident that keeps firing therefore stops updating its issue silently —
the thread simply goes quiet, which reads exactly like "it stopped happening".

Measured

Admin/_LogIncident/af1ee515fdf60bd1 on memex.systemorph.com, read 2026-09-11:

occurrences:              7
occurrencesAtLastComment: 5
status:                   Failed
error:                    "Resource not accessible by integration"
issueUrl:                 https://github.com/Systemorph/MeshWeaver/issues/3876
lastModified:             2026-09-10T02:36:05Z

Two occurrences never reached the issue. The scanner itself is healthy — it is updating
occurrences/lastSeen and other incidents were written at 06:30:14Z today — so this is the
commenting leg alone.

Why the failure mode is worse than the missing comments

  1. It is silent by construction. status/error live on the incident node, which nobody reads
    unless they already suspect something. The issue thread shows no error, no gap marker, nothing.
  2. A quiet thread is evidence of the wrong thing. Prebuilt bundle/module routes must answer an absent _complete seal with 503/404, not an unhandled FileNotFoundException #3876 was closed today partly on "no
    occurrence in ~31 h"
    . That reading happened to be sound — the count and lastSeen come from the
    scanner, not the commenter — but a reader who conflated "the thread is quiet" with "the incident
    is quiet" would have drawn the same conclusion from a tracker that was simply mute. The two are
    one careless sentence apart.
  3. It fails toward "fine". Like every other instrument that bit this fleet today — a Not found
    that means denied, a sinceMinutes that means requested, a zero-step job that means
    refused — the broken state is indistinguishable from the healthy one at a glance.

Probable cause

Resource not accessible by integration is GitHub's refusal when an App token lacks a permission
for the resource. The App used here would need issues: write on Systemorph/MeshWeaver.
Worth checking against the known constraint that the org's App has historically carried
contents:write, emails:write, metadata:read, pull_requests:write — issues absent — which is
the same class of gap that blocks the workflow-dispatch and pin-bump lanes
(Memex#247, and the App-permission half of the pin-bump story).

🚨 Verify before acting: read the App's actual permission set rather than inheriting that list,
and never print a token value — names, lengths and verdicts only.

What a fix needs

  1. Grant the permission (or use a credential that has it), then confirm by a real post, not by
    the absence of an error.
  2. 🚨 Make the failure loud. A commenter that cannot post must surface somewhere a person looks —
    the platform bell, a health entry, or the Fleet board — not only in a field on the node. Whatever
    is chosen needs a positive signal: the ConfigsTargeting zero-match Warning is the precedent
    this fleet already uses for "I could not do the thing you think I did".
  3. Control in both directions: a refused post surfaces and names the incident; a successful post
    advances occurrencesAtLastComment and surfaces nothing. Without the second, "everything is
    broken" would pass the first.
  4. Consider whether occurrencesAtLastComment lagging occurrences by more than a threshold is
    itself the cheap detector — it is already stored, already truthful, and needs no new plumbing.

Not a duplicate

No existing issue covers the commenter leg. Memex#247 is the workflow-dispatch refusal by the same
App class and is likely the same root grant, but it is a different surface and a different remedy.

🤖 Generated with Claude Code

Activity

  1. added a commit that references this issue on Sep 11, 2026
  2. rbuergi commented on Sep 11, 2026

    @rbuergi
    ContributorAuthor

    Measured cause, and what now makes it loud

    Cause: the App has no issues permission, so issues:write is the gap. I read App meshweaver-cloud (id 4220566) with an App JWT, printing names only, never the key or a token:

    • App permissions: contents:write, emails:write, metadata:read, pull_requests:write
    • Installation 144517285 (Systemorph, all repos): contents:write, metadata:read, pull_requests:write

    It is not one incident. The App opened 261 issues here (newest #3986) and commented on 0 issues ever; its 39 comments are all on PRs. A direct read of #1172, #1198 and #1127 shows only human comments. So every recurrence comment since the watcher's first run on 2026-08-10 was refused. Issue creation still works, which is exactly why the pipeline looked healthy.

    The grant is an org-owner action. I cannot do it.

    1. github.com/organizations/Systemorph/settings/apps/meshweaver-cloud/permissions → Repository permissions → Issues: Read and write → Save changes.
    2. github.com/organizations/Systemorph/settings/installations/144517285 → Review request → Accept new permissions.
    3. Verify: an App-JWT GET /app/installations lists "issues": "write". Then the next recurrence comment lands and advances that incident's occurrencesAtLastComment. (Prebuilt bundle/module routes must answer an absent _complete seal with 503/404, not an unhandled FileNotFoundException #3876 is closed now, so its next comment would REOPEN it.)

    Loud, once. A refused File/Comment now rings the platform bell (Admin/_Notification). It names the incident, the issue, and how many occurrences never reached it, and logs one [IssueRefused] Warning. It rings once per refusal episode, tracked by a new RefusalAnnouncedAt field that a landed post clears. Both directions are pinned with exact-text controls, and each was falsified by mutation.

    Is the occurrencesAtLastComment lag enough alone? No. It is normal for up to CommentInterval (6 h). Its age is not stored (LastCommentedAt is null after filing). It is blind to the File leg. Suppressed incidents lag by design. And it is passive. It is the number the bell quotes, not the alarm.

    Write-up: Red-log watching and automatic ticketing → "A refused update is loud — once".
    PRs: #4033 (merged at 14:55Z: catalog keys and doc) · Systemorph/MeshWeaver.Plugins#1666 (the bell and its controls, in CI). Leaving this issue open until the grant lands and a real comment is observed.

    🤖 Generated with Claude Code

  3. rbuergi commented on Sep 13, 2026

    @rbuergi
    ContributorAuthor

    Closing — the commenter posts, and a refusal is loud

    Adjudicated 2026-09-13. Both halves of "what a fix needs" are met and each was confirmed by a positive signal, not by the absence of an error.

    1. The commenter reaches its issues (confirmed by real posts)

    The control instance (memex.systemorph.com) now comments as systemorph-com[bot] (App id 4918443, installation 161076693, issues: write — names and ids only). Via REST (issues/comments?since=2026-09-11, 400 comments paged newest-first, oldest 2026-09-11T20:49Z), that App has posted 14 issue comments between 2026-09-12T08:01:08Z and 2026-09-13T03:43:08Z — recurrence updates on #3883, #3545 (×6), #2157, #3659, #1134, #1246, #2681 and #1140. Before 2026-09-12 it had posted 0 issue comments in that window; meshweaver-cloud[bot] has still posted 0 comments on issues (PRs only), which is consistent with the comment above.

    Read on the control instance (systemorph MCP server, get @Admin/_LogIncident/log-burst-header-only-memex-cloud/data/):

    status:                   Filed
    occurrences:              887
    occurrencesAtLastComment: 887
    lastCommentedAt:          2026-09-12T21:13:18Z
    error:                    (absent)
    

    — and the matching comment landed on #2681 at 2026-09-12T21:14:09Z. That is the "control in both directions" the issue asked for: a successful post advances occurrencesAtLastComment and surfaces nothing.

    The meshweaver-cloud installation (144517285) also reads issues: write now, so the App the incident originally ran under is no longer short of the grant either.

    2. A refused post is loud

    Residual, not a reason to keep this open

    Admin/_LogIncident/af1ee515fdf60bd1 (the node this issue measured, → #3876, closed) still carries status: Failed / error: Resource not accessible by integration. It has not fired since 2026-09-10T02:35Z (lastSeen), so the commenter never had a reason to retry it — that field is a stale record of the last attempt, not a live failure. It will clear itself on the next occurrence, or stay as history if #3876 never recurs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions