[observability] Observability Coverage Report - 2026-08-30

[github-actions[bot]]

Executive Summary

A 30-run broad fetch for the last 7 days hit the configured cap within roughly 100 minutes of recent activity, so the detailed audit was scoped to a breadth-first 9-run sample across 7 workflows. Within that audited sample, observability file coverage was complete: all 9 firewall-enabled runs exposed access.log, and all 9 MCP-enabled runs exposed telemetry via rpc-messages.jsonl.

Quality was stronger on MCP than on firewall validation depth. MCP telemetry was structurally healthy in all 9 analyzed runs with 76 valid JSONL entries, 27 tool calls, and no recorded MCP errors. Firewall logs were present in all runs, but only 5 of 9 showed both allowed and denied traffic; the remaining 4 runs logged allowed traffic only, which is a warning under the stricter coverage rubric.

The main limitation is failure-path visibility outside the audited sample: the broad fetch also returned 14 failed runs whose local download contained no runtime artifacts (awf-config.json, firewall logs, or MCP logs), so those failures could not be assessed for runtime observability from the fetched data alone.

Key Alerts and Anomalies

Note

No critical issues detected in the audited sample. No firewall-enabled run was missing access.log, and no MCP-enabled run was missing both gateway.jsonl and rpc-messages.jsonl.

Warning

MCP coverage relied entirely on the canonical fallback rpc-messages.jsonl. No gateway.jsonl files were discovered in the 9 analyzed runs, so per-call duration metrics were not available from the preferred structured gateway format.

Warning

Firewall quality warnings were recorded for 4 runs because denied traffic was not observed in access.log: §33342414332, §33341923603, §33338944238, §33338944251. File coverage is complete, but allow/deny-path validation is not consistent run to run.

Warning

The broad fetch also contained 14 failed runs with no runtime artifacts in the local download, including §33340027432 and §33339339563. These were excluded from coverage denominators because firewall/MCP applicability could not be established from the fetched artifacts.

Coverage Summary

Component	Runs Analyzed	Logs Present	Coverage	Status
AWF Firewall (access.log)	9	9	100.0%	⚠️
MCP Gateway (gateway.jsonl or rpc-messages.jsonl)	9	9	100.0%	✅

📋 Detailed Run Analysis

Firewall-Enabled Runs

Discovered access-log paths before classification: all audited firewall runs resolved access.log at run-<id>/sandbox/firewall/logs/access.log.

Workflow	Run ID	access.log	Entries	Allowed	Blocked	Status
Daily Documentation Healer (in_progress)	§33342414332	✅	115	59	0	⚠️
Cache directory setup	§33341923603	✅	199	104	0	⚠️
PR Sous Chef	§33341788156	✅	71	40	1	✅
PR Sous Chef	§33341410609	✅	53	30	1	✅
Issue Monster	§33341377404	✅	50	20	1	✅
Issue Monster	§33339708691	✅	50	20	1	✅
Matt Pocock Skills Reviewer	§33338944238	✅	118	48	0	⚠️
Ponytail Reviewer	§33338944302	✅	94	40	2	✅
Impeccable Skills Reviewer	§33338944251	✅	114	50	0	⚠️

Missing Firewall Logs (access.log)

None in the audited sample.

MCP-Enabled Runs

Discovered MCP telemetry paths before classification: all audited MCP runs resolved telemetry at run-<id>/mcp-logs/rpc-messages.jsonl; no gateway.jsonl files were found in the selected sample.

Workflow	Run ID	Telemetry Source	Entries	Servers	Tool Calls	Errors	Status
Daily Documentation Healer (in_progress)	§33342414332	rpc-messages.jsonl	8	1	3	0	✅
Cache directory setup	§33341923603	rpc-messages.jsonl	8	2	1	0	✅
PR Sous Chef	§33341788156	rpc-messages.jsonl	6	1	2	0	✅
PR Sous Chef	§33341410609	rpc-messages.jsonl	4	1	1	0	✅
Issue Monster	§33341377404	rpc-messages.jsonl	14	1	6	0	✅
Issue Monster	§33339708691	rpc-messages.jsonl	14	1	6	0	✅
Matt Pocock Skills Reviewer	§33338944238	rpc-messages.jsonl	10	1	4	0	✅
Ponytail Reviewer	§33338944302	rpc-messages.jsonl	6	1	2	0	✅
Impeccable Skills Reviewer	§33338944251	rpc-messages.jsonl	6	1	2	0	✅

Missing MCP Telemetry (no gateway.jsonl or rpc-messages.jsonl)

None in the audited sample.

🔍 Telemetry Quality Analysis

Firewall Log Quality

• Total access.log entries analyzed: 864
• Domains accessed: 10 unique
• Blocked requests: 6 (0.7%)
• Most accessed domains: api.githubcopilot.com, o205451.ingest.us.sentry.io, otlp-gateway-prod-eu-west-2.grafana.net

Gateway Log Quality

• Telemetry source: rpc-messages.jsonl in all 9 analyzed MCP runs; gateway.jsonl was not present in the sample
• Total entries analyzed: 76
• MCP servers used: github, safeoutputs
• Total tool calls: 27
• Error rate: 0.0%
• Average response time: N/A (rpc-messages.jsonl did not provide direct duration fields in this sample)

Additional Telemetry

• agent-stdio.log present in all 9 audited runs
• safeoutputs.jsonl present in all 9 audited runs
• awf-config.json present in all 9 audited runs; aw_info.json was not present in this sample
• agent_output.json was present in all 9 audited runs; an agent_output/ directory was not present in this sample

Healthy Runs Summary

5 runs met the stricter healthy threshold for both firewall and MCP telemetry: §33341788156, §33341410609, §33341377404, §33339708691, §33338944302.

Recommended Actions

1. Emit gateway.jsonl alongside rpc-messages.jsonl so MCP investigations can use direct duration and status metrics instead of the lower-fidelity RPC fallback alone.
2. Add a lightweight denied-egress smoke path, or equivalent synthetic validation, so firewall coverage consistently proves both allow and deny behavior instead of only allowed traffic.
3. Investigate why 14 failed runs in the fetched set had no runtime artifacts in the local download, and ensure firewall and MCP artifacts are uploaded on failure whenever the agent runtime starts.

📊 Historical Trends

No historical baseline was computed in this run. The initial repository-wide fetch hit the 30-run cap within a narrow recent slice, so this report is a current-state coverage audit rather than a week-over-week trend view.

References: §33342414332, §33341788156, §33340027432

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• ab.chatgpt.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "ab.chatgpt.com"

See Network Configuration for more information.

Generated by 📊 Daily Observability Report for AWF Firewall and MCP Gateway · codex · gpt54 · 82.4 AIC · ⌖ 10.1 AIC · ⊞ 14.4K · ◷

• expires on Aug 31, 2026, 3:52 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Daily Observability Report for AWF Firewall and MCP Gateway.

A newer discussion is available at Discussion #57525.

[github-actions[bot]]

Me caveman speak: smoke test done! Great success! 🚀

Warning

Firewall blocked 7 domains

The following domains were blocked by the firewall during workflow execution:

• accounts.google.com
• android.clients.google.com
• clients2.google.com
• contentautofill.googleapis.com
• dc.services.visualstudio.com
• www.google.com
• www.gstatic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "dc.services.visualstudio.com"
    - "www.google.com"
    - "www.gstatic.com"

See Network Configuration for more information.

📰 BREAKING: Report filed by Smoke Copilot - AOAI (Entra) · copilot · o40mini · 42.9 AIC · ⌖ 4.29 AIC · ⊞ 19.4K · ◷
_{Comment /smoke-copilot-aoai-entra to run again
Add label smoke to run again}

[github-actions[bot]]

👋 Beep boop! The ARM64 smoke-test agent zoomed through here on its tiny aarch64 legs, checked all the observability boxes, and left a shiny virtual sticker behind. Multi-arch supremacy confirmed! 🦾🤖

📰 BREAKING: Report filed by Smoke Copilot ARM64 · copilot · auto · 51.6 AIC · ⌖ 3.23 AIC · ⊞ 9.1K · ◷
_{Comment /smoke-copilot-arm to run again}