[observability] Observability Coverage Report - 2026-08-09

[github-actions[bot]]

Executive Summary

A 30-run repository-wide fetch from the last 7 days produced an 18-run analysis set after enforcing the breadth cap and per-workflow limits. Within that sample, 15 runs showed firewall activity and 2 runs showed MCP session activity, but none of those runs contained the canonical raw observability files required for reliable post-incident debugging.

Recursive discovery under each selected run-<id>/ folder found no access.log files, no gateway.jsonl files, and no rpc-messages.jsonl files. Some runs still exposed derived summaries in summary.json and run_summary.json with firewall request counts and limited session counts, but those summaries are not a replacement for the raw artifacts needed to trace blocked egress decisions or individual MCP tool calls.

The most important operational consequence is simple: when a firewall-enabled run fails on network behavior, or when an MCP-enabled run fails inside tool orchestration, the downloaded artifacts currently do not preserve the raw evidence needed for root-cause analysis. This is a repo-wide observability gap, not an isolated workflow regression.

Key Alerts and Anomalies

🔴 Critical Issues:

• PR Sous Chef run §31338952148 had derived firewall activity (64 requests) but no discoverable access.log.
• Daily Project Performance Summary Generator (Using MCP Scripts) run §31335342952 had derived firewall activity (171 requests) but no discoverable access.log.
• Copilot PR Prompt Pattern Analysis run §31335227689 had derived firewall activity (40 requests) but no discoverable access.log.
• Cache directory setup run §31342423042 had derived firewall activity (72 requests) but no discoverable access.log.
• PR Sous Chef run §31342278171 had derived firewall activity (26 requests) but no discoverable access.log.
• Detection Analysis Report run §31341405042 had derived firewall activity (236 requests) but no discoverable access.log.
• 9 additional firewall-enabled runs in the analyzed set had the same missing-access.log condition.
• Daily Project Performance Summary Generator (Using MCP Scripts) run §31335342952 showed MCP session activity (45 tool calls) but neither gateway.jsonl nor rpc-messages.jsonl was present.
• Copilot PR Prompt Pattern Analysis run §31335227689 showed MCP session activity (10 tool calls) but neither gateway.jsonl nor rpc-messages.jsonl was present.

⚠️ Warnings:

• aw_info.json was absent from all 18 analyzed run folders, so firewall and MCP configuration could not be confirmed from the canonical metadata file.
• safe_output.jsonl was absent from all 18 analyzed run folders; only high-level safe-output summaries were available when present.

Coverage Summary

Component	Runs Analyzed	Logs Present	Coverage	Status
AWF Firewall (access.log)	15	0	0.0%	🔴
MCP Gateway (gateway.jsonl or rpc-messages.jsonl)	2	0	0.0%	🔴

📋 Detailed Run Analysis

Firewall-Enabled Runs

Workflow	Run ID	access.log	Entries	Allowed	Blocked	Status
PR Sous Chef	§31338952148	❌	0 raw	64 derived	0 derived	🔴
Daily Project Performance Summary Generator (Using MCP Scripts)	§31335342952	❌	0 raw	171 derived	0 derived	🔴
Copilot PR Prompt Pattern Analysis	§31335227689	❌	0 raw	40 derived	0 derived	🔴
Cache directory setup	§31342423042	❌	0 raw	72 derived	0 derived	🔴
PR Sous Chef	§31342278171	❌	0 raw	26 derived	0 derived	🔴
Detection Analysis Report	§31341405042	❌	0 raw	236 derived	0 derived	🔴
ESLint Monster	§31338309090	❌	0 raw	14 derived	0 derived	🔴
Duplicate Code Detector	§31338127189	❌	0 raw	120 derived	0 derived	🔴
Daily Regulatory Report Generator	§31336433499	❌	0 raw	80 derived	0 derived	🔴
Agentic Workflow Audit Agent	§31336387687	❌	0 raw	291 derived	0 derived	🔴
Daily Safe Output Tool Optimizer	§31335801203	❌	0 raw	226 derived	0 derived	🔴
Contribution Check	§31335675356	❌	0 raw	223 derived	0 derived	🔴
Lockfile Statistics Analysis Agent	§31333834644	❌	0 raw	87 derived	0 derived	🔴
Daily Team Evolution Insights	§31333747111	❌	0 raw	58 derived	0 derived	🔴
PR Sous Chef	§31341574451	❌	0 raw	54 derived	0 derived	🔴

Missing Firewall Logs (access.log)

Workflow	Run ID	Date	Link
PR Sous Chef	31338952148	2026-08-09	§31338952148
Daily Project Performance Summary Generator (Using MCP Scripts)	31335342952	2026-08-09	§31335342952
Copilot PR Prompt Pattern Analysis	31335227689	2026-08-09	§31335227689
Cache directory setup	31342423042	2026-08-09	§31342423042
PR Sous Chef	31342278171	2026-08-09	§31342278171
Detection Analysis Report	31341405042	2026-08-09	§31341405042
ESLint Monster	31338309090	2026-08-09	§31338309090
Duplicate Code Detector	31338127189	2026-08-09	§31338127189
Daily Regulatory Report Generator	31336433499	2026-08-09	§31336433499
Agentic Workflow Audit Agent	31336387687	2026-08-09	§31336387687
Daily Safe Output Tool Optimizer	31335801203	2026-08-09	§31335801203
Contribution Check	31335675356	2026-08-09	§31335675356
Lockfile Statistics Analysis Agent	31333834644	2026-08-09	§31333834644
Daily Team Evolution Insights	31333747111	2026-08-09	§31333747111
PR Sous Chef	31341574451	2026-08-09	§31341574451

MCP-Enabled Runs

Workflow	Run ID	Telemetry Source	Entries	Servers	Tool Calls	Errors	Status
Daily Project Performance Summary Generator (Using MCP Scripts)	§31335342952	❌ None	0 raw	0	45 derived	0	🔴
Copilot PR Prompt Pattern Analysis	§31335227689	❌ None	0 raw	0	10 derived	0	🔴

Missing MCP Telemetry (no gateway.jsonl or rpc-messages.jsonl)

Workflow	Run ID	Date	Link
Daily Project Performance Summary Generator (Using MCP Scripts)	31335342952	2026-08-09	§31335342952
Copilot PR Prompt Pattern Analysis	31335227689	2026-08-09	§31335227689

🔍 Telemetry Quality Analysis

Firewall Log Quality

• Total access.log entries analyzed: 0 raw entries across 0 files
• Derived firewall requests observed in summaries: 1762
• Blocked requests observed in summaries: 0 (0.0%)
• Domains observed in summaries: 16 unique
• Most accessed derived domains: 151.101.0.223:443, 151.101.128.223:443, 151.101.192.223:443, 151.101.64.223:443, 167.82.48.223:443

Gateway Log Quality

• Telemetry source: none of the MCP-enabled runs exposed gateway.jsonl or rpc-messages.jsonl
• Total raw MCP entries analyzed: 0
• Derived MCP session events from summaries: 169
• Derived tool calls from summaries: 55
• Derived failed tool calls from summaries: 0
• Average response time: N/A because no raw gateway or RPC telemetry files were present

Additional Telemetry Artifacts

• summary.json / run_summary.json: present in all 18 analyzed runs
• agent-stdio.log: present in 15/18 analyzed runs
• aw_info.json: present in 0/18 analyzed runs
• safe_output.jsonl: present in 0/18 analyzed runs

Healthy Runs Summary

No analyzed run had complete raw observability coverage for the required firewall or MCP log artifacts.

Recommended Actions

1. Restore publication of raw Squid access.log artifacts for every firewall-enabled run, including workflow-call layouts, and validate the path in CI with a post-run assertion.
2. Restore publication of raw MCP telemetry (gateway.jsonl preferred, rpc-messages.jsonl acceptable fallback) for every run with MCP session activity, and fail the workflow when both files are absent.
3. Include aw_info.json and safe_output.jsonl in the downloaded artifacts so configuration and actuation can be audited without reconstructing behavior from summaries.

📊 Historical Trends

No prior retained observability baseline was available in the downloaded artifacts, so this report only establishes the current daily snapshot.

---

Report generated automatically by the Daily Observability Report workflow
Analysis window: Last 7 days | Runs analyzed: 18

Generated by 📊 Daily Observability Report for AWF Firewall and MCP Gateway · gpt54 · 81.8 AIC · ⌖ 10.2 AIC · ⊞ 14K · ◷

• expires on Aug 10, 2026, 4:01 PM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-08-11T00:01:14.670Z.

Closed by Workflow