[aw-failures] AWF firewall startup fails — getaddrinfo EAI_AGAIN awmg-cli-proxy, managed-gateway DNS not resolvable, agent never [Content truncated due to length]

[github-actions]

Problem statement

1. The AWF firewall fails to start because the in-network tcp-tunnel cannot resolve the managed-gateway sidecar hostname awmg-cli-proxy (getaddrinfo EAI_AGAIN — temporary DNS failure).
2. The awf-cli-proxy DIFC probe then fails fast after 10 attempts (~31s) and the run aborts at the Execute <engine> CLI step. The agent is never invoked, so 100% of the affected runs are lost.
3. Severity: P1 — recurred across ≥2 scheduled smoke cycles inside a ~33-minute window (12:40–13:13 UTC, 2026-06-25) and also caught a production workflow (Auto-Triage Issues). Untracked by any open agentic-workflows issue.

Affected workflows & runs (6h window)

Run	Workflow	Created (UTC)
§28172712935 (representative)	Auto-Triage Issues	2026-06-25T13:13
§28172037694	Smoke Copilot - AOAI (Entra)	2026-06-25T13:02
§28172034582	Smoke Copilot	2026-06-25T13:02
§28171956092	Smoke Antigravity	2026-06-25T13:01
§28171088767	Smoke Copilot - AOAI (Entra)	2026-06-25T12:46
§28171084471	Smoke Copilot	2026-06-25T12:46
§28170777281 (comparator, prior cycle)	Smoke Antigravity	2026-06-25T12:40

Evidence (representative §28172712935)

Firewall startup log tail

[tcp-tunnel] Forwarding localhost:18443 → awmg-cli-proxy:18443
[cli-proxy] DIFC proxy probe failed (attempt 1/10, diagnosis=not-yet-ready (ECONNREFUSED)), retrying in 1s...
[cli-proxy] DIFC proxy probe failed (attempt 2..5/10, diagnosis=unknown), retrying...
[tcp-tunnel] Upstream error (::1:48576): getaddrinfo EAI_AGAIN awmg-cli-proxy
[ERROR] Fatal error: Error: AWF firewall failed to start: awf-cli-proxy could not connect to the external DIFC proxy (or exited before establishing a connection). Failing fast to avoid repeated in-agent retries. The agent was never invoked.
Process exiting with code: 1

Probable root cause

1. DNS resolution of the managed-gateway sidecar awmg-cli-proxy returns EAI_AGAIN when the tcp-tunnel/cli-proxy begins probing — the awmg container/DNS record is not yet resolvable at that instant.
2. The fail-fast probe (10 attempts, ~31s) elapses before the name resolves, so the firewall aborts instead of riding out a transient managed-gateway startup race.
3. Recurrence across two consecutive smoke cycles points to a managed-gateway (awmg) provisioning / DNS-readiness ordering race, not a one-off blip.

Relationship to existing issues

• Related but distinct from AWF cli-proxy DIFC proxy probe failed (diagnosis=unknown) on GitHub Enterprise Cloud data residency (*.ghe.com) #41225 (DIFC proxy probe failed diagnosis=unknown on *.ghe.com). AWF cli-proxy DIFC proxy probe failed (diagnosis=unknown) on GitHub Enterprise Cloud data residency (*.ghe.com) #41225 is a data-residency tenant where the proxy is reachable but the forwarded gh api returns unexpected results. These failures occur on standard github.com runners and fail one step earlier: the upstream host awmg-cli-proxy is not resolvable at all (EAI_AGAIN). Same firewall-fails-to-start surface, different root cause.

Proposed remediation

1. Gate the DIFC probe on awmg-cli-proxy DNS resolvability — retry name resolution with backoff before counting probe attempts — so a transient EAI_AGAIN during managed-gateway startup is absorbed rather than fatal.
2. Surface EAI_AGAIN / DNS-resolution failures as a distinct probe diagnosis (e.g. dns-not-resolvable) instead of the generic unknown (also addresses AWF cli-proxy DIFC proxy probe failed (diagnosis=unknown) on GitHub Enterprise Cloud data residency (*.ghe.com) #41225 question 3).
3. Investigate the managed-gateway (awmg) container/DNS provisioning order so the sidecar hostname is registered before the cli-proxy begins probing.

Success criteria / verification

1. Scheduled smoke workflows (Smoke Copilot, Smoke Copilot - AOAI (Entra), Smoke Antigravity) conclude success across consecutive cycles with no AWF firewall failed to start errors.
2. A transient EAI_AGAIN on awmg-cli-proxy no longer aborts startup (absorbed by the DNS-readiness gate).
3. Probe logs distinguish DNS-resolution failure from connection-refused / unknown.

References: §28172712935 · §28172034582 · §28171956092

Related to #41225
Related to #39884

Generated by 🔍 [aw] Failure Investigator (6h) · 190.5 AIC · ⌖ 34.2 AIC · ⊞ 5.3K · ◷

• expires on Jul 2, 2026, 5:41 AM UTC-08:00

[pelikhan]

@copilot awf firewall issues should be detected and reflected in the agent failure issue.

[pelikhan]

Agent failure issue should include the a link to

https://github.com/github/gh-aw-firewall/blob/main/docs/diagnosing-awf-failures.md