Skip to content

fix(device): expose Direct listener and TLS diagnostic stages - #342

Merged
abruption merged 3 commits into
mainfrom
fix/macos-direct-341
Oct 28, 2026
Merged

abruption merged 3 commits into
mainfrom
fix/macos-direct-341

Conversation

@abruption

@abruption abruption commented Oct 27, 2026 •

Copy link
Copy Markdown
Owner

Summary

A receiver can print ready:true while the operating system never delivers a Direct connection to its accept callback. During #341's two-Mac tests, the client connected and later returned peer_tls/timeout, but released v1.3.1 provided no Direct receiver-stage record to distinguish this from a handshake failure.

Add directReadyConfirmed:false to the startup JSON while preserving ready:true. Opt-in --diagnostic-events now records fixed-field Direct accept/TLS observations and one normal-shutdown counter snapshot, including zero counts. The failed candidate fixture emitted a zero-accept summary; successful pair/probe controls emitted 3 accepted and 3 completed TLS handshakes, with no native invocation.

Refs #341, which remains OPEN pending this diagnostic/documentation work. A later same-bind test on the original released runtime succeeded in both directions after the owner allowed Python.app's incoming connections. This supports pending Application Firewall consent as the blocker in this incident; the PR did not restore the network path.

Behavior and limits

  • Preserve binds, native/Relay transports, certificate/ALPN checks, handshake deadlines, pairing state and no-automatic-retry behavior. Existing WebSocket events/attempt IDs are unchanged.
  • Direct records use accepted and peer_tls phases with saturated per-receiver counters. Each phase permits at most one record per 60 monotonic seconds; TLS success/failure share that limit. A separate summary is attempted once at normal close. Counters may differ because of admission refusal, in-flight/cancelled work or saturation.
  • Exception classification and logging are best-effort. They cannot replace the original handshake exception; cancelled handshakes are not counted as failures. Output contains no addresses, peer/session IDs, message data or exception text. Existing synchronous stderr can still block; this is a record-count limit, not a hard write deadline.
  • Add four-language operator guidance and an experiment record/plain-TCP scaffold. experiments/ remains excluded from wheel/sdist. CI exercises the scaffold's stalled-client timeout handling with loopback only; manual self/server/tailnet experiments remain separate. Existing automated tests also use isolated loopback sockets.
  • The PR makes no runtime version bump, global install, automated network/firewall/ACL change or production service change. The owner explicitly approved the OS firewall prompt in the follow-up below. No merge or release is included.

Actual two-Mac validation

macOS 27.0.1 arm64 on both machines: mini Tailscale Network Extension 1.104.1, MacBook tailscaled 1.104.2. Baseline environments: Homebrew Python 3.14.9/OpenSSL 4.1.0, session-peer 1.3.1, cryptography 50.0.2, websockets 17.3.

  • Plain asyncio under Kqueue/Select and a raw threaded socket control reproduced the accept stall outside session-peer. Four released-CLI pairing cases (both directions, tailnet or explicitly approved 20-second wildcard listeners) timed out. Wildcard binding did not establish a workaround.
  • Released and candidate code both paired successfully from the mini to a MacBook python.org Python 3.13.7 receiver, followed by an authenticated probe.
  • Before the owner's firewall approval, the reverse control succeeded only through an explicitly selected 20-second private-address byte forwarder into the mini's loopback TLS receiver. The forwarder did not terminate TLS. This topology is distinct from a working native Homebrew tailnet listener.
  • Final candidate native mini failure: accepted/TLS-completed/TLS-failed counters 0/0/0 at normal close, client pending with retry forbidden. Both positive controls: 3/3/0, paired on both ends, authenticated probe successful, zero native operations. All test receivers stopped normally.
  • Interpreter distributions differ: python.org 3.13.7 and the system Python passed applicable TCP controls; uv 3.13.7/Homebrew 3.15.1 controls failed on the mini. Correcting an ad-hoc signature on a newly copied mini-only bundle did not fix its TCP control. Neither a minor-version cause nor a signature-only repair is established.

The committed experiment record distinguishes released/candidate runs and self/remote/negative controls. Personal addresses, private paths, device fingerprints, native IDs, invitations, keys and raw private logs are excluded. Existing user pairing state was not retried or reset; each live experiment used separate new test identities. No model/native-agent delivery was attempted.

Firewall-approval follow-up

After the owner reported allowing Python.app's incoming connections, both Macs passed the originally failing released-v1.3.1 Homebrew Python 3.14.9 direct pairing and authenticated probe on the same tailnet bind addresses and per-machine ports. Loaded app/CLI source hashes matched the published tag; no candidate source, python.org runtime, forwarding process or wildcard was used. Fresh test identities preserved the original user pending state, and native invocations were zero. This supports the pending Application Firewall permission as the practical cause in this tested incident, not a protocol fix supplied by this PR. The updated four-language guidance and chronological experiment record link the observed result with Apple's documented prompt behavior.

See the public before/after evidence. Historical pre-approval negative and forwarded controls are retained, not presented as current requirements. No general explanation for unrelated macOS/Tailscale failures or independent audit of each OS grant is claimed.

Checks and independent review

  • Final Python 3.13.7 Relay suite: 354 pass, development mode with ResourceWarning treated as error. Focused set: 85 pass; new diagnostic tests: 16 pass. Documentation: 65 pass. Generator, scaffold compilation and diff checks pass.
  • The original admission test asserted one total log before the new shutdown summary existed. Its single affected method now explicitly asserts exactly one admission record plus exactly one summary and retains the original counter/rate/redaction assertions; no production behavior was suppressed to satisfy it.
  • Before PR creation, Claude Code 2.1.310 (response-confirmed claude-opus-5-5) and Codex CLI 0.164.0 (isolated default; backend model identity unexposed) performed two parallel, noninteractive, read-only source review rounds without tools/tests/network/write authority. Both final reviews completed successfully without mandatory code corrections. Input SHA-256: first f3a0c5ec5f59c7fc9efe8a76bdd1e46e58cfa788f06eb05d9744eb25062d27fc; final 571735513c84a1e55642b2585c53f11001d1a2fa46ad714b0af28453e1eb584e. Hashes remained unchanged. The coordinator checked shutdown ordering and the JSON v1 additive-field contract against source and corrected evidence/version wording; those are coordinator checks, not reviewer-run tests.
  • Antigravity CLI 1.4.8 explicitly selected gemini-3.8-flash-medium, medium effort, fresh plan/sandbox structured translation QA without repository exposure or permission bypass. Two successful rounds: 163.626 and 227.679 seconds; input hashes 8afe3662a22cd2c4cc89b8f545da1ff9bd07354baa8a7d78547e9742d5c65fc1 and 0dc87366d127935f0c8eafee5b77247459f8208974e699ef3f63f2aae15b7e8b. Root verified/apply-reviewed the final narrow attempts/admission/TLS-termination/deadline terminology corrections and reran documentation checks; no third identical-final-wording model review is claimed. The selected model is recorded; backend identity is not independently exposed.

Current PR CI/automatic reviews are a separate gate. No authorization to merge or publish follows from these test/review results.

Automatic-review follow-up

The automatic code review found that Python 3.9/3.10 use a distinct asyncio.TimeoutError class in the experimental probe. A stalled-loopback regression reproduced the traceback on actual Python 3.9.6. Both client and writer-cleanup handlers now catch asyncio.TimeoutError; the regression passes on Python 3.9 and 3.13. This follow-up changes only the experimental scaffold and its test/documentation, not the tested production receiver. Original receiver source hashes and two-Mac outcomes remain valid. Current-head CI and renewed review are tracked after the follow-up push.

Follow-up translation and evidence checks

The owner-approval comparison was independently checked against both before/after result records by a read-only subagent. Loaded released-runtime source hashes were separately matched to the v1.3.1 tag on both Macs. The four-language firewall follow-up passed Antigravity CLI 1.4.8 / explicitly selected gemini-3.8-flash-medium / medium / fresh plan+sandbox structured QA in 112.967 seconds with no further corrections; frozen input SHA-256 5d97547a49249f5e8b6b56cb5938a3ec2411d101ed192850606655506016263e. No model tools, repository access, permission bypass, or backend-identity verification is claimed. Documentation tests (65), generator and diff checks pass. This follow-up changes documentation/manifest only; receiver source and earlier code-review results are unchanged.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-27T12:33:50.123661Z 6613f80 Manual request
🔒 Security Review ✅ Completed 2026-10-27T09:12:19.878305Z beb467b PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: beb467b8f2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@abruption

Copy link
Copy Markdown
Owner Author

@codex review

Please review current head 37e558bf8c268e218ff0898f821c3ccf98c57473. The pre-3.11 experimental-probe timeout finding is fixed and regression-tested on actual Python 3.9/3.13; its thread has been addressed. The follow-up only changes the scaffold, its loopback regression and documentation. Tested production receiver code and hardware outcomes are unchanged.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: 37e558bf8c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@abruption

Copy link
Copy Markdown
Owner Author

@codex security review

Please check current head 37e558bf8c268e218ff0898f821c3ccf98c57473 after the experimental probe compatibility fix. Receiver code is unchanged from the earlier reviewed head; the new regression uses only an owned short-lived loopback socket.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 👍

Reviewed commit: 37e558bf8c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@abruption

Copy link
Copy Markdown
Owner Author

Current-head verification

Head 37e558bf8c268e218ff0898f821c3ccf98c57473: all 28 checks pass, including CI run 40479370317. The latest Codex code re-review completed without findings (receipt); the original experimental-probe finding is fixed and its thread resolved. The initial security review completed without findings on beb467b; production receiver code is identical in the follow-up. A new security verdict for the follow-up head is not asserted.

The automatic-review timeout defect was reproduced on actual Python 3.9.6 before the fix, then passed as a stalled-loopback regression on Python 3.9 and 3.13. Final local production-code checks remain 354 Relay tests and 65 documentation tests, plus generator/diff/scaffold checks. The added core regression is separate from those counts.

All owned live-test servers exited normally. After retaining sanitized observations, the 11 owned fixture-state directories on each Mac (test keys, invitations and pending/paired test records), temporary python.org test environment and copied signature-control bundle were removed. The original stage5d identities/pending state, installed runtimes, service and network settings were preserved. The PR worktree remains clean for review.

The earlier #341 live evidence comment and current experiment record distinguish the failing native listener and successful controlled topologies. #341 stays open; no merge or publication has been performed.

@abruption abruption added this to the v1.3.2 milestone Oct 27, 2026
@abruption

Copy link
Copy Markdown
Owner Author

Firewall approval follow-up — 2026-10-27

The owner observed the macOS incoming-connection prompt for Python.app and reported allowing it. Before any new receiver-code installation, the same Homebrew Python 3.14.9 raw-TCP controls then passed on both Macs. Fresh pairing fixtures repeated the same tailnet bind addresses and per-machine ports as the earlier released-v1.3.1 failures.

Released-v1.3.1 receiver after approval Result
Mac mini Pairing exit 0; client/server paired; authenticated probe successful; receiver exited normally; native calls 0.
MacBook Pairing exit 0; client/server paired; authenticated probe successful; receiver exited normally; native calls 0.

Both tests used the existing Homebrew Python 3.14.9/OpenSSL 4.1.0 environments with session-peer 1.3.1, cryptography 50.0.2 and websockets 17.3. No python.org runtime, TCP forwarder, wildcard bind or candidate source override was used. Loaded app.py and cli.py SHA-256 values on both Macs matched the published v1.3.1 tag. These released-runtime trials have no new diagnostic counters; the candidate's earlier zero/3-3-0 summaries must not be substituted for them.

This before/after comparison supports pending macOS Application Firewall approval as the blocker in this tested incident. Apple's documentation explains that incoming attempts are denied until the alert is answered. The firewall stayed enabled. The owner reported approval; this report does not claim independent inspection of each OS approval entry or a universal explanation for other macOS/Tailscale failures.

PR #342 remains useful for truthful startup/readiness and accept/TLS diagnostics, but it did not restore these connections: the original released runtime worked after the owner's OS permission action. Earlier negative, signature and forwarded-topology results remain historical observations. No upstream Tailscale/CPython defect is established by this case.

Existing user pending state was not retried, reset or replaced. Each new experiment used disposable test identities, no native message was sent, and the receiver lifetime was 20 seconds. The updated four-language guidance and experiment record are being added to the existing PR. #341 remains open pending that diagnostic/documentation work; milestone v1.3.2 is unchanged. No merge or publication is authorized or performed by this follow-up.

@abruption

Copy link
Copy Markdown
Owner Author

@codex review

Please check documentation-only follow-up 6613f806acdaf53edad3f271ad777c94725e7bae. After the owner approved the actual macOS incoming-connection prompt, both Macs passed the same previously failing released-v1.3.1 Homebrew direct pairing/probe on the same bind addresses and ports, with fresh identities and no proxy/runtime/source override. The follow-up records this scoped before/after evidence in four languages and keeps the historical negative controls. It explicitly attributes recovery to the owner's OS permission, not the diagnostic PR. Production receiver source is unchanged. Gemini translation QA and a separate read-only artifact audit passed.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: 6613f806ac

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@abruption

Copy link
Copy Markdown
Owner Author

Firewall follow-up verified

Documentation-only head 6613f806acdaf53edad3f271ad777c94725e7bae passes all 28 checks, including CI run 40504142742. Its latest Codex code re-review completed without findings. Existing review threads remain resolved; production receiver source is unchanged from the earlier reviewed candidate.

The owner-approved Python.app firewall prompt was followed by successful two-way direct pairing and authenticated probes on the original released Homebrew runtime and the same bind addresses/ports, with no proxy or alternative interpreter. Both global firewalls remained enabled. This confirms recovery in the tested setup and supports pending app-level firewall approval as the incident's blocker; it is not evidence that the diagnostic PR repaired transport or that all macOS failures share this cause.

After preserving metadata, the two owned fixture states per Mac, their test keys/invitations and the remote temporary probe directory were removed. All 20-second test receivers exited normally with zero native invocations. Original user pending state and identities were not retried or reset; no operational mesh pairing was silently completed. The owner's OS permission is retained.

The issue and PR remain open in v1.3.2 for normal review/merge. No merge, release or global runtime installation was performed.

@abruption
abruption merged commit dac1c30 into main Oct 28, 2026
28 checks passed
@abruption
abruption deleted the fix/macos-direct-341 branch October 28, 2026 00:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant