Skip to content

[Bug]: T3 Connect runs any cloudflared on PATH without a version check; Homebrew 2023.8.2 crash-loops on --output #13964

Closed
#17275
Closed
[Bug]: T3 Connect runs any cloudflared on PATH without a version check; Homebrew 2023.8.2 crash-loops on --output#13964
#17275
Labels
bugSomething is broken or behaving incorrectly.via-triageFiled through npx t3 triage

Description

@JorgeMenaDev
JorgeMenaDev
opened on Sep 27, 2026

Before submitting

  • I searched existing issues and did not find a duplicate.
  • I included enough detail to reproduce or investigate the problem.

Area

packages/contracts or packages/shared

Steps to reproduce

  1. On macOS arm64, have an old Homebrew cloudflared on PATH. Mine was 2023.8.2, installed in 2023 and never upgraded.
  2. Make sure T3's managed copy isn't installed yet (~/.t3/tools/cloudflared/2026.5.2/darwin-arm64/cloudflared absent).
  3. Enable T3 Connect on that Mac from the desktop app.
  4. Open the environment from the iOS app.

Expected behavior

T3 runs a cloudflared that supports the flags it passes, either its pinned managed copy or a PATH binary it has checked. If the tunnel can't stay up, the host and the phone say so.

Actual behavior

The phone shows Failed to connect. Reconnecting... Reason: Relay could not reach the environment endpoint (endpoint_request_failed) forever. My other Mac, on the same account, connects fine.

The relay error hides the real problem. RelayClient.resolve checks the override env var, then the managed path, then any cloudflared on PATH. It doesn't check the PATH binary's version, and it reports version: CLOUDFLARED_VERSION for it anyway:

t3code/packages/shared/src/relayClient.ts

Lines 224 to 262 in de251fc

const resolve: RelayClientShape["resolve"] = Effect.gen(function* () {
const config = yield* loadCloudflaredConfig;
if (Option.isSome(config.executableOverride)) {
return (yield* isExecutableFile(config.executableOverride.value))
? {
status: "available",
executablePath: config.executableOverride.value,
source: "override",
version: CLOUDFLARED_VERSION,
}
: { status: "missing", version: CLOUDFLARED_VERSION };
}
if (yield* isExecutableFile(managedPath)) {
return {
status: "available",
executablePath: managedPath,
source: "managed",
version: CLOUDFLARED_VERSION,
};
}
const pathExecutable = yield* resolvePathExecutable;
if (pathExecutable) {
return {
status: "available",
executablePath: pathExecutable,
source: "path",
version: CLOUDFLARED_VERSION,
};
}
return releaseAsset
? { status: "missing", version: CLOUDFLARED_VERSION }
: {
status: "unsupported",
platform,
arch,
version: CLOUDFLARED_VERSION,
};
});

Because a PATH binary counts as available, the managed copy never gets installed. The server then spawns it with --output default:

t3code/apps/server/src/cloud/ManagedEndpointRuntime.ts

Line 315 in de251fc

["tunnel", "--no-autoupdate", "--loglevel", "info", "--output", "default", "run"],

cloudflared 2023.8.2 doesn't have that flag, so it exits right away:

$ TUNNEL_TOKEN=x /opt/homebrew/bin/cloudflared tunnel --no-autoupdate --loglevel info --output default run
Incorrect Usage. flag provided but not defined: -output

The supervisor restarts it with backoff (1, 2, 4, 8, 16, 32 s, then every 60 s) and nothing registers. recoverManagedCloudTunnel gets Relay internal error: upstream_unavailable every 2 minutes. None of this reaches the UI. I found it by reading server.trace.ndjson and running the binary by hand.

Impact

Blocks work completely

Version or commit

T3 Code (Nightly) 0.0.43-nightly.20260926.2282; code refs at main @ de251fc

Environment

macOS 27.0 (Darwin 27.0.0) arm64, MacBook Pro. Homebrew cloudflared 2023.8.2 on PATH. T3 Connect iOS app.

Logs or stack traces

# server.trace.ndjson, "Relay client process started; waiting for tunnel connection" (same tunnelId each time)
16:26:03 pid 3103
16:26:03 pid 3538
16:26:04 pid 4264
16:26:06 pid 4910
16:26:10 pid 5205
16:26:18 pid 6151
16:26:34 pid 7166
16:27:06 pid 9145
16:28:06 pid 13702

# environment.cloud.recoverManagedCloudTunnel
16:28:03 Failure  EnvironmentHttpInternalServerError: T3 Connect: Relay internal error: upstream_unavailable.
16:30:03 Failure  EnvironmentHttpInternalServerError: T3 Connect: Relay internal error: upstream_unavailable.
16:32:03 Success  (after installing the managed 2026.5.2 copy, see workaround)

Relay trace ID from the phone: 00dd36630a221ea911bd41bf682473b5.

Workaround

Put T3's pinned binary at the managed path. resolve checks it before PATH, and the server picks it up on its next restart attempt, with no app restart:

DEST=~/.t3/tools/cloudflared/2026.5.2/darwin-arm64
curl -fsSL -o /tmp/cf.tgz https://github.com/cloudflare/cloudflared/releases/download/2026.5.2/cloudflared-darwin-arm64.tgz
shasum -a 256 /tmp/cf.tgz   # ba94054c9fd4297645093d59d51442e5e546d07bb0516120e694a13d5b216d38
tar -xzf /tmp/cf.tgz -C /tmp && mkdir -p "$DEST" && install -m 755 /tmp/cloudflared "$DEST/cloudflared"

brew upgrade cloudflared also works. It breaks again whenever T3 bumps CLOUDFLARED_VERSION, though, because the managed path changes and T3 falls back to PATH.

Possible fixes

  1. Prefer the managed copy. Install it when missing, even if PATH has a cloudflared, and keep PATH only as a fallback.
  2. Or run cloudflared --version on a PATH candidate and skip it below a minimum version. Report the real version, not CLOUDFLARED_VERSION.
  3. When the relay client crash-loops, show that in the connect status. The phone's endpoint_request_failed doesn't point to the host.

Related but different causes: #7447 (tunnel counted as running while unreachable), #8844 (same phone error on Windows, maybe the same PATH issue).

Activity

  1. juliusmarminge commented on Sep 27, 2026

    @juliusmarminge
    juliusmarminge
    on Sep 27, 2026
    Member

    Triage

    Confirmed on current main. This is a real bug, and it is not a duplicate of #7447 or #8844.

    RelayClient.resolve treats any executable named cloudflared on PATH as available, and it labels that binary version: CLOUDFLARED_VERSION (2026.5.2) without running it. The desktop link flow and t3 connect both skip the managed install when status is available, and install returns early in that case. The connector then spawns whatever was resolved:

    tunnel --no-autoupdate --loglevel info --output default run

    --output exists in cloudflared 2025.6.1 and later. Homebrew 2023.8.2 exits immediately with flag provided but not defined: -output. The supervisor backoff matches the pid cadence in the report: immediate, then 1s, 2s, 4s, 8s, 16s, 32s, then 60s. Placing the pinned binary at ~/.t3/tools/cloudflared/2026.5.2/<platform>/cloudflared works because the managed path is checked before PATH, and the next reconcile picks it up without an app restart. brew upgrade cloudflared also works until the next CLOUDFLARED_VERSION bump, which changes the managed directory and falls through to PATH again.

    --output default was added in #9386 so the connector keeps seeing classic text (Registered tunnel connection, ERR/WRN/FTL/PNC, and the tunnel-rejection lines). That flag is right for the pinned binary. It is not safe to pass to an arbitrary PATH binary. The same hole exists for T3CODE_CLOUDFLARED_PATH: an override is accepted with no version check and is also reported as 2026.5.2.

    The phone's endpoint_request_failed is the relay failing to mint against a tunnel that never registered. The host reports the connector as running as soon as spawn succeeds, and Incorrect Usage is logged at debug because it has no ERR/WRN/FTL/PNC token. That status gap is the same class of problem as #7447, not this bug. #8844 is the same phone error on Windows, with no confirmed PATH binary. upstream_unavailable from recoverManagedCloudTunnel is the relay's provisioning error, throttled to once every two minutes. It follows from the connector never staying up.

    Fix: a PATH binary must not count as available while the managed copy is missing. Install the pinned build, and keep PATH only as a fallback when this platform has no managed asset. If a PATH or T3CODE_CLOUDFLARED_PATH binary is still used, run cloudflared version (not --version; the pinned Windows build rejects --version) and skip anything older than 2025.6.1. Report that version, not CLOUDFLARED_VERSION.

    The workaround in the report is the right one.

  2. juliusmarminge
    added
    bugSomething is broken or behaving incorrectly.
    via-triageFiled through npx t3 triage
    on Sep 27, 2026

  3. Gongzhui commented on Oct 4, 2026

    @Gongzhui
    Gongzhui
    on Oct 4, 2026

    Confirmed on T3 Code (Nightly) 0.0.46-nightly.20261004.2648, macOS arm64 — same crash loop with Homebrew cloudflared 2025.5.0 on PATH. Symptoms identical: spawned with --output default, exits on flag provided but not defined: -output, restarts every ~60s with zero output logged, iOS app shows Relay could not reach the environment endpoint while the Connections UI reports the environment as available.

    Workaround verified: manually downloading the pinned 2026.5.2 darwin-arm64 release (sha256 ba94054c9fd4297645093d59d51442e5e546d07bb0516120e694a13d5b216d38) into ~/.t3/tools/cloudflared/2026.5.2/darwin-arm64/cloudflared makes resolution prefer the managed binary; tunnel registered within seconds and the phone connects.

    One more observation supporting a fix: since the PATH fallback reports version: CLOUDFLARED_VERSION without running it, T3 never even attempts to download the managed copy — the presence of any cloudflared on PATH permanently suppresses installation of a working one.

  4. notgiorgi commented on Oct 6, 2026

    @notgiorgi
    notgiorgi
    on Oct 6, 2026

    Confirmed on stable 0.0.45 (desktop, macOS arm64) with Homebrew cloudflared 2025.2.0 on PATH. Same crash loop: flag provided but not defined: -output, respawn at 1s → 60s backoff, never registers; other devices show endpoint_request_failed.

    Workaround: brew upgrade cloudflared (2026.10.0 works). The server picked up the new binary on the next retry, no restart needed.

  5. moosashah commented on Oct 8, 2026

    @moosashah
    moosashah
    on Oct 8, 2026

    Thank you @notgiorgi and @JorgeMenaDev! This identified the problem on my Mac too: stable desktop 0.0.45 on macOS arm64, with Homebrew cloudflared 2025.2.1 rejecting --output and the phone showing endpoint_request_failed.

    I used the original post's workaround and installed the checksum-verified pinned 2026.5.2 binary under ~/.t3/tools/cloudflared/2026.5.2/darwin-arm64/, leaving my other Homebrew tunnels unchanged. T3 picked it up automatically on the next retry, and its readiness endpoint now reports HTTP 200 with 4 active connections. No app restart was needed. Thanks for documenting this!

    I've been having this issue for a while and thought it was because of getting a new phone

  6. juliusmarminge
    closed this as completedin #17275on Oct 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething is broken or behaving incorrectly.via-triageFiled through npx t3 triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions