[bug] v2 self-host: memory grows to 5–10 GB with uptime; "Telemetry export failed" (transport) rate climbs until restart #2216

Executor version

2.0.0-beta.10 (ghcr.io/usefulsoftwareco/executor-selfhost:beta)

How do you run Executor?

Self-host (Docker) on Railway, 1 replica, with a volume at /app/data.

Integration involved

None in particular. Light MCP traffic from a few clients.

What happened

Container memory (RSS) grows steadily with uptime and only drops on restart. Over this billing cycle, memory was about 90% of the hosting cost.

The memory graph has a sawtooth shape: it climbs between deploys and restarts, then resets.

OTLP export is configured to an external collector (OTEL_EXPORTER_OTLP_ENDPOINT=http://<collector>:4318, otel/opentelemetry-collector-contrib:0.162.0). The collector is healthy, uses ~90 MB of RAM, and keeps receiving traces and metrics from both executor-selfhost and executor-app.

Correlated log signal

After about a day of uptime, the server starts logging Telemetry export failed. The rate rises with uptime and drops to zero after a restart:

Every entry has the same annotations:

{"executor.telemetry.failure":"transport","executor.telemetry.signal":"other","executor.telemetry.rejected_records":0}
spans: {}

The logged fiberId went from #2568204 to #7385470 over about 6.5 h, roughly 200 new fibers/s on an almost idle instance.

My guess, which I have not confirmed: memory growth starts before the export failures, so the failures may be a symptom. A high idle span volume, as described in #2194 with the 1 s scheduler poll traced, combined with an export path that buffers or retries without a bound, could keep growing heap. It could also be a separate leak that eventually slows down the exporter.

Other startup lines, possibly unrelated:

• service motel: The compatibility flag nodejs_compat became the default... (the bundled Motel still starts even with an OTLP endpoint set)
• [Better Auth]: Field lastRequest in table rateLimit has a different type in the database. Expected number but got int8.

What you expected

Memory stays roughly flat on an almost idle self-host instance. Telemetry export failures should not build up over time, and buffers should be bounded.

Steps to reproduce

1. Run executor-selfhost:beta (2.0.0-beta.10) with OTEL_EXPORTER_OTLP_ENDPOINT pointing at an OTel collector.
2. Leave it running with light traffic for 1–2 days.
3. Watch container RSS, and grep logs for Telemetry export failed.

Possibly related

• [bug] v2 self-host: idle container burns ~13% CPU and writes ~190 GiB/day once bundled Motel hits its 1 GiB cap #2194 (idle scheduler poll fully traced, Motel churn)
• Memory Leak #1565 (earlier desktop memory leak report)