Skip to content

selfhost-env-reference generator misses POSTHOG_SERVER_NAME/POSTHOG_MIN_SEVERITY/POSTHOG_REPO_MIN_SEVERITY #8627

Description

@JSONbored

⚠️ Definition of Done: this issue must be completed in full, in a single PR. Do not split this
work across multiple PRs, and do not defer any Deliverable below to a follow-up issue. A PR that
satisfies only some of the Deliverables, stubs a required test, or leaves a checkbox
partially-done does NOT resolve this issue and will be closed.

Context

scripts/gen-selfhost-env-reference.ts walks a fixed set of source files (DEFAULT_SOURCE_ROOTS,
including src/selfhost) with a TypeScript AST visitor (collectEnvReads) and writes every self-host
env var it finds into apps/loopover-ui/src/lib/selfhost-env-reference.ts — the generated reference
operators read to know which env vars self-host actually supports. The visitor only recognizes a
specific, enumerated set of access shapes: env.FOO, env["FOO"], a destructured
const {FOO} = env, and calls to a small allowlist of named helpers (envString,
parsePositiveIntEnv, resolveLocalStoreDbPath, createCliProvider) — see
scripts/gen-selfhost-env-reference.ts:23-33 (isStaticEnvHelperCall/PROCESS_ENV_NAME_HELPERS/
ENV_NAME_LITERAL_ARG_HELPERS) and isEnvContainer (scripts/gen-selfhost-env-reference.ts:150-157).

src/selfhost/posthog.ts (#8287's PostHog error-tracking sink, replacing the deleted
src/selfhost/sentry.ts) reads three real, operator-facing config vars through access patterns the
generator does not recognize:

  1. POSTHOG_MIN_SEVERITY / POSTHOG_REPO_MIN_SEVERITY (src/selfhost/posthog.ts:81) — passed
    as string-literal arguments to resolveSeverityThreshold(...), a helper not in
    ENV_NAME_LITERAL_ARG_HELPERS. (Pre-existing gap: the old SENTRY_MIN_SEVERITY/
    SENTRY_REPO_MIN_SEVERITY calls to the same helper in the now-deleted sentry.ts were never
    detected either — confirmed by diffing the pre-migration committed
    apps/loopover-ui/src/lib/selfhost-env-reference.ts, which never listed them. Documented here for
    completeness but not this issue's core regression.)
  2. POSTHOG_SERVER_NAME (src/selfhost/posthog.ts:179) — this one is a regression the
    migration introduced.
    The deleted sentry.ts read the equivalent var as a plain
    env.SENTRY_SERVER_NAME property access on a typed env parameter, which the generator DOES
    recognize — confirmed: SENTRY_SERVER_NAME is present in the pre-migration committed
    selfhost-env-reference.ts (git show 0e93a850c~1:apps/loopover-ui/src/lib/selfhost-env-reference.ts
    lists it, attributed to src/selfhost/sentry.ts). The new posthog.ts reads the same
    config knob through a globalThis cast instead:
    // src/selfhost/posthog.ts:179
    properties.server_name = nonBlank((globalThis as unknown as { process?: { env?: Record<string, string | undefined> } }).process?.env?.POSTHOG_SERVER_NAME) ?? hostname();
    isEnvContainer requires the base of a .env access to be a bare process/c identifier (after
    unwrapping simple casts on env itself); (globalThis as unknown as {...}).process is a
    property-access expression, not an identifier, so this access is invisible to the visitor. The
    result: POSTHOG_SERVER_NAME is silently absent from the committed
    apps/loopover-ui/src/lib/selfhost-env-reference.ts today, even though its Sentry-era predecessor
    was correctly listed there.

Verified absent by inspection of the currently-committed generated file:
grep -n "POSTHOG_MIN_SEVERITY\|POSTHOG_REPO_MIN_SEVERITY\|POSTHOG_SERVER_NAME" apps/loopover-ui/src/lib/selfhost-env-reference.ts
returns zero matches, while POSTHOG_API_KEY/POSTHOG_ENVIRONMENT/POSTHOG_HOST/POSTHOG_RELEASE
(all read as plain env.FOO elsewhere, e.g. src/selfhost/otel.ts) are present.

This is the same class of bug #6993 ("selfhost-env-reference generator misses AI review-pipeline env
vars") already fixed once for a different blind spot — the generator's coverage of real access
patterns in this codebase keeps trailing what the codebase actually does, and each trailing gap
silently drops a real, operator-relevant config var from the doc operators are meant to trust as
complete.

Requirements

  • The generator must recognize POSTHOG_SERVER_NAME's access pattern in
    src/selfhost/posthog.ts:179 specifically: a property access on .env whose base ultimately
    resolves to globalThis.process via an intermediate as unknown as {...} cast, not just a direct
    process/c identifier.
  • The generator must also recognize the resolveSeverityThreshold(env, repo, "VAR_A", "VAR_B")
    call shape (arguments 2 and 3 are env var name literals) so POSTHOG_MIN_SEVERITY and
    POSTHOG_REPO_MIN_SEVERITY are captured too — closing the pre-existing gap alongside the migration
    regression, since both live in the exact same function this issue is already touching
    (src/selfhost/posthog.ts's resolvePostHogMinSeverity) and leaving one fixed while the other
    stays broken in the same file would be an inconsistent half-fix.
  • No soft-matching or file-scoped special-casing (e.g. hardcoding POSTHOG_SERVER_NAME as a literal
    exception) — the fix must generalize the AST visitor's pattern recognition so any future var read the
    same way is also caught, matching how selfhost-env-reference generator misses AI review-pipeline env vars #6993's fix generalized rather than special-cased.
  • After the fix, regenerating the reference (npm run selfhost:env-reference) must produce a diff that
    adds exactly POSTHOG_MIN_SEVERITY, POSTHOG_REPO_MIN_SEVERITY, and POSTHOG_SERVER_NAME (each
    attributed to src/selfhost/posthog.ts) and changes nothing else.

Deliverables

  • scripts/gen-selfhost-env-reference.ts's isEnvContainer (or a new sibling check) recognizes
    the (x as unknown as {process?: {env?: ...}}).process?.env?.FOO shape used at
    src/selfhost/posthog.ts:179, verified by a new unit test in
    test/unit/selfhost-env-reference-script.test.ts that feeds a fixture source string using this
    exact shape into collectSelfHostEnvVars/collectEnvReads and asserts the var name is returned.
  • scripts/gen-selfhost-env-reference.ts recognizes the
    resolveSeverityThreshold(envArg, repoArg, "VAR_A", "VAR_B") call shape (both string-literal name
    arguments captured), verified by a new unit test asserting both POSTHOG_MIN_SEVERITY and
    POSTHOG_REPO_MIN_SEVERITY are returned from a fixture exercising this exact call.
  • npm run selfhost:env-reference is re-run and its output
    (apps/loopover-ui/src/lib/selfhost-env-reference.ts) is committed with the resulting diff, adding
    POSTHOG_MIN_SEVERITY, POSTHOG_REPO_MIN_SEVERITY, and POSTHOG_SERVER_NAME (all three, in the
    same PR — a PR regenerating the reference without the generator fix, or fixing the generator without
    committing the regenerated output, does not resolve this issue).
  • npm run selfhost:env-reference -- --check passes clean against the newly-committed output (this
    is the same drift check CI runs).

All four Deliverables are required together in one PR — the generator fix and the regenerated,
committed output are two halves of the same change and neither is meaningful without the other.

Test Coverage Requirements

scripts/gen-selfhost-env-reference.ts sits under scripts/**, which codecov.yml's top-level
ignore list explicitly excludes — this PR's patch coverage on that file will NOT be gated by
Codecov, so do not expect codecov/patch to fail or pass based on it. This does not relax the
testing bar: test/unit/selfhost-env-reference-script.test.ts still runs in npm run test:ci and
must pass locally and in CI regardless of Codecov's blind spot. Add a passing fixture test AND a
fixture that deliberately does NOT match (to prove the new branch doesn't over-match unrelated code)
for both the new globalThis-cast branch and the new resolveSeverityThreshold-call branch — four
new test cases minimum, none of them stubbed.

Expected Outcome

apps/loopover-ui/src/lib/selfhost-env-reference.ts — and the docs page it feeds — lists every real
self-host PostHog config var, matching the completeness the pre-migration Sentry-era reference had for
the equivalent vars. An operator configuring severity thresholds or a custom server name for PostHog
error tracking can find both documented in the generated reference, the same way they could for Sentry
before this migration.

Links & Resources

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions