Skip to content

selfhost(pg): julianday() is never translated #9648

Description

@JSONbored

⚠️ Definition of Done: this issue must be completed in full, in a single PR. Do not split this
work across multiple PRs, and do not defer any Deliverable below to a follow-up issue. A PR that
satisfies only some of the Deliverables, stubs a required test, or leaves a checkbox
partially-done does NOT resolve this issue and will be closed.

Context

listSubmitterCohortRows (src/review/submitter-reputation.ts:412) is the per-submitter cohort tally behind
the AMS-vs-human comparison (src/review/ams-miner-cohort.ts:118). Its SQL uses SQLite's julianday():

AVG(CASE WHEN po.decision = 'merged' AND pr.merged_at IS NOT NULL
         THEN (julianday(pr.merged_at) - julianday(pr.created_at)) * 86400000 ELSE NULL END) AS avgMergeMs

storage(env) here is env.DB (line 42), which on the self-host Postgres backend is the pg adapter — every
statement goes through translateSql (src/selfhost/pg-dialect.ts:232). translateFunctions (line 66)
translates strftime, datetime, CURRENT_TIMESTAMP, date, json_extract, json_each and instr. It has
no rule for julianday, and Postgres has no such function, so the statement fails outright with
function julianday(text) does not exist.

The failure is invisible: line 438's catch { return []; } swallows it, and the caller treats [] as "no
activity in the window". So on every self-host Postgres deployment the submitter-cohort surface reports an
empty cohort forever, with no error in logs, /health, or preflight.

This is the exact failure shape pg-dialect.ts documents twice for its own prior incidents — date() (#8171,
lines 79-84: "silently bucketed NOTHING on self-host") and instr() (#9084, lines 98-112: "the failure lands
in a fail-safe read path that swallows it to an empty result"). julianday is the third member of that family
and is the only remaining untranslated SQLite-only scalar function in src/** (verified by sweeping src/ for
julianday|json_group_array|group_concat|printf|iif|unixepochjulianday at line 421 is the sole hit).

There is no guard that would have caught it: test/unit/selfhost-pg-dialect.test.ts and
test/unit/pg-dialect-numeric-and-nesting.test.ts test each translation rule against hand-written SQL strings;
nothing scans the real src/** query text for functions the translator does not handle.

Requirements

  • translateFunctions in src/selfhost/pg-dialect.ts translates julianday(<expr>) to the Postgres
    equivalent Julian Day Number as a numeric value:
    (EXTRACT(EPOCH FROM (<expr>)::timestamptz) / 86400.0 + 2440587.5).
    This must preserve the existing arithmetic exactly: a difference of two translated calls multiplied by
    86400000 must still yield milliseconds.
  • The rule must use ::timestamptz, matching the existing date() and strftime() rules in the same function,
    so the TEXT ISO timestamps the app writes are interpreted identically.
  • A new drift guard must fail CI when a SQLite-only scalar function with no translation rule appears in src/**
    SQL. The guard's checked set is exactly: julianday, unixepoch, json_group_array, json_array_length,
    group_concat, printf, iif, glob, randomblob, total. A function in that set may appear in src/**
    only if translateFunctions/translateSql demonstrably rewrites it — the guard asserts this by feeding
    translateSql("SELECT <fn>(x) FROM t") and requiring the output no longer contains the function name.

⚠️ Required pattern: add the rule inside the existing translateFunctions .replace(...) chain in
src/selfhost/pg-dialect.ts:66-113, with a header comment in the same style as the date() (#8171) and
instr() (#9084) rules naming the swallow path it closes. Implement the guard as a vitest case in
test/unit/selfhost-pg-dialect.test.ts that reads src/**/*.ts from disk — mirroring
test/unit/retention.test.ts:728's "every policy table has an index leading with its retention column
somewhere in migrations/" drift guard, which reads the real tree the same way. What does NOT satisfy this
issue: rewriting listSubmitterCohortRows' SQL to avoid julianday instead of teaching the translator (the
translator is the shared seam and the next such query would break again); adding the translation without the
guard; or implementing the guard as a scripts/ CLI rather than a test (it must run inside the existing
vitest suite).

Deliverables

  • translateFunctions in src/selfhost/pg-dialect.ts rewrites julianday(col) and nested/whitespaced
    forms, asserted by new cases in test/unit/selfhost-pg-dialect.test.ts including the exact expression
    from src/review/submitter-reputation.ts:421.
  • A named regression test asserting translateSql applied to listSubmitterCohortRows' full statement text
    contains no remaining julianday.
  • The drift guard described above exists as a test that fails when any of the ten listed functions appears
    in src/** without a translation rule, and passes against the tree as it stands after this fix.
  • The guard's own failure mode is covered: a test that the guard's detection helper flags a synthetic
    untranslated function string (so the guard is not trivially green).

All Deliverables above are required in a single PR. A PR that satisfies only some of them — for example adding
the julianday rule without the drift guard from Deliverable 3 — does not resolve this issue.

Test Coverage Requirements

This repo enforces 99%+ Codecov patch coverage, branch-counted, on src/**. Both
src/selfhost/pg-dialect.ts and src/review/submitter-reputation.ts are inside coverage.include. Every arm
of the new replace rule (matched / not matched) needs a test, and the Deliverable-2 test is the required named
regression test for this fix.

Expected Outcome

listSubmitterCohortRows returns real rows on the self-host Postgres backend instead of silently returning
[], so buildAmsMinerCohort reports a populated cohort there; and a future SQLite-only scalar function added
to a src/** query fails CI instead of being swallowed into an empty result on Postgres.

Links & Resources

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions