Skip to content

Live Oura ring: overnight vitals (skin temp / RHR / HRV) never surface — banked IBI is stored but no hrSample is derived, so the nightly pipeline scores the night NULL #728

Closed

Description

@pipiche38

Summary

For a live Oura ring (oura-<uuid>, sourceKind = oura, active), the nightly skin-temperature reading — and, by the same mechanism, resting HR and HRV — never appears on the vital-detail screens. The Skin Temperature screen shows only WHOOP-imported days and stops at the last import.

Root cause: the ring banks overnight IBI (rrInterval) and skin temp (skinTempSample), but no overnight hrSample is ever produced from that IBI. The nightly analytics pipeline gates on hrSample, so with zero overnight HR the day-owner probe finds no data, ownership falls back to a device with nothing for that day, and the whole night scores NULL — taking skin temp down with it even though the skin-temp samples cover the full night.

Evidence (staging DB, live ring oura-5C4C0BF8-…, overnight 00:00–08:00 local)

Stream 21 Jul 22 Jul Covers night?
rrInterval (banked IBI) 27,080 25,816 ✅
skinTempSample 546 545 ✅
hrSample 0 0 ❌

hrSample for the ring exists only in daytime live-streaming windows (hours when the app was open and connected). Resulting dailyMetric rows for those days are under my-whoop-noop with skinTempDevC, restingHr, avgHrv, totalSleepMin all NULL.

Why it cascades

  1. IntelligenceEngine.resolveDayOwner probes store.hrSamples(deviceId:, night window, limit:1) → 0 → the live ring is not a candidate-with-data → owner falls back (to my-whoop, which has no data those days).
  2. The nightly funnel additionally requires guard hr.count >= 200 in the night window (IntelligenceEngine day loop) → fails regardless of owner.
  3. skinTempDevC / restingHr / avgHrv land NULL.
  4. The Skin Temperature vital screen (MetricExplorerView → Repository.exploreSeries(key:"skin_temp")) reads the per-day DailyMetric.skinTempDevC column + imported WHOOP metricSeries — neither gets a live-Oura value → the Oura days are blank.

Key insight

The overnight HR chart works because that path derives HR from the banked IBI at read time (#511). The daily analytics pipeline instead reads persisted hrSample rows, which are never materialized from the overnight IBI. The two paths disagree — display shows overnight HR, scoring sees none.

Where it lives (both platforms — parity)

  • Storage mapping: Packages/WhoopStore/…/OuraStreamMapping.swift (+ Kotlin com.noop.oura.OuraStreamMapping) — banked IBI (0x60/0x80/0x6E) → .ibi → rrInterval only; live .hr push → hrSample. No banked-window HR derivation.
  • Gate: Strand/Data/IntelligenceEngine.swift resolveDayOwner (HR presence probe) and the day loop's guard hr.count >= 200; Kotlin twins in com.noop.analytics / RegistryDayOwnerSource.

Scope / not-a-regression

  • Not caused by the Oura sleep-persist work — sleep persists from the phase-code stream, independent of HR, so it correctly shows "Oura". Only the HR-gated vitals (skin temp / RHR / HRV) are affected.
  • Pre-existing structural gap in the live-Oura path; affects the ring the moment it's the active device.

Proposed approach

Option A (recommended — the complete fix): derive + persist hrSample from the banked Oura IBI

When the banked overnight IBI is decoded and stored to rrInterval, also compute HR and persist hrSample under the same deviceId, anchored to ring-time (same anchor as the IBI, per #677 — never drain-arrival now):

  • bpm = round(60000 / ibiMs), with the existing honesty guards (drop implausible bpm; already applied for live HR).
  • Cadence: ~27k IBI/night is too dense to store 1:1 as HR; aggregate to a sane cadence (e.g. one HR per rolling window / per second) so hrSample volume matches the live path.
  • Dedup: the daytime live .hr push already writes hrSample; the banked derivation should fill only the gap and dedup by ts so a window covered by both isn't double-counted.
  • Parity: implement as byte-identical Swift/Kotlin twins with a pure, fixture-tested derivation (belongs in Packages / com.noop.oura, not the app layer).
  • Payoff: overnight hrSample ≥ 200 in the night window → resolveDayOwner routes to the live ring → the funnel computes skinTempDevC, restingHr, avgHrv and scores the night. Lights up skin temp and RHR and HRV and proper day-ownership in one change.

Option B (narrower, less complete)

Teach resolveDayOwner's presence probe and the funnel's HR gate to treat rrInterval as HR presence for an Oura owner, and derive an in-memory HR series from IBI inside the funnel. Avoids new stored rows but spreads Oura-special-casing across analytics and still needs an HR series — Option A is cleaner and reusable.

Related follow-up (separate)

The skin_temp daily column is a deviation (DailyMetric.skinTempDevC, ±°C) while the WHOOP import metricSeries is absolute °C — so even once an Oura day scores, it would render on a different scale than the import rows in the readings table. Worth reconciling (own issue) so Oura and WHOOP skin temp read consistently.

Activity

  1. pipiche38 commented on Jul 23, 2026

    @pipiche38
    Author

    #728 — IBI→hrSample derivation: offline simulation vs the ring's own 0x5D HR

    Validated the derivation against a real overnight from the staging DB, cross-checked against the ring's
    own 0x5D hr_bpm (its 5-min average HR) as ground truth.

    Approach under test

    Derive hrSample from the banked IBI (rrInterval): every IBI decoded from one record shares that
    record's ring-time, so group by ring-time (= per record) and emit one median HR per record —
    HR = round(60000 / median(IBI)) over the record's physiological beats (IBI 300–2000 ms, bpm 30–220).
    This is what the pipeline needs, because the daily rebuild reads hrSample and gates on hr.count >= 200
    in the night window.

    Simulation

    • Window: a real persisted overnight, 22:48 → 08:20 (9.5 h), ~30k in-range banked IBIs.
    • Applied the exact algorithm to rrInterval; also tested a Malik 20%-of-median within-record filter.
    • Compared to the 17 0x5D hr_bpm samples in the same window (overall distribution + per-5-min).

    Results

    HR series n median mean p5 (RHR proxy) min max
    #728 (per-record median) 5,101 52 58.9 44 38 162
    #728 + Malik 20% filter 5,011 52 58.8 43 36 168
    0x5D (ring 5-min avg) 17 48 49.5 46 46 59

    Per-5-min agreement (window median of #728 vs the 0x5D sample): bias +6 bpm, mean |diff| ~7 bpm,
    worst windows ±40 bpm (near-wake movement: 07:57 and 08:20 blow out to 73–82 while 0x5D stays ~48–50).

    Findings

    1. Density goal met. Live Oura ring: overnight vitals (skin temp / RHR / HRV) never surface — banked IBI is stored but no hrSample is derived, so the nightly pipeline scores the night NULL #728 yields 5,101 HR rows vs the ring's 17 — comfortably passes the ≥200
      gate and routes the day-owner, so the Oura night will score (unblocks RHR / skin-temp / HRV).
    2. Robust stats agree with the ring. Median 52 vs 48; RHR proxy p5 = 44 vs the ring's floor 46
      (within ~2–3 bpm). The ~4–6 bpm median offset is the expected instantaneous-IBI vs ring-smoothed-average
      difference.
    3. Raw series is noisier than the ring's summary. A high tail (up to 162 bpm) from motion / arousal
      / near-wake movement inflates the mean by ~+9 bpm. Fine for a percentile-based RHR; would bias
      any mean-HR metric.
    4. Within-record Malik filter doesn't help (52→52, mean 58.9→58.8) — the artifacts are whole noisy /
      motion records
      , not within-record outliers (the per-record median already handles those). Cleaning them
      needs a cross-record running-median reject or a motion-gate (0x47/0x6B), which is what the
      ring itself uses.

    Verdict

    The core of #728 is validated for its purpose: it produces the dense HR the pipeline needs, the night
    scores, and the RHR the pipeline derives (a low percentile) matches the ring within a couple bpm. The
    0x5D cross-check confirmed the median and RHR are right. The known limitation is noise in the upper tail,
    which does not affect the percentile-based RHR / skin-temp gating.

    Recommendation — path forward

    1. Ship Live Oura ring: overnight vitals (skin temp / RHR / HRV) never surface — banked IBI is stored but no hrSample is derived, so the nightly pipeline scores the night NULL #728 as-is to unblock nightly scoring (RHR / skin temp / HRV) — the goal is met and the derived
      RHR is honest.
    2. Follow-up refinement (separate): add a motion-gate (drop records overlapping 0x47/0x6B
      motion) and/or a cross-record running-median reject to bring the series toward 0x5D's cleanliness —
      worth doing before any downstream metric relies on mean HR.
    3. Keep 0x5D as the permanent ground-truth cross-check (and the direct, clean source for avgHrv
      via its RMSSD).

    (Simulation scripts: sim728.py / sim728b.py; run against the staging whoop.sqlite.)

  2. pipiche38 commented on Jul 23, 2026

    @pipiche38
    Author
  3. added a commit that references this issue on Jul 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions