Repository navigation
Vital detail: show the value on a single reading for every metric, not just Fitness Age - #156
Merged
Merged
Conversation
…t just Fitness Age #154 fixed the "value on the card, 'Not enough history yet' on tap" contradiction for Fitness Age only. Vitality (another weekly-ish computed score) hits the same gate, and so would any metric sitting at one reading. Generalize the single-reading value card from key == "fitness_age" to any metric, with generic copy ("One reading so far — your trend chart fills in once a second reading lands"), and make the header subtitle follow suit. Matches iOS, which already renders the value hero for any metric at >= 1 point. Fitness Age's zero-reading readiness card stays fitness-specific; genuine no-data (size 0) still shows the honest note.
ryanbr
pushed a commit
that referenced
this pull request
Jul 15, 2026
…up) (#415) * feat(store): persist the WHOOP 5.0 v26 raw PPG waveform (#156 follow-up) The strap's 24 Hz optical PPG buffer (type-47 v26 records) was fully decoded (ppg_waveform, 24 i16 ADC samples/record) but only ever used to derive a per-second HR estimate (ppgHrSample, v12) — the waveform itself was discarded right after, and explicitly excluded from the undecodable-record reject archive as "known-and-unstored by design". This reverses that: the samples now survive in their own durable stream. - WhoopProtocol: Streams gains ppgWaveform: [PpgWaveformSample] (ts + raw i16 samples), populated by extractHistoricalStreams alongside the existing ppgHr derivation. isEmpty/CodingKeys/decoder updated so a waveform-only decode (too little context for a confident HR estimate) still counts as decoded data, and older JSON fixtures without the key still decode. - WhoopStore: new v27 migration creates ppgWaveformSample (deviceId, ts, samples BLOB), one row per second like every sibling stream, but with the 24 samples packed as compact little-endian i16 (2 bytes/sample) instead of 24 scalar rows — a v26-heavy night costs about the same order of magnitude as one extra per-second stream, not 24x that. Wired into the existing insert(_:deviceId:) choke point (persist-only, mirrors steps/sleepState/ ppgHr) so the Collector/Backfiller paths pick it up with no changes of their own. Added ppgWaveformSample to DeviceRegistryStore.deviceScopedTables so "delete all of this device's data" clears it too (caught by the existing coverage test). - No new pruning: matches every other durable per-second table (hrSample, spo2Sample, ... are never pruned either). This is decoded biometric history, not the transient raw outbox — PrunePolicy's ~50 MB cap governs only rawBatch and is untouched. - Android's Kotlin HistoricalStreams.kt/PpgHr.kt has the identical decode-then-discard (only ppgHrSample persists); no Room table or entity exists for the raw waveform. Left as a follow-up, not implemented here. swift test is green in both packages (WhoopProtocol 281 tests, WhoopStore 266 tests), and the macOS Strand scheme builds clean. * feat(android): Kotlin twin of the v26 raw PPG waveform store (parity with PR #415) Ports the durable WHOOP 5.0 v26 optical-PPG waveform storage from WhoopStore (issue #156 follow-up). WHOOP straps connect on Android, so this twins BOTH the Room schema AND the store-write plumbing: - Room migration v19 -> v20 + @entity ppgWaveformSample (deviceId, ts, samples BLOB; PK deviceId, ts), matching the GRDB v27-ppg-waveform table. Pinned by PpgWaveformMigrationTest. (v19 is the efficiency-heal on this branch, so the PPG table takes v20, twinning Swift's v27 = next slot after v26.) - StreamPersistence.packPpgSamples/unpackPpgSamples: little-endian i16, BYTE-identical to the Swift WhoopStore packer so a .noopbak round-trips. - extractHistoricalStreams now also collects the raw per-second waveform into StreamBatch.ppgWaveform; WhoopRepository.insert packs + persists it (IGNORE-dedupe by deviceId, ts); ppgWaveformSamples() reads it back. - DeviceRegistry device-data wipe clears ppgWaveformSample too (privacy parity). Tests: PpgWaveformMigrationTest (schema + pack/unpack + insert wiring) and Whoop5PpgWaveformStreamTest (extraction over the same real v26 fixture the Swift Whoop5PpgWaveformTests uses).
DX23876
pushed a commit
to DX23876/noop
that referenced
this pull request
Jul 22, 2026
…neralize Vital detail: show the value on a single reading for every metric, not just Fitness Age
DX23876
pushed a commit
to DX23876/noop
that referenced
this pull request
Jul 22, 2026
…ollow-up) (ryanbr#415) * feat(store): persist the WHOOP 5.0 v26 raw PPG waveform (ryanbr#156 follow-up) The strap's 24 Hz optical PPG buffer (type-47 v26 records) was fully decoded (ppg_waveform, 24 i16 ADC samples/record) but only ever used to derive a per-second HR estimate (ppgHrSample, v12) — the waveform itself was discarded right after, and explicitly excluded from the undecodable-record reject archive as "known-and-unstored by design". This reverses that: the samples now survive in their own durable stream. - WhoopProtocol: Streams gains ppgWaveform: [PpgWaveformSample] (ts + raw i16 samples), populated by extractHistoricalStreams alongside the existing ppgHr derivation. isEmpty/CodingKeys/decoder updated so a waveform-only decode (too little context for a confident HR estimate) still counts as decoded data, and older JSON fixtures without the key still decode. - WhoopStore: new v27 migration creates ppgWaveformSample (deviceId, ts, samples BLOB), one row per second like every sibling stream, but with the 24 samples packed as compact little-endian i16 (2 bytes/sample) instead of 24 scalar rows — a v26-heavy night costs about the same order of magnitude as one extra per-second stream, not 24x that. Wired into the existing insert(_:deviceId:) choke point (persist-only, mirrors steps/sleepState/ ppgHr) so the Collector/Backfiller paths pick it up with no changes of their own. Added ppgWaveformSample to DeviceRegistryStore.deviceScopedTables so "delete all of this device's data" clears it too (caught by the existing coverage test). - No new pruning: matches every other durable per-second table (hrSample, spo2Sample, ... are never pruned either). This is decoded biometric history, not the transient raw outbox — PrunePolicy's ~50 MB cap governs only rawBatch and is untouched. - Android's Kotlin HistoricalStreams.kt/PpgHr.kt has the identical decode-then-discard (only ppgHrSample persists); no Room table or entity exists for the raw waveform. Left as a follow-up, not implemented here. swift test is green in both packages (WhoopProtocol 281 tests, WhoopStore 266 tests), and the macOS Strand scheme builds clean. * feat(android): Kotlin twin of the v26 raw PPG waveform store (parity with PR ryanbr#415) Ports the durable WHOOP 5.0 v26 optical-PPG waveform storage from WhoopStore (issue ryanbr#156 follow-up). WHOOP straps connect on Android, so this twins BOTH the Room schema AND the store-write plumbing: - Room migration v19 -> v20 + @entity ppgWaveformSample (deviceId, ts, samples BLOB; PK deviceId, ts), matching the GRDB v27-ppg-waveform table. Pinned by PpgWaveformMigrationTest. (v19 is the efficiency-heal on this branch, so the PPG table takes v20, twinning Swift's v27 = next slot after v26.) - StreamPersistence.packPpgSamples/unpackPpgSamples: little-endian i16, BYTE-identical to the Swift WhoopStore packer so a .noopbak round-trips. - extractHistoricalStreams now also collects the raw per-second waveform into StreamBatch.ppgWaveform; WhoopRepository.insert packs + persists it (IGNORE-dedupe by deviceId, ts); ppgWaveformSamples() reads it back. - DeviceRegistry device-data wipe clears ppgWaveformSample too (privacy parity). Tests: PpgWaveformMigrationTest (schema + pack/unpack + insert wiring) and Whoop5PpgWaveformStreamTest (extraction over the same real v26 fixture the Swift Whoop5PpgWaveformTests uses).
This was referenced Jul 26, 2026
ryanbr
added a commit
that referenced
this pull request
Jul 26, 2026
The chart and the average read different tables. The chart uses WhoopDao.hrSamples, which UNIONs measured hrSample with PPG-derived ppgHrSample (#156). hrWindowStats, which backs the Avg HR, read hrSample alone. fillWorkoutHrFromStrap needs 60 samples before it shows a number. On a WHOOP 5 the firmware banks v26 PPG instead of v18 HR per second, so a PPG-heavy workout draws a full chart and still counts under 60 measured rows — Avg HR renders blank. That is exactly the reporter's "as the HR data is shown on the chart it can be calculated", and it explains why only SOME workouts are affected: it depends on how PPG-heavy that particular window is. Manual and Health Connect rows both hit it because both pass through the same gate. iOS never had this. Its twin reduces store.hrSamples, the coalescing read — so this was also a parity break: same data, different number per platform. The Android query looks like a deliberate optimisation, aggregate in SQL rather than materialise up to 8000 rows, and its own comment sells that ("no hrSamples LIMIT truncation"). It is the better shape; it just dropped the PPG union on the way. Now aggregates the same measured-∪-PPG row set, with the identical anti-join, so a measured second is never double-counted by its PPG estimate. maxHr can now be a PPG-derived peak — correct, and the point: the whole function exists to keep the shown number agreeing with the graph and zones, which already include those rows. Verified against real SQLite with the SQL extracted from source: on a 20-measured / 400-PPG window the aggregate returns n=415 avg=141.98 max=146, identical to reducing the chart's union, where the old query returned n=20 and fell below the floor. Not verified by CI: no SQLite driver on the JVM unit-test classpath and android.yml is disabled. Single-query change, only caller is fillWorkoutHrFromStrap.
ryanbr
added a commit
that referenced
this pull request
Jul 26, 2026
The chart and the average read different tables. The chart uses hrBuckets and the zone/Effort paths use hrSamples — all three UNION measured hrSample with PPG-derived ppgHrSample (#156). hrWindowStats, which backs Avg HR, read hrSample alone. Three of four consumers of the same workout window counted the union; the fourth did not. fillWorkoutHrFromStrap needs 60 samples before it shows anything. On a WHOOP 5 the firmware banks v26 PPG instead of v18 HR per second, so a PPG-heavy workout draws a full chart and still counts under 60 measured rows — Avg HR renders blank. That is the reporter (#836) exactly: "as the HR data is shown on the chart it can be calculated", only SOME workouts affected (PPG density varies per window), and manual AND Health Connect both hit because both pass the same gate. Effort is blocked by the same gate, which is why the report said "all details", plural. Both platforms now aggregate the same measured-∪-PPG rows, with the identical anti-join, so a measured second is never double-counted by its PPG estimate. maxHr can now be a PPG-derived peak — correct, and the point: this function exists to keep the shown number agreeing with the graph and zones, which already included those rows. Stored PPG rows already cleared a 0.3 confidence floor at estimate time on both platforms, so nothing unvetted enters. iOS had a second, independent bug. It reduced store.hrSamples(limit: 8000), so any workout over ~2h13m at 1 Hz reported the mean of its FIRST 8000 samples as the session average — 131 bpm against a true 135 on a 3h session with drifting HR. Adds WhoopStore.hrWindowStats, the Kotlin query's twin, uncapped. Swift keeps CAST(ROUND(p.bpm) AS INTEGER) because its ppgHrSample.bpm column is REAL where Kotlin's is INTEGER; both decoders emit whole bpm (#219), and each query returns (415, 141.8867469879518, 146) against its own platform's schema. Put in WhoopStore rather than Repository on purpose: swift-packages.yml runs Packages/** tests while Strand/ is the app target nothing compiles, so the query and its five tests get real CI coverage the old in-Repository reduce never had. Also improves #833 rather than fighting it. That freeze fix moved an 8000-row materialise-and-reduce off the main actor; the common path now materialises no rows at all. Rows are read only for a strain backfill, where the 8000 cap correctly stays — it bounds the strain window, not the average. Same shape as Kotlin. No schema change, no migration, read-only. Empty windows still return n=0 / null avg so the caller guard is unchanged, and a device with no PPG at all (WHOOP 4, or a 5 that banked v18 throughout) reads byte-identically to before. Retires Repository.reduceWorkoutHr, which had no callers left, and its test file. Not verified by CI: the Kotlin query has not been through Room KSP (android.yml is disabled), though hrBuckets ten lines above is the same aggregate-over-union-subquery shape and ships today; and the Repository.swift wiring is in the uncompiled app target. A staging build exercises both. Unconfirmed: whether the reporter's specific workouts are PPG-heavy. The mechanism is established from the code; their export would confirm it.
simoncad7
pushed a commit
to simoncad7/noop
that referenced
this pull request
Aug 17, 2026
…neralize Vital detail: show the value on a single reading for every metric, not just Fitness Age
simoncad7
pushed a commit
to simoncad7/noop
that referenced
this pull request
Aug 17, 2026
…ollow-up) (ryanbr#415) * feat(store): persist the WHOOP 5.0 v26 raw PPG waveform (ryanbr#156 follow-up) The strap's 24 Hz optical PPG buffer (type-47 v26 records) was fully decoded (ppg_waveform, 24 i16 ADC samples/record) but only ever used to derive a per-second HR estimate (ppgHrSample, v12) — the waveform itself was discarded right after, and explicitly excluded from the undecodable-record reject archive as "known-and-unstored by design". This reverses that: the samples now survive in their own durable stream. - WhoopProtocol: Streams gains ppgWaveform: [PpgWaveformSample] (ts + raw i16 samples), populated by extractHistoricalStreams alongside the existing ppgHr derivation. isEmpty/CodingKeys/decoder updated so a waveform-only decode (too little context for a confident HR estimate) still counts as decoded data, and older JSON fixtures without the key still decode. - WhoopStore: new v27 migration creates ppgWaveformSample (deviceId, ts, samples BLOB), one row per second like every sibling stream, but with the 24 samples packed as compact little-endian i16 (2 bytes/sample) instead of 24 scalar rows — a v26-heavy night costs about the same order of magnitude as one extra per-second stream, not 24x that. Wired into the existing insert(_:deviceId:) choke point (persist-only, mirrors steps/sleepState/ ppgHr) so the Collector/Backfiller paths pick it up with no changes of their own. Added ppgWaveformSample to DeviceRegistryStore.deviceScopedTables so "delete all of this device's data" clears it too (caught by the existing coverage test). - No new pruning: matches every other durable per-second table (hrSample, spo2Sample, ... are never pruned either). This is decoded biometric history, not the transient raw outbox — PrunePolicy's ~50 MB cap governs only rawBatch and is untouched. - Android's Kotlin HistoricalStreams.kt/PpgHr.kt has the identical decode-then-discard (only ppgHrSample persists); no Room table or entity exists for the raw waveform. Left as a follow-up, not implemented here. swift test is green in both packages (WhoopProtocol 281 tests, WhoopStore 266 tests), and the macOS Strand scheme builds clean. * feat(android): Kotlin twin of the v26 raw PPG waveform store (parity with PR ryanbr#415) Ports the durable WHOOP 5.0 v26 optical-PPG waveform storage from WhoopStore (issue ryanbr#156 follow-up). WHOOP straps connect on Android, so this twins BOTH the Room schema AND the store-write plumbing: - Room migration v19 -> v20 + @entity ppgWaveformSample (deviceId, ts, samples BLOB; PK deviceId, ts), matching the GRDB v27-ppg-waveform table. Pinned by PpgWaveformMigrationTest. (v19 is the efficiency-heal on this branch, so the PPG table takes v20, twinning Swift's v27 = next slot after v26.) - StreamPersistence.packPpgSamples/unpackPpgSamples: little-endian i16, BYTE-identical to the Swift WhoopStore packer so a .noopbak round-trips. - extractHistoricalStreams now also collects the raw per-second waveform into StreamBatch.ppgWaveform; WhoopRepository.insert packs + persists it (IGNORE-dedupe by deviceId, ts); ppgWaveformSamples() reads it back. - DeviceRegistry device-data wipe clears ppgWaveformSample too (privacy parity). Tests: PpgWaveformMigrationTest (schema + pack/unpack + insert wiring) and Whoop5PpgWaveformStreamTest (extraction over the same real v26 fixture the Swift Whoop5PpgWaveformTests uses).
simoncad7
pushed a commit
to simoncad7/noop
that referenced
this pull request
Aug 17, 2026
…br#841) The chart and the average read different tables. The chart uses hrBuckets and the zone/Effort paths use hrSamples — all three UNION measured hrSample with PPG-derived ppgHrSample (ryanbr#156). hrWindowStats, which backs Avg HR, read hrSample alone. Three of four consumers of the same workout window counted the union; the fourth did not. fillWorkoutHrFromStrap needs 60 samples before it shows anything. On a WHOOP 5 the firmware banks v26 PPG instead of v18 HR per second, so a PPG-heavy workout draws a full chart and still counts under 60 measured rows — Avg HR renders blank. That is the reporter (ryanbr#836) exactly: "as the HR data is shown on the chart it can be calculated", only SOME workouts affected (PPG density varies per window), and manual AND Health Connect both hit because both pass the same gate. Effort is blocked by the same gate, which is why the report said "all details", plural. Both platforms now aggregate the same measured-∪-PPG rows, with the identical anti-join, so a measured second is never double-counted by its PPG estimate. maxHr can now be a PPG-derived peak — correct, and the point: this function exists to keep the shown number agreeing with the graph and zones, which already included those rows. Stored PPG rows already cleared a 0.3 confidence floor at estimate time on both platforms, so nothing unvetted enters. iOS had a second, independent bug. It reduced store.hrSamples(limit: 8000), so any workout over ~2h13m at 1 Hz reported the mean of its FIRST 8000 samples as the session average — 131 bpm against a true 135 on a 3h session with drifting HR. Adds WhoopStore.hrWindowStats, the Kotlin query's twin, uncapped. Swift keeps CAST(ROUND(p.bpm) AS INTEGER) because its ppgHrSample.bpm column is REAL where Kotlin's is INTEGER; both decoders emit whole bpm (ryanbr#219), and each query returns (415, 141.8867469879518, 146) against its own platform's schema. Put in WhoopStore rather than Repository on purpose: swift-packages.yml runs Packages/** tests while Strand/ is the app target nothing compiles, so the query and its five tests get real CI coverage the old in-Repository reduce never had. Also improves ryanbr#833 rather than fighting it. That freeze fix moved an 8000-row materialise-and-reduce off the main actor; the common path now materialises no rows at all. Rows are read only for a strain backfill, where the 8000 cap correctly stays — it bounds the strain window, not the average. Same shape as Kotlin. No schema change, no migration, read-only. Empty windows still return n=0 / null avg so the caller guard is unchanged, and a device with no PPG at all (WHOOP 4, or a 5 that banked v18 throughout) reads byte-identically to before. Retires Repository.reduceWorkoutHr, which had no callers left, and its test file. Not verified by CI: the Kotlin query has not been through Room KSP (android.yml is disabled), though hrBuckets ten lines above is the same aggregate-over-union-subquery shape and ships today; and the Repository.swift wiring is in the uncompiled app target. A staging build exercises both. Unconfirmed: whether the reporter's specific workouts are PPG-heavy. The mechanism is established from the code; their export would confirm it.
This was referenced Sep 9, 2026
ryanbr
added a commit
that referenced
this pull request
Sep 9, 2026
The strap log told a user NOOP could not decode a layout it decodes. The test behind that line asked whether a record carried any of `heart_rate`, `gravity_x` or `ppg_waveform`, a list that has to be extended by hand for every layout NOOP learns and twice silently was not. #156 was v25 and v26 warning after they had decoded for releases, and v20 has warned the same way ever since it was mapped, because the 5/MG optical record decodes to block counts and per-block headers and to none of those three names. Suppressing the line for v20 would have been the wrong correction, because only half of what it said was wrong. Those records really do carry no heart rate and no motion, and a night made only of them really cannot be staged, so silencing it would have removed the only thing in the log explaining why a user's nights were empty. The question splits instead: whether the layout decodes is asked of the dispatch table the decoder itself switches on, which cannot drift from what NOOP decodes; whether the record carries anything scoreable stays the field test, which is what that test was always really measuring. A mapped layout carrying nothing scoreable gets its own line saying exactly that. The verdict is taken over the layout rather than over whichever record came first, so one thin or off-wrist record cannot condemn its layout for a session. Android now reports every distinct layout in a chunk rather than only the first record's, which is the mixed-layout strap the guidance exists for, and emits the same two messages Swift does; it previously printed a bare version and no explanation at all. The reject archive's decode-outcome test now asks whether a record yielded a unix AND either a heart rate or gravity, matching Swift, rather than merely whether it decoded. That changes nothing today, because every layout mapped on Android either yields a heart rate or has its own durable stream. It stops agreeing the moment a layout is mapped whose records carry neither, which is what porting the optical and raw-IMU decoders does, so landing it first makes that port safe rather than a trade. Both decisions are pure functions with twins on the other platform, and the layout test is driven off the dispatch set rather than a literal list, so a layout added there without a signature field cannot bring the warning back a third time. Also here: the frontier work this branch opened with came OUT. It measured the offload backlog against the newest persisted row, which a record that becomes no rows can never advance, and the replacement read the HISTORY_END unix as the acked section's end time. The comment at that call site says it carries the strap's own clock. If that is the live RTC the safety property inverts and it would close every gap on every strap, so it is out until the field's meaning is established from a capture. The diagnosis stands and is recorded on #1992.
suprfire
pushed a commit
to suprfire/noop
that referenced
this pull request
Oct 10, 2026
…for how far tonight's data can be trusted (ryanbr#15/ryanbr#27) Composes the honesty signals that were scattered across consumers into ONE per-night figure: stream coverage, PPG-derived-HR fill share (ryanbr#156), gravity span sparsity and >20-min dropouts (the ryanbr#345 sparse test), and off-wrist seconds inside the bed window (ryanbr#500/ryanbr#504). Channels a device never carries (thermal/SpO2 on WHOOP 4.0) are ABSENT from the weighted mean, never scored zero - a 4.0 night is not punished for hardware it does not own; absent components renormalize the weights (the Charge nil-renorm idiom). Persisted as shadow keys night_trust / night_wear_frac under the computed source: instrumentation only, never shown, never fed to any score, accruing on real devices so the trust surface and later alert gating ship on measured data. Wiring respects the ryanbr#1524 JaCoCo per-method budget: analyzeRecentOnCpu measured 57,412 instrumented bytes inline against its 55,700 ceiling, so the census (recordNightTrust/nightTrustCensus), the six per-night shadow-row appends (shadowMetricRows/nightTrustShadowRows), the cache-hit replay (replayCachedNight), the cache store (storeCachedNightScan) and the sleep-diag line (sleepDayDiagLine) are lifted into named helpers; the census call site is one statement. Gates: NightlyTrustTest 13/13 (clean night, off-wrist, PPG-fill, gravity gap/sparse, device-without-thermal renorm, degenerate inputs); IntelligenceEngineJacocoBudgetTest 4/4; full com.noop.analytics.* suite green; doc_comment_lint OK; i18n audit: Android section clean (no new hardcoded copy) - its iOS xcstrings ru/zh-Hant FAILs are pre-existing at HEAD and this diff touches android/ + CHANGELOG.md only.
This was referenced Oct 24, 2026
ecg: R16 records decode but have nowhere to be stored, and the raw-archive floor is 64 seconds
#2847
Closed
ryanbr
added a commit
that referenced
this pull request
Oct 25, 2026
Refs #2847, #891, #2839, #156, #752. #2839 taught both platforms to decode the record. It had nowhere to put one. Layout version 16 stayed out of `mappedWhoop5HistoricalVersions`, so every offloaded R16 record went down the unmapped route into `RawHistoryArchive`, a 5 MiB forensic ring whose per-version floor protects 64 informative lines. For a record of one strap-second that is 64 seconds of waveform, and @DX23876 measured a reading evicted down to that floor within a couple of hours of ordinary MG traffic, because R20 and R21 records compete for the same file constantly. ## The table `ecgWaveformSample` on both platforms, one row per record: deviceId, ts, sequence, record. GRDB `v48-ecg-waveform`, Room 41 to 42, same column order and the same primary key, so a `.noopbak` round-trips. The key is (deviceId, ts, SEQUENCE), three columns where every per-second stream beside it keys on two, and that is a re-review finding rather than a first draft. R16 carries a `subseconds` field at all, and the opening records of a real run declare 0 samples and then 245 before the first full 500, so two of them can be stamped inside one strap-second. On (deviceId, ts) the second of that pair hit ON CONFLICT DO NOTHING and was dropped with no error anywhere, which measured as 2 rows decoded and 1 row stored. Re-offloading the same record still dedupes, because its ts AND its sequence both repeat; both halves are pinned separately. `record` is the record's INNER bytes verbatim, all 1,572 of them. The wire form rather than a decoded field set, and that is the load-bearing choice: - the waveform is 500 three-byte slots of signed 18-BIT samples, so no integer blob holds one. @meta1971 measured them reaching plus or minus 126,977 over 13,178,493 samples, which settles the width with data rather than a guess; - the 22 lead-off halfwords have no attested byte order, so a decoded column would bake a guess into storage. Verbatim bytes let a later re-reading happen with no re-pull, which is what @meta1971 asked for on the issue; - `Whoop5Ecg.parseR16` is already pinned byte-for-byte against fixtures, so the row needs no reader of its own. `sequence` is lifted out so a reading can be ordered and gap-checked in SQL without unpacking 1,572 bytes a row. ## Retention 36,000 records per device, which is 60 x 600: sixty readings at the strap's own ten-minute session limit, the unit this data arrives in. @meta1971's month on one MG is the arithmetic behind it. 140 readings in 31 days, about 4.5 a day, a median of roughly 3 minutes (182 records), 7 of the 140 reaching the cap. So the same 36,000 rows holds about 198 readings and 44 days of that shape, and still 60 readings and 13 days if every one ran long. At ~1,600 B a row that is a ~58 MB ceiling per device, in the same bracket as the PPG waveform's ~70 MB. A row cap, not a reading cap, and it behaves like one anyway: the sweep deletes the OLDEST rows, so the only reading it can cut in half is the oldest one still in the window. Every newer reading stays whole. That is what the issue's request for "a cap counted in readings" was after, and it needs no reading-grouping machinery. Not an age-based drop. Readings are discrete events, so a wall-clock cutoff empties the table for a user who takes one a week, and a waveform has no aggregate that survives its samples. The sweep budget is 600 records, one full reading, deliberately far tighter in proportion than the PPG table's 10,000: that budget's known restart hole is theoretical at 28,800 rows a night and would be the normal case at a few hundred records a week. ## The archive hand-off, and the order of the two halves Only now does version 16 enter the mapped set, with its archive skip bound to the INTEGRITY VERDICT and not to the version byte, the same way v26's is. Mapping a version takes its records off the unconditional-archive branch, and the strap frees them on our trim ack either way, so a version mapped before its rows had a table would have been freed and kept nowhere. A record that fails the verdict banks no row and is archived like any other broken frame. The skip also asks for the record's SHAPE before the verdict, which is tighter than the v26 clause beside it and is the second re-review finding. `parseR16` accepts exactly one inner length, so a type-47 revision-16 frame of any other length banks no row however intact it is: a 1,484-byte one with both CRCs valid was skipped here AND dropped by the extraction, stored nowhere, its bytes freed by the next trim ack. Before the mapping that frame was archived by the unmapped-layout rule, so the hole arrived with the mapping, which is exactly the outcome this filter exists to prevent. The hole this still shares with v26 is stated in both decoders and pinned by a named test rather than left to be discovered: an intact record whose own @15 fails the #547 plausibility gate banks no row and is not archived either. Closing it needs `rejectedHistoricalRecords` to know a wall clock it has no parameter for, and it would widen the archive for v26's far larger population at the same time, so it belongs in a change that covers both. ## Nothing reads these rows, on purpose #752 bounds what may be read off this signal: no rhythm classification of ours, no alerts, no clinical wording. The rows exist so a later analysis can run over the ORIGINAL samples instead of a summary invented at ingest time. Nothing is named `heart_rate` or `gravity_*` either: `averageHR` is the strap's own average over a reading, so publishing it under the scoring funnel's name would feed a recovery input from a quantity nobody has validated for it. It ships as `ecg_avg_hr`, readable and inert. ## Two downstream readouts the mapping would have made wrong Mapping a version changes what the strap log says about it, and both lines had to move with it. The #1992 line asks the user to report a layout NOOP cannot score from. For a layout NOOP decodes, banks whole and scores nothing from by design, that is a request for a report about working software, so `HistoricalLayoutSupport` gains a fourth case, `decodesIntoItsOwnStream`, and a neutral line. What it gives up is stated on the case: a night made only of these records could not be staged and this case says nothing about that, which is tolerable because such a night does not occur. The Connection-mode firmware line printed "UNMAPPED (no motion/HR decoded)" from a second field list. After the mapping that would have contradicted the line above it in the same session, so both platforms now derive it from the one verdict function. ## Verification HARDWARE-UNVALIDATED, and the BLE path is not what moved: this is the decode and storage path, driven by fixtures and by the shared oracle. What is proven: - the shared decoder oracle gains an R16 frame fixture and a stream batch, so both platforms must produce the SAME single row and the same eight decoded fields from the SAME bytes. The zeros in that batch are the load-bearing half, since they are what fails if either platform routes `averageHR` into the `hr` stream; - the stored bytes are asserted against the FRAME, not against a re-encode of the decode, because a codec that packed decoded samples back into slots could be byte-identical for every sample it decoded and still differ in the padding past `sampleCount`; - the row decodes back through `parseR16` to the same record the frame gives; - two records stamped in one strap-second both reach the store AND both survive it, and a record declaring zero samples is banked rather than skipped, because the row here is the record and its header marks where a reading began; - an intact v16 frame of the wrong length stays in the archive; - retention keeps the newest rows, does not sweep under its budget, resets its budget, is per device, and never evicts another device's rows; - Room's generated v42 export matches the hand-written migration SQL exactly, and schema_oracle.json carries the table for both platforms. WhoopProtocol 857 tests, WhoopStore 634, Android 789 classes / 6743 tests, Tools 185, doc_comment_lint clean, i18n gate clean against origin/main, parity ratchet errors=0. What is NOT covered, said plainly: no MG has offloaded a real R16 record into this table. Every fixture here is synthetic, built from the documented layout, because the two real sanitised fixtures on the issue are measurements of @meta1971's own strap. A first real offload is what proves the ts reading, and that reading is the one inference in the change: @15 is the slot every other 5/MG historical layout carries unix in, and the #547 gate is what turns the inference into a per-record check, so a wrong reading costs the status quo rather than a mis-dated row.
ryanbr
added a commit
that referenced
this pull request
Oct 25, 2026
Refs #2847, #891, #2839, #156, #752. #2839 taught both platforms to decode the record. It had nowhere to put one. Layout version 16 stayed out of `mappedWhoop5HistoricalVersions`, so every offloaded R16 record went down the unmapped route into `RawHistoryArchive`, a 5 MiB forensic ring whose per-version floor protects 64 informative lines. For a record of one strap-second that is 64 seconds of waveform, and @DX23876 measured a reading evicted down to that floor within a couple of hours of ordinary MG traffic, because R20 and R21 records compete for the same file constantly. ## The table `ecgWaveformSample` on both platforms, one row per record: deviceId, ts, sequence, record. GRDB `v48-ecg-waveform`, Room 41 to 42, same column order and the same primary key, so a `.noopbak` round-trips. The key is (deviceId, ts, SEQUENCE), three columns where every per-second stream beside it keys on two, and that is a re-review finding rather than a first draft. R16 carries a `subseconds` field at all, and the opening records of a real run declare 0 samples and then 245 before the first full 500, so two of them can be stamped inside one strap-second. On (deviceId, ts) the second of that pair hit ON CONFLICT DO NOTHING and was dropped with no error anywhere, which measured as 2 rows decoded and 1 row stored. Re-offloading the same record still dedupes, because its ts AND its sequence both repeat; both halves are pinned separately. `record` is the record's INNER bytes verbatim, all 1,572 of them. The wire form rather than a decoded field set, and that is the load-bearing choice: - the waveform is 500 three-byte slots of signed 18-BIT samples, so no integer blob holds one. @meta1971 measured them reaching plus or minus 126,977 over 13,178,493 samples, which settles the width with data rather than a guess; - the 22 lead-off halfwords have no attested byte order, so a decoded column would bake a guess into storage. Verbatim bytes let a later re-reading happen with no re-pull, which is what @meta1971 asked for on the issue; - `Whoop5Ecg.parseR16` is already pinned byte-for-byte against fixtures, so the row needs no reader of its own. `sequence` is lifted out so a reading can be ordered and gap-checked in SQL without unpacking 1,572 bytes a row. ## Retention 36,000 records per device, which is 60 x 600: sixty readings at the strap's own ten-minute session limit, the unit this data arrives in. @meta1971's month on one MG is the arithmetic behind it. 140 readings in 31 days, about 4.5 a day, a median of roughly 3 minutes (182 records), 7 of the 140 reaching the cap. So the same 36,000 rows holds about 198 readings and 44 days of that shape, and still 60 readings and 13 days if every one ran long. At ~1,600 B a row that is a ~58 MB ceiling per device, in the same bracket as the PPG waveform's ~70 MB. A row cap, not a reading cap, and it behaves like one anyway: the sweep deletes the OLDEST rows, so the only reading it can cut in half is the oldest one still in the window. Every newer reading stays whole. That is what the issue's request for "a cap counted in readings" was after, and it needs no reading-grouping machinery. Not an age-based drop. Readings are discrete events, so a wall-clock cutoff empties the table for a user who takes one a week, and a waveform has no aggregate that survives its samples. The sweep budget is 600 records, one full reading, deliberately far tighter in proportion than the PPG table's 10,000: that budget's known restart hole is theoretical at 28,800 rows a night and would be the normal case at a few hundred records a week. ## The archive hand-off, and the order of the two halves Only now does version 16 enter the mapped set, with its archive skip bound to the INTEGRITY VERDICT and not to the version byte, the same way v26's is. Mapping a version takes its records off the unconditional-archive branch, and the strap frees them on our trim ack either way, so a version mapped before its rows had a table would have been freed and kept nowhere. A record that fails the verdict banks no row and is archived like any other broken frame. The skip also asks for the record's SHAPE before the verdict, which is tighter than the v26 clause beside it and is the second re-review finding. `parseR16` accepts exactly one inner length, so a type-47 revision-16 frame of any other length banks no row however intact it is: a 1,484-byte one with both CRCs valid was skipped here AND dropped by the extraction, stored nowhere, its bytes freed by the next trim ack. Before the mapping that frame was archived by the unmapped-layout rule, so the hole arrived with the mapping, which is exactly the outcome this filter exists to prevent. The hole this still shares with v26 is stated in both decoders and pinned by a named test rather than left to be discovered: an intact record whose own @15 fails the #547 plausibility gate banks no row and is not archived either. Closing it needs `rejectedHistoricalRecords` to know a wall clock it has no parameter for, and it would widen the archive for v26's far larger population at the same time, so it belongs in a change that covers both. ## Nothing reads these rows, on purpose #752 bounds what may be read off this signal: no rhythm classification of ours, no alerts, no clinical wording. The rows exist so a later analysis can run over the ORIGINAL samples instead of a summary invented at ingest time. Nothing is named `heart_rate` or `gravity_*` either: `averageHR` is the strap's own average over a reading, so publishing it under the scoring funnel's name would feed a recovery input from a quantity nobody has validated for it. It ships as `ecg_avg_hr`, readable and inert. ## Two downstream readouts the mapping would have made wrong Mapping a version changes what the strap log says about it, and both lines had to move with it. The #1992 line asks the user to report a layout NOOP cannot score from. For a layout NOOP decodes, banks whole and scores nothing from by design, that is a request for a report about working software, so `HistoricalLayoutSupport` gains a fourth case, `decodesIntoItsOwnStream`, and a neutral line. What it gives up is stated on the case: a night made only of these records could not be staged and this case says nothing about that, which is tolerable because such a night does not occur. The Connection-mode firmware line printed "UNMAPPED (no motion/HR decoded)" from a second field list. After the mapping that would have contradicted the line above it in the same session, so both platforms now derive it from the one verdict function. ## Verification HARDWARE-UNVALIDATED, and the BLE path is not what moved: this is the decode and storage path, driven by fixtures and by the shared oracle. What is proven: - the shared decoder oracle gains an R16 frame fixture and a stream batch, so both platforms must produce the SAME single row and the same eight decoded fields from the SAME bytes. The zeros in that batch are the load-bearing half, since they are what fails if either platform routes `averageHR` into the `hr` stream; - the stored bytes are asserted against the FRAME, not against a re-encode of the decode, because a codec that packed decoded samples back into slots could be byte-identical for every sample it decoded and still differ in the padding past `sampleCount`; - the row decodes back through `parseR16` to the same record the frame gives; - two records stamped in one strap-second both reach the store AND both survive it, and a record declaring zero samples is banked rather than skipped, because the row here is the record and its header marks where a reading began; - an intact v16 frame of the wrong length stays in the archive; - retention keeps the newest rows, does not sweep under its budget, resets its budget, is per device, and never evicts another device's rows; - Room's generated v42 export matches the hand-written migration SQL exactly, and schema_oracle.json carries the table for both platforms. WhoopProtocol 857 tests, WhoopStore 634, Android 789 classes / 6743 tests, Tools 185, doc_comment_lint clean, i18n gate clean against origin/main, parity ratchet errors=0. What is NOT covered, said plainly: no MG has offloaded a real R16 record into this table. Every fixture here is synthetic, built from the documented layout, because the two real sanitised fixtures on the issue are measurements of @meta1971's own strap. A first real offload is what proves the ts reading, and that reading is the one inference in the change: @15 is the slot every other 5/MG historical layout carries unix in, and the #547 gate is what turns the inference into a per-record check, so a wrong reading costs the status quo rather than a mis-dated row.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #154 — reported next on Vitality.
The bug
Same contradiction #154 fixed for Fitness Age: the Today card shows a Vitality value, but tapping through shows "Not enough history yet." #154 special-cased
key == "fitness_age", so Vitality (and any other metric sitting at one reading) still fell to the generic note.Fix
Generalize the single-reading value display from Fitness-Age-only to any metric:
size == 1(any key) → show the value ("Latest N · as of ") + generic "One reading so far — your trend chart fills in once a second reading lands."size == 1→ "Your latest reading — trend to follow").size == 0, non-fitness) still shows the honest "Not enough history yet" note.This matches iOS, whose
MetricExplorerViewalready renders the value hero for any metric at>= 1point — so it's Android catching up, and it stops the whack-a-mole (Fitness Age → Vitality → whatever's next).Compiles clean. UI-state gate, so verification is visual. iOS unchanged — already correct.