fix(ble): read GET_DATA_RANGE newest at any offset + reject future words so auto-sync can't stall by tanarchytan · Pull Request #286 · ryanbr/noop

tanarchytan · 2026-07-11T16:38:07Z

Problem

On a WHOOP 4.0, automatic (periodic) sync silently stops — data only advances on connect or a manual "Sync now". The strap-log shows:

Backfill: not auto-continuing (#1012) — the strap-reported newest banked record reads 12482h AHEAD of the wall clock … (#928) and skipped (PERIODIC) … clock future-dated

…even though the strap's clock is fine and its records are dated today.

Root cause

dataRangeNewestUnix scans 4-byte words aligned from offset 7. On WHOOP 4 the strap's newest-record u32 sits at byte offset 8 — off that grid — so the scan straddles it and returns null, leaving a stale/garbage word latched in strapNewestTs.

Decoding three real captured GET_DATA_RANGE frames (the #451 — for offset analysis log) makes it concrete — the true newest is right there at offset 8, dated today:

frame	newest @ offset 8
`aa100057305d22009968526a…`	1783785625 → 2026-07-11 16:00:25
`aa10005730612200a268526a…`	1783785634 → 2026-07-11 16:00:34
`aa100057307c2200e768526a…`	1783785703 → 2026-07-11 16:01:43

The old aligned scan returns null on all three.

This was harmless until #228 wired strapNewestTs into BackfillPolicy: a future-reading value now makes isFutureDatedNewest skip the automatic periodic/strap offload entirely, so a single bad read stalls auto-sync.

Fix

Scan every byte offset, and prefer the newest word at or behind the wall clock — a device cannot record the future, so a lone future-dated straddle is a decode artifact, not the frontier. Only when every plausible word is future — a genuinely future-set RTC (#928) — return the future max, so isFutureDatedNewest still fires and the auto-continue / periodic guards correctly refuse the range.

Family-agnostic: it repairs the WHOOP 4 offset and hardens 5/MG regardless of their frame layout, and cannot regress a frame the old aligned scan already read.

Relates to #928 / the #451 offset-analysis logging; removes the false-future-date path that also underlies #266.

Test plan

./gradlew :app:testFullDebugUnitTest --tests com.noop.ble.DataRangeScanTest — new test pins three real 4.0 frames (newest = today, previously missed) + future-straddle rejected + genuine-future preserved + within-48h-skew present. Green.
Full com.noop.ble.* suite green.
Swift twin (BLEManager.dataRangeNewestUnix) changed identically for the parity contract, but NOT compiled here (needs macOS/Xcode) — please verify on the app-build leg before merge.
On-strap (recommended): on a 4.0, confirm the clock future-dated skip line disappears and periodic sync returns to ~15 min; capture one 5.0/MG GET_DATA_RANGE frame to confirm the read there too.

…rds so auto-sync can't stall dataRangeNewestUnix scanned 4-byte words aligned FROM offset 7. On WHOOP 4 the strap's newest-record u32 sits at byte offset 8, off that grid, so the scan straddled it and returned null — leaving a stale/garbage word latched in strapNewestTs. That was harmless until ryanbr#228 fed strapNewestTs into BackfillPolicy: a future-reading value now makes isFutureDatedNewest skip the automatic periodic/strap offload entirely, so one bad read stalls automatic sync (observed: a 4.0 whose newest record was in fact today read ~520 days future and stopped auto-syncing; manual "Sync now" still worked). Scan every byte offset instead, and prefer the newest word at or behind the wall clock (a device cannot record the future, so a lone future-dated straddle is a decode artifact, not the frontier). Only when EVERY plausible word is future — a genuinely future-set RTC (ryanbr#928) — return the future max, so isFutureDatedNewest still fires and the auto-continue / periodic guards correctly refuse the range. Family-agnostic: repairs the WHOOP 4 offset and hardens 5/MG regardless of their frame layout; cannot regress a frame the old aligned scan already read. New DataRangeScanTest pins three of David's real 4.0 GET_DATA_RANGE frames (newest = today, previously missed) plus the future-straddle and genuine-future cases. Swift twin (BLEManager.dataRangeNewestUnix) changed identically for parity; NOT compiled here (needs macOS/Xcode) — verify before merge.

tanarchytan · 2026-07-11T16:55:39Z

On-strap validation ✅ (WHOOP 4.0)

Built latest main + this fix, installed on a real WHOOP 4.0 that was exhibiting the future-dated read, and triggered a sync. The false future-date path is gone and the newest reads correctly.

Before (same band, pre-fix): not auto-continuing (#1012) — the strap-reported newest banked record reads 12482h AHEAD … clock future-dated → periodic offload skipped.

After (WhoopBleClient log, live):

Strap newest banked record: 2026-07-11 18:51 (from data range)
Backfill: 10 day(s) banked; newest=2026-07-11, dashboard-today=2026-07-11 (present)
Backfill: session ended — reason=HISTORY_COMPLETE

Newest now reads today / the current minute (was null → stale future word before).
Zero AHEAD / clock future-dated / not auto-continuing lines across the whole session.
Offloads complete normally; the periodic path is no longer skipped.

Decoding the three live #451 frames captured during the sync, with the shipped function:

aa10005730b922009874526a…  -> 1783788696  2026-07-11 18:51  (+0.0h vs now)
aa10005730be2200a174526a…  -> 1783788705  2026-07-11 18:51  (+0.0h vs now)
aa10005730c12200d674526a…  -> 1783788758  2026-07-11 18:52  (+0.0h vs now)

Newest lands on the wall clock, ±0h — exactly as expected.

Remaining (optional, low-risk since the fix is offset-agnostic): a 5/MG GET_DATA_RANGE capture to confirm the read there too. The Swift twin still needs a macOS compile on the app-build leg.

tanarchytan · 2026-07-11T16:59:42Z

5/MG confirmed too ✅

Ran the same on a WHOOP 5/MG (WHOOP 5/MG: clock synced in the log, puffin framing). The 5/MG GET_DATA_RANGE response is a different shape from the 4.0 — 32 bytes with the puffin envelope vs the 4.0's 20 — and the offset-agnostic scan reads it correctly:

4.0 format  (20 B): aa10005730f92200ff75526a…        -> 2026-07-11 (today, ±0h)
5/MG puffin (32 B): aa011800010022e128022276526a3333… -> 2026-07-11 (today, ±0h)

Live log: Strap newest banked record: 2026-07-11 18:58 (from data range), no AHEAD / clock future-dated lines on either strap.

So the fix is validated on both families on real hardware. Only open item is the Swift twin's macOS compile on the app-build leg.

…latforms CI-test the parity (#286 follow-up) (#307) dataRangeNewestUnix decodes strapNewestTs, which gates automatic sync (isFutureDatedNewest -> BackfillPolicy), so it must stay byte-identical Swift/ Kotlin. #286 left it inline in app-target BLEManager/WhoopBleClient with a Kotlin test only — a Swift-side edit could silently drift the parity (the same gap the Oura #306 follow-up closed). Move the pure scan into WhoopProtocol.DataRange (Swift, builds+tests on Linux) and com.noop.protocol.DataRange (Kotlin), byte-identical twins. BLEManager and WhoopBleClient keep their signatures as thin wrappers that pass their own futureSkew constant, so all call sites + the existing DataRangeScanTest are unchanged. Adds DataRangeTests (Swift) mirroring the Kotlin fixtures — the real WHOOP 4.0 offset-8 frames + the future/skew cases. WhoopProtocol swift test 259/259 incl. 4 new DataRangeTests (run on Linux); Android compileFullDebugKotlin + DataRangeScanTest green. BLEManager compile: app-build.

…he aligned-scan protection (#286 follow-up) (#308) Second sibling in the GET_DATA_RANGE parser. Like dataRangeNewestUnix (extracted in #307), dataRangeOldestUnix was inline in app-target BLEManager/WhoopBleClient with NO test on either platform — a parity-critical decoder (it reports the banked span = backlog DEPTH a deep oldest-first drain must cover, #364). Re-review of #307 surfaced that the oldest read must NOT be "made consistent" with the newest by scanning every offset. The minimum is fragile where the maximum is not: a real WHOOP 4.0 frame (aa10..2263) carries a spurious plausible straddle at byte offset 6 (1754857506, ~11 months BEFORE the true newest at offset 8). An any-offset MIN would grab it and report a bogus ~11-month backlog; the aligned- from-7 grid skips it. So this extracts the aligned scan VERBATIM (no behaviour change) into WhoopProtocol.DataRange.oldestUnix + com.noop.protocol twin, keeps the app-target methods as thin wrappers, and adds Swift+Kotlin tests that PIN the protection (real frame -> nil; grid-aligned words -> min) so a future refactor can't quietly reintroduce the false-backlog bug. WhoopProtocol swift test 262/262 (+3 oldest); Android compileFullDebugKotlin + DataRangeScanTest green. BLEManager compile: app-build.

…rds so auto-sync can't stall (ryanbr#286) dataRangeNewestUnix scanned 4-byte words aligned FROM offset 7. On WHOOP 4 the strap's newest-record u32 sits at byte offset 8, off that grid, so the scan straddled it and returned null — leaving a stale/garbage word latched in strapNewestTs. That was harmless until ryanbr#228 fed strapNewestTs into BackfillPolicy: a future-reading value now makes isFutureDatedNewest skip the automatic periodic/strap offload entirely, so one bad read stalls automatic sync (observed: a 4.0 whose newest record was in fact today read ~520 days future and stopped auto-syncing; manual "Sync now" still worked). Scan every byte offset instead, and prefer the newest word at or behind the wall clock (a device cannot record the future, so a lone future-dated straddle is a decode artifact, not the frontier). Only when EVERY plausible word is future — a genuinely future-set RTC (ryanbr#928) — return the future max, so isFutureDatedNewest still fires and the auto-continue / periodic guards correctly refuse the range. Family-agnostic: repairs the WHOOP 4 offset and hardens 5/MG regardless of their frame layout; cannot regress a frame the old aligned scan already read. New DataRangeScanTest pins three of David's real 4.0 GET_DATA_RANGE frames (newest = today, previously missed) plus the future-straddle and genuine-future cases. Swift twin (BLEManager.dataRangeNewestUnix) changed identically for parity; NOT compiled here (needs macOS/Xcode) — verify before merge.

…latforms CI-test the parity (ryanbr#286 follow-up) (ryanbr#307) dataRangeNewestUnix decodes strapNewestTs, which gates automatic sync (isFutureDatedNewest -> BackfillPolicy), so it must stay byte-identical Swift/ Kotlin. ryanbr#286 left it inline in app-target BLEManager/WhoopBleClient with a Kotlin test only — a Swift-side edit could silently drift the parity (the same gap the Oura ryanbr#306 follow-up closed). Move the pure scan into WhoopProtocol.DataRange (Swift, builds+tests on Linux) and com.noop.protocol.DataRange (Kotlin), byte-identical twins. BLEManager and WhoopBleClient keep their signatures as thin wrappers that pass their own futureSkew constant, so all call sites + the existing DataRangeScanTest are unchanged. Adds DataRangeTests (Swift) mirroring the Kotlin fixtures — the real WHOOP 4.0 offset-8 frames + the future/skew cases. WhoopProtocol swift test 259/259 incl. 4 new DataRangeTests (run on Linux); Android compileFullDebugKotlin + DataRangeScanTest green. BLEManager compile: app-build.

…he aligned-scan protection (ryanbr#286 follow-up) (ryanbr#308) Second sibling in the GET_DATA_RANGE parser. Like dataRangeNewestUnix (extracted in ryanbr#307), dataRangeOldestUnix was inline in app-target BLEManager/WhoopBleClient with NO test on either platform — a parity-critical decoder (it reports the banked span = backlog DEPTH a deep oldest-first drain must cover, ryanbr#364). Re-review of ryanbr#307 surfaced that the oldest read must NOT be "made consistent" with the newest by scanning every offset. The minimum is fragile where the maximum is not: a real WHOOP 4.0 frame (aa10..2263) carries a spurious plausible straddle at byte offset 6 (1754857506, ~11 months BEFORE the true newest at offset 8). An any-offset MIN would grab it and report a bogus ~11-month backlog; the aligned- from-7 grid skips it. So this extracts the aligned scan VERBATIM (no behaviour change) into WhoopProtocol.DataRange.oldestUnix + com.noop.protocol twin, keeps the app-target methods as thin wrappers, and adds Swift+Kotlin tests that PIN the protection (real frame -> nil; grid-aligned words -> min) so a future refactor can't quietly reintroduce the false-backlog bug. WhoopProtocol swift test 262/262 (+3 oldest); Android compileFullDebugKotlin + DataRangeScanTest green. BLEManager compile: app-build.

…rds so auto-sync can't stall (ryanbr#286) dataRangeNewestUnix scanned 4-byte words aligned FROM offset 7. On WHOOP 4 the strap's newest-record u32 sits at byte offset 8, off that grid, so the scan straddled it and returned null — leaving a stale/garbage word latched in strapNewestTs. That was harmless until ryanbr#228 fed strapNewestTs into BackfillPolicy: a future-reading value now makes isFutureDatedNewest skip the automatic periodic/strap offload entirely, so one bad read stalls automatic sync (observed: a 4.0 whose newest record was in fact today read ~520 days future and stopped auto-syncing; manual "Sync now" still worked). Scan every byte offset instead, and prefer the newest word at or behind the wall clock (a device cannot record the future, so a lone future-dated straddle is a decode artifact, not the frontier). Only when EVERY plausible word is future — a genuinely future-set RTC (ryanbr#928) — return the future max, so isFutureDatedNewest still fires and the auto-continue / periodic guards correctly refuse the range. Family-agnostic: repairs the WHOOP 4 offset and hardens 5/MG regardless of their frame layout; cannot regress a frame the old aligned scan already read. New DataRangeScanTest pins three of David's real 4.0 GET_DATA_RANGE frames (newest = today, previously missed) plus the future-straddle and genuine-future cases. Swift twin (BLEManager.dataRangeNewestUnix) changed identically for parity; NOT compiled here (needs macOS/Xcode) — verify before merge.

…latforms CI-test the parity (ryanbr#286 follow-up) (ryanbr#307) dataRangeNewestUnix decodes strapNewestTs, which gates automatic sync (isFutureDatedNewest -> BackfillPolicy), so it must stay byte-identical Swift/ Kotlin. ryanbr#286 left it inline in app-target BLEManager/WhoopBleClient with a Kotlin test only — a Swift-side edit could silently drift the parity (the same gap the Oura ryanbr#306 follow-up closed). Move the pure scan into WhoopProtocol.DataRange (Swift, builds+tests on Linux) and com.noop.protocol.DataRange (Kotlin), byte-identical twins. BLEManager and WhoopBleClient keep their signatures as thin wrappers that pass their own futureSkew constant, so all call sites + the existing DataRangeScanTest are unchanged. Adds DataRangeTests (Swift) mirroring the Kotlin fixtures — the real WHOOP 4.0 offset-8 frames + the future/skew cases. WhoopProtocol swift test 259/259 incl. 4 new DataRangeTests (run on Linux); Android compileFullDebugKotlin + DataRangeScanTest green. BLEManager compile: app-build.

…he aligned-scan protection (ryanbr#286 follow-up) (ryanbr#308) Second sibling in the GET_DATA_RANGE parser. Like dataRangeNewestUnix (extracted in ryanbr#307), dataRangeOldestUnix was inline in app-target BLEManager/WhoopBleClient with NO test on either platform — a parity-critical decoder (it reports the banked span = backlog DEPTH a deep oldest-first drain must cover, ryanbr#364). Re-review of ryanbr#307 surfaced that the oldest read must NOT be "made consistent" with the newest by scanning every offset. The minimum is fragile where the maximum is not: a real WHOOP 4.0 frame (aa10..2263) carries a spurious plausible straddle at byte offset 6 (1754857506, ~11 months BEFORE the true newest at offset 8). An any-offset MIN would grab it and report a bogus ~11-month backlog; the aligned- from-7 grid skips it. So this extracts the aligned scan VERBATIM (no behaviour change) into WhoopProtocol.DataRange.oldestUnix + com.noop.protocol twin, keeps the app-target methods as thin wrappers, and adds Swift+Kotlin tests that PIN the protection (real frame -> nil; grid-aligned words -> min) so a future refactor can't quietly reintroduce the false-backlog bug. WhoopProtocol swift test 262/262 (+3 oldest); Android compileFullDebugKotlin + DataRangeScanTest green. BLEManager compile: app-build.

ryanbr merged commit f8f1013 into ryanbr:main Jul 12, 2026

ryanbr mentioned this pull request Jul 12, 2026

ble: extract GET_DATA_RANGE newest-parse into WhoopProtocol so both platforms CI-test the parity (#286 follow-up) #307

Merged

tanarchytan deleted the fix/datarange-newest-offset branch July 14, 2026 12:22

Navigation Menu

Repository navigation

fix(ble): read GET_DATA_RANGE newest at any offset + reject future words so auto-sync can't stall - #286

fix(ble): read GET_DATA_RANGE newest at any offset + reject future words so auto-sync can't stall#286
ryanbr merged 1 commit into
ryanbr:mainfrom
tanarchytan:fix/datarange-newest-offset

tanarchytan commented Jul 11, 2026

Uh oh!

tanarchytan commented Jul 11, 2026

Uh oh!

tanarchytan commented Jul 11, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

tanarchytan commented Jul 11, 2026

Problem

Root cause

Fix

Test plan

Uh oh!

tanarchytan commented Jul 11, 2026

On-strap validation ✅ (WHOOP 4.0)

Uh oh!

tanarchytan commented Jul 11, 2026

5/MG confirmed too ✅

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants