Repository navigation
docs(whoop5): pin dynamic_acceleration and record that byte 43 is not respiration (#520) - #812
Merged
Merged
Conversation
… respiration (#520) A third-party decoder surfaced in #520 reads a u8 at payload offset 35 — frame offset 43 — and labels it a raw respiration rate. It is the third byte of the little-endian dynamic_acceleration float32 at 41. Disproved against the six real v18 frames already in decoder_oracle.json. The three f32s at 45/49/53 give |g| = 0.9985..1.0106 on every frame, including off-wrist; three independent floats cannot land on unit magnitude at a wrong offset, so the float grid is certain and 41..44 is the preceding float. It reads 0.006..0.033 g, right for a resting gravity-removed magnitude. Byte 43 as a rate gives 6..195 "brpm", worst on the off-wrist frame — 195 breaths/min from a strap nobody is wearing, with heart_rate@22 = 0. Adds dynamic_acceleration to the shared oracle fixture so both platforms assert it. No test code needed: the Swift default branch and the Kotlin is-Double branch already handle arbitrary double keys within tolerance (1e-4; values are rounded to 6dp, so max error 5e-7), and testOracleCopiesAreIdentical keeps the two copies in sync. A future offset change here now fails a test instead of silently reintroducing a fabricated vital sign. Also adds the @41 row to the v18 table. The field has been decoded on both platforms all along but was absent from the spec, which is part of why an outside reader could mistake one of its bytes for a channel of its own. Documented as decoded-but-unused: step_motion_counter@57 and activity_class@63 are what the motion paths consume. Docs and fixture only. No decoder change, no schema, no scores, no strings.
The previous wording said the gravity triplet makes the 4-byte grid certain "which makes 41-44 the preceding float". That inference is too strong: sitting on the grid doesn't make a span a float, and 41-44 could in principle be four independent bytes. Having flagged the mantissa tautology two paragraphs down, the argument should hold itself to the same standard. Restated as the three things actually observed: the grid is fixed by |g|; read as an f32 on that grid 41-44 is physically coherent across worn, off-wrist and two straps; read as a rate byte 43 is impossible. A byte belongs to one field, so the coherent reading wins and the impossible one is refuted by its own values. Same conclusion, no new claims — only the step that didn't follow.
…rroboration (#715) Two findings from reviewing a9eelsh/whoop-local, neither of which changes any code. Byte 82. That project reads it as WHOOP's own computed SpO2 percentage, emitted only during sleep, attributing it to hardware RE rather than app decompilation. It fits something already in our table and never explained: aux_byte_82 is nonzero only while sleep_state = asleep, which is exactly what a sleep-only channel looks like. It cannot be confirmed here. All 21 v18 frames in the tree — six oracle frames plus fifteen across the Swift and Kotlin suites — are sleep_state = 0 (awake) and all 21 carry byte 82 = 0. Consistent with the reading, and with several others; absence while awake distinguishes nothing. The value range in one asleep record settles it: 90-100 is a percentage, 0-3 is a state code. Recorded as an open hypothesis so the byte stays raw and unnamed until then, and so the next reader does not have to rediscover the lead. v20 optical. The same project lists six 20-bit channel offsets derived separately: frame 47, 247, 1313, 1513, 1735, 1935. Those are exactly the six slots of our non-empty blocks (0, 3, 4 — the corpus shows counts are always [25, 0, 0, 25, 25]) out of the ten this document already lists. Two unrelated methods agreeing on which six slots carry data, and where, is the strongest confirmation the layout has. Stated explicitly in the doc: that agreement is about STRUCTURE. Neither project has a labelled capture, so the v20 sensor-identity question is untouched — knowing where the bytes are is not evidence about what produced them. Docs only. No decoder change, no schema, no scores, no strings.
ryanbr
force-pushed
the
docs/byte43-not-respiration
branch
from
July 26, 2026 02:16
8793a20 to
f82bc5d
Compare
The note said "all 21 v18 frames in the tree". That counted OCCURRENCES. There are six distinct v18 frames, the ones in decoder_oracle.json, from two straps; the Swift and Kotlin suites share fixtures, so each recurs in 2-7 files and a naive grep returns 21. The conclusion is unchanged — every frame we have is awake with byte 82 = 0, so the SpO2 identification still cannot be tested here. But 21 independent frames would be a reasonable sample and six from two straps is not, and the note existed to be honest about exactly that. Restated, with the 3.5x grep inflation called out so the next person recounting does not reach the same wrong number.
The note asked for an asleep capture, but the answer is probably already on disk somewhere. The original "nonzero only while asleep" observation cannot have come from the six fixtures — every one of them is awake. It came from the out-of-tree ~258k-record corpus this document cites twice elsewhere, once for hr_fixed_8_8@36 and once for status_word@75 "occurring as often awake as asleep" — which means that corpus contains asleep records. So whoever holds it can resolve this with one query: the distribution of byte 82 where sleep_state != 0. 90-100 confirms a percentage, 0-3 a state code. No new capture, no waiting on a third party. Worth writing down because the note as it stood pointed at the slowest route while a faster one was implied by two other paragraphs in the same file.
Swept their decoder against every v18 byte we still carry as raw, rather than only the ones they advertise in their README. Three results. @40 gets a name. They decode it as `signal_quality`. Plausible for a byte sitting beside the HR fields, but it appears only in their code — no README entry, no source tag, no supporting analysis — so it stays raw here and is recorded as a lead rather than adopted. Noting it so the next person looking at that byte starts with a hypothesis instead of nothing. @63 is corroborated. They decode the same offset as an activity class with the same `<= 2` gate we use (#316), reached independently. That is a second source on a field we had from one. The SpO2 gate is evidence. Their code clamps byte 82 to 70..100 and drops anything outside instead of storing it unconditionally. Code shaped that way implies its author saw values inside the band and values outside it worth rejecting — which is what a real percentage with invalid samples looks like, and not what a 0-3 state code looks like. Recorded as suggestive, not decisive: a gate can also be purely defensive. Nothing adopted, nothing decoded. Their remaining unmatched offsets checked out as coincidental number collisions, not claims. Deliberately NOT touching the Credits table: it documents projects whose findings this codebase PORTS, and we have ported nothing from whoop-local — the inline #715 citations are the accurate form of credit here.
ryanbr
added a commit
that referenced
this pull request
Jul 26, 2026
…ded to run (#103) (#814) Carries @dnesdan's #808 harness. The design is theirs and is unchanged: nightly in-band @82 mean while asleep, paired against the export's blood_oxygen_pct, Pearson r / MAE / bias, an offset-specificity scan over 74-92, checklist gates, --batch for multiple devices, and a --postable summary that omits raw SpO2 values. That is the right instrument for the #103 bar. The rest of this change is about the ground it landed on. THE SUITE COULD NOT RUN ON LINUX Tools/linux-capture/ and tools/linux-capture/ were two case-variant directories with ZERO overlapping filenames — 11 and 17 files. On macOS they are one directory and everything works; on Linux they are separate and every cross-boundary import fails. validate_spo2_candidate.py imports whoop_activity (upper) and whoop_frame (lower), so it ran in neither. correlate_ground_truth.py was already broken this way on main. Consolidated into tools/linux-capture/ — the side holding whoop_frame.py, the README and most of the scripts, and the path the docs cite. Repo-root Tools/ is untouched. before: Tools/linux-capture Ran 2 FAILED (errors=2) tools/linux-capture Ran 104 OK after: tools/linux-capture Ran 159 OK (skipped=1) "21 TESTS, OK" NEVER INCLUDED THE NEW TEST test_whoop_activity.py is pytest-style — 21 bare def test_* functions, no unittest.TestCase — so python3 -m unittest, the command the README documents and the only runner in requirements.txt, collects nothing from it and exits 0. That is indistinguishable from passing. The 21 was 8 + 13 from the other two files; the new @82 tri-mode decoder test sat in the file contributing zero and had never executed. Added unittest's load_tests hook rather than rewriting 21 working tests. Running them surfaced 4 real failures, all pytest-only dependencies: three took the tmp_path fixture, one called pytest.skip. Rewritten to stdlib (tempfile.TemporaryDirectory, unittest.SkipTest), both honoured by pytest too, so all 21 run under either runner. BYTE-82 DOCS FROM #812 WERE WRONG #812 documented @82 as an unexplained raw byte with a promising third-party lead. It is already decoded on both platforms as spo2_candidate_82 — tri-mode, sleep-only, with a guard test stopping it reaching spo2Pct and a full write-up in WHOOP5_DEEP_DATA.md. The error came from grepping aux_byte_82, reading a table row that omits the candidate, and never grepping spo2. Rewritten to the real position: the identification is not the open question, the split evidence is — 8 nights at corr +0.99 including a 92% desat and a 98% high, offset-specific in a 74-92 scan, against two nights on the #103 device that moved OPPOSITE to the app value. Resolving that contradiction is exactly what this harness does, which is a better argument for it than the one #808 made for itself. Also repoints five WHOOP5_DEEP_DATA.md paths the directory move broke, including a link that would 404 and two copy-paste commands that fail on a case-sensitive filesystem. Python tooling and docs only: no app code, no BLE path, no schema, no scores, no strings. Verified on Linux, which is the platform that mattered here — 159 tests, OK. Known gap: no workflow runs this suite, so nothing stops it silently breaking again. Co-authored-by: Dan Urbanek <abc@dnesdan.cz>
This was referenced Jul 26, 2026
simoncad7
pushed a commit
to simoncad7/noop
that referenced
this pull request
Aug 17, 2026
…ded to run (ryanbr#103) (ryanbr#814) Carries @dnesdan's ryanbr#808 harness. The design is theirs and is unchanged: nightly in-band @82 mean while asleep, paired against the export's blood_oxygen_pct, Pearson r / MAE / bias, an offset-specificity scan over 74-92, checklist gates, --batch for multiple devices, and a --postable summary that omits raw SpO2 values. That is the right instrument for the ryanbr#103 bar. The rest of this change is about the ground it landed on. THE SUITE COULD NOT RUN ON LINUX Tools/linux-capture/ and tools/linux-capture/ were two case-variant directories with ZERO overlapping filenames — 11 and 17 files. On macOS they are one directory and everything works; on Linux they are separate and every cross-boundary import fails. validate_spo2_candidate.py imports whoop_activity (upper) and whoop_frame (lower), so it ran in neither. correlate_ground_truth.py was already broken this way on main. Consolidated into tools/linux-capture/ — the side holding whoop_frame.py, the README and most of the scripts, and the path the docs cite. Repo-root Tools/ is untouched. before: Tools/linux-capture Ran 2 FAILED (errors=2) tools/linux-capture Ran 104 OK after: tools/linux-capture Ran 159 OK (skipped=1) "21 TESTS, OK" NEVER INCLUDED THE NEW TEST test_whoop_activity.py is pytest-style — 21 bare def test_* functions, no unittest.TestCase — so python3 -m unittest, the command the README documents and the only runner in requirements.txt, collects nothing from it and exits 0. That is indistinguishable from passing. The 21 was 8 + 13 from the other two files; the new @82 tri-mode decoder test sat in the file contributing zero and had never executed. Added unittest's load_tests hook rather than rewriting 21 working tests. Running them surfaced 4 real failures, all pytest-only dependencies: three took the tmp_path fixture, one called pytest.skip. Rewritten to stdlib (tempfile.TemporaryDirectory, unittest.SkipTest), both honoured by pytest too, so all 21 run under either runner. BYTE-82 DOCS FROM ryanbr#812 WERE WRONG ryanbr#812 documented @82 as an unexplained raw byte with a promising third-party lead. It is already decoded on both platforms as spo2_candidate_82 — tri-mode, sleep-only, with a guard test stopping it reaching spo2Pct and a full write-up in WHOOP5_DEEP_DATA.md. The error came from grepping aux_byte_82, reading a table row that omits the candidate, and never grepping spo2. Rewritten to the real position: the identification is not the open question, the split evidence is — 8 nights at corr +0.99 including a 92% desat and a 98% high, offset-specific in a 74-92 scan, against two nights on the ryanbr#103 device that moved OPPOSITE to the app value. Resolving that contradiction is exactly what this harness does, which is a better argument for it than the one ryanbr#808 made for itself. Also repoints five WHOOP5_DEEP_DATA.md paths the directory move broke, including a link that would 404 and two copy-paste commands that fail on a case-sensitive filesystem. Python tooling and docs only: no app code, no BLE path, no schema, no scores, no strings. Verified on Linux, which is the platform that mattered here — 159 tests, OK. Known gap: no workflow runs this suite, so nothing stops it silently breaking again. Co-authored-by: Dan Urbanek <abc@dnesdan.cz>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The third-party decoder surfaced in #520 reads a
u8at payload offset 35 — frame offset 43 — and labels it a raw respiration rate. It's the third byte of the little-endiandynamic_accelerationfloat32 at 41.Disproved against the six real v18 frames already sitting in
decoder_oracle.json:whoop5_v18_real_wornwhoop5_v18_real_one_rrwhoop5_v18_real_offwristwhoop5_v18_real_ack_capturewhoop5_v18_real_device2_hr57whoop5_v18_real_device2_hr63The three f32s at 45/49/53 give |g| = 0.9985…1.0106 on every frame including off-wrist. Three independent floats can't land on unit magnitude at a wrong offset, so the float grid there is certain, which makes 41–44 the preceding float — and it reads 0.006–0.033 g, right for a resting gravity-removed magnitude. Reading byte 43 as a rate instead gives 6…195 "brpm", worst on the off-wrist frame: 195 breaths/min from a strap nobody is wearing, with
heart_rate@22= 0.Worth stating the tautology to avoid, since it looks like proof and isn't: byte 43 is byte 2 of that float by construction, so reproducing it from the exponent/mantissa bits shows nothing. The evidence is the gravity anchor plus the off-wrist value.
The fixture change is the whole test
dynamic_accelerationgoes into the shared Swift↔Kotlin oracle, so both platforms assert it with no new test code — Swift'sdefaultbranch and Kotlin'sis Doublebranch already handle arbitrary double keys withintolerance. Expected values are computed from each fixture's own bytes and rounded to 6dp, so max error is 5e-7 against a 1e-4 tolerance, andtestOracleCopiesAreIdenticalstops either platform editing its copy alone.A future offset change here now fails a test instead of silently reintroducing a fabricated vital sign.
Also:
@41was missing from the specThe field has been decoded on both platforms all along (
Interpreter.swift:414,HistoricalStreams.kt:305) but had no row in the v18 table — part of why an outside reader could mistake one of its bytes for a channel of its own. Now documented, including that it's decoded-but-unused:step_motion_counter@57andactivity_class@63are what the motion paths actually consume.The related claim is also settled in the doc: WHOOP 5 v18 has no raw respiration channel.
respRateRawOff = 80is set on the 4.0HIST_V24layout only, andAnalyticsEnginealready notes "WHOOP5 v18 carries no raw resp ADC" where it derives the rate from RSA.Verification
Docs and fixture only — no decoder change, no schema, no scores, no strings. JSON parses, both copies byte-identical, i18n audit clean. Neither oracle test was executed locally (no Swift toolchain, no JDK here); the Swift half runs under
swift-packages, the Kotlin half doesn't whileandroid.ymlis disabled.Leaves #520 open — the v20 optical question it raised is unrelated and still open.