Repository navigation
Merge pull request #3 from OpenStrap/fix/background-sync-keepalive - #4
Merged
Merged
Conversation
Fix background sync: hold BLE connection alive + persistent flusher + honest upload count
abdulsaheel
added a commit
that referenced
this pull request
Jun 26, 2026
Merge pull request #3 from OpenStrap/fix/background-sync-keepalive
This was referenced Aug 23, 2026
abdulsaheel
added a commit
that referenced
this pull request
Oct 31, 2026
* feat(compute): per-category calibration policy for the live 100 Hz pedometer
Calibrates the counter that already exists — pedometer()/livePedometer() in
analytics, fed from the live IMU stream and banked as live_coverage source
'strap_100hz'. Nothing here touches BLE or the strap: every value stays on the
phone, so there is nothing a factory reset has to undo.
Two kinds of calibration, and the second is the one that pays:
* POSITIVE (slow/brisk/jog) — the walk produces N real steps and the detector
reports some other number; the correction is a SCALE, gain = truth/raw.
* NEGATIVE (everyday) — carrying, lifting, rearranging: the detector reports a
count and the true step count is ~0. No scale fixes that: 0/raw is 0, and a
gain of 0 would credit nothing everywhere. What the run teaches is a
BOUNDARY (the highest confidence any non-gait run reached), so windows no
more rhythmic than the wearer's own household movement are credited nothing.
The negative run is what addresses OVER-count, the expensive error —
live_step_runs.dart records the owner's own measurement of rhythmic arm work
scored as gait: +199.5%, 217 true steps read as 650.
No prior weight, deliberately deviating from lib/data/step_calibration.dart:
that prior guards an INFERRED reference (a phone alongside the band), while this
one is SUPERVISED (the wearer counts their own steps). Admissibility and the
clamp already bound a bad run, and blending 3 pseudo-runs of 1.0 meant a first
calibration resolved to 0.977 instead of 0.909 — a user who walks their
calibration walk and sees nothing change has been told the feature does not
work. Repeat runs still converge: the estimator is a ratio of SUMS, so later
runs outweigh earlier ones by the steps they carry.
Credit order is the whole policy: the negative probe refuses first (a fabricated
step cannot be recovered, a missed one can), then cadence picks a calibrated
category, then anything uncalibrated is credited EXACTLY as today — the feature
is invisible until the wearer calibrates.
Verified against a Python mirror: 42/42 cases.
* fix(compute): everyday is a low positive scale against the PHONE, not a zero
Corrects the design in the previous commit. Household movement is not zero
steps — the wearer really does walk to the next room — so rejecting the window
outright would delete real steps exactly where a number was asked for.
Both families are now the same maths (gain = truth / raw) and differ only in
their REFERENCE and RANGE:
* gait (slow/brisk/jog): reference is the wearer's own count; range 0.5..2.0.
* everyday: reference is the PHONE's pedometer over the same span, because
nobody counts steps while doing chores; range 0.05..1.0.
The everyday floor is 0.05 precisely so the correction can be large without
ever reaching zero, and an everyday run whose reference counted nothing is
REFUSED (noReferenceSteps) rather than fitted as a 0.0 gain — a phone on a table
is the absence of a reference, not a measured zero.
The everyday reference is the phone because the phone is the better sensor for
this one quantity, which lib/health/phone_pedometer.dart already argues: at the
wrist, arm work out-accelerates walking (stirring ~104 mg, chopping ~139 mg vs
walking ~66 mg ENMO), while a pocket-carried device observes trunk motion.
Driving is NOT this file's job. lib/data/live_coverage_policy.dart already drops
band credit overlapping a phone-confirmed motionless hour unless the span's own
density looks like gait (kConfirmedStillMinSpm = 40, above the measured 22-27 spm
false-positive ceiling and below real walking). The everyday gain is a backstop
on what survives that rule.
Measured on the mirror: a household window of 140 raw steps with a fitted 0.1375
gain credits 19, not 0; a driving-like window at 12 spm credits 3 of 25; an
uncalibrated family credits raw exactly as today.
Verified against the Python mirror: 43/43 cases.
* feat(compute,data): persist the live-pedometer calibration; align its gait band
Persistence (lib/data/db.dart):
* live_step_calibration — the fitted gain per activity family
* live_step_calibration_run — the individual runs behind it
Separate tables from step_calibration/step_calibration_day on purpose: that pair
calibrates the gen5 ON-CHIP counter (a factor on raw ticks, per device family and
wearing), these calibrate the LIVE 100 Hz pedometer (a scale per activity family,
different reference, different range, different version namespace). Sharing a
table would let one model overwrite the other's stored value.
Both tables are registered in the export AND import lists — a calibration cannot
be rebuilt once raw is pruned, so a backup that dropped it would silently lose
the wearer's own work.
Stored rows are read back defensively: an unknown category or reference is
skipped rather than guessed (attaching a run to the wrong family would corrupt
that family's fit with another one's evidence), and a gain is re-checked against
its category's clamp, so a hand-edited or restored row cannot present a number
the fit would never have produced.
BAND FIX. The gait band was 30-300 spm — AN-2554's step-INTERVAL bound, which is
what the counter accepts, not what identifies a minute as walking.
lib/ble/live_cadence.dart already defines that: 60-200 spm, the same band
livePedometer treats as confident gait. Two definitions of gait in one codebase
would let the calibration build bands the session logic disagrees with, so the
constant now matches and a run whose own cadence sits outside the band no longer
defines one — a 55 spm calibration run is not a walking band, and a 40 spm
minute is, in live_cadence.dart's words, a few steps to the kettle.
The pair is redeclared (this file stays dependency-free) and PINNED by
test/step_calibration_band_pin_test.dart in both directions, the same
DERIVED-OR-PINNED convention live_step_runs.dart uses for kRouteTypeKeys.
test/step_calibration_fit_test.dart carries the mirror's cases in flutter_test
form. Verified: 46/46 against the Python mirror. NOT verified: Dart syntax —
neither test file has been run (no Flutter toolchain was available where this was
authored).
* feat(state): bank live runs through the wearer's calibration
Wires the per-category fit into the ONE place the persistence path applies the
step trim — _bankGaitRuns — plus the session checkpoint, which must use the same
function because recovery banks its numbers verbatim.
The invariant is one FACTOR per run, and it is now expressed as one FUNCTION per
path. When a family is calibrated, the calibrated gain REPLACES ana.StepParams.gain
rather than multiplying into it: this file's own history records that applying
the gain at both ends 'shipped a silent x1.23'. When nothing is calibrated the
call falls back to ana.StepParams.gain, so an uncalibrated app banks exactly the
numbers it banks today, to the digit.
The run's cadence comes from the run itself — its raw count over its own sampled
span — which is how lib/ble/live_cadence.dart already defines one, and the RAW
count is what picks the family. Gain last, per that file's 'gate first, gain
last' rule: a gain applied before the cadence test stretches the band by whatever
the gain is.
The calibration is cached per session (the fit reads the whole run history; a
session banks a handful of runs), with invalidateLiveStepCalibration() for the
calibration screen to call after a save.
Verified against the Python mirror: 54/54, including the regression guard that a
calibrated run yields 100 from raw 110 and NOT 122 (double-applied) or 111 (the
fallback multiplied in).
* feat(compute): calibration session — the wearer counts, the band does not terminate
Adds CalibrationSession: one routine's state (start, live band count, phone
reference for everyday, finish -> run-or-refusal).
THE TRAP THIS AVOIDS. The obvious design is 'count the steps as the wearer walks
and stop at 100'. That calibrates nothing: the run would end when the COUNTER
reached 100, rawSteps would be 100 by construction, and every wearer would fit a
gain of exactly 1.0 forever — a number shaped like a calibration and worth
nothing. A reference must be independent of the instrument it calibrates. So the
WEARER counts for the gait families, the PHONE is the reference for everyday, and
the band's live count is feedback on the screen rather than the terminator. The
gap between the two IS the measurement: 100 walked, 112 seen, gain 0.893.
Also corrects a fabrication in the previous commit: peakToPeakG and confidence
were being written as 0.0 when the caller had never measured them. A real still
reading is 0, so a made-up zero is indistinguishable from a measurement at
exactly the end of the scale where the discriminator lives. Both fields are now
nullable (column and model), absent by default, and populated only from a real
PedometerResult; a present-but-NaN value is refused rather than stored.
A silent band is its own outcome (bandSilent), not dressed up as a
StepRunRejection — that enum judges a run that exists, and this one never did.
Relevant context: app_state.dart's v56 note records that a previous GUIDED STEP
CALIBRATION was removed because its only reader was the deleted 1 Hz estimator
and it 'told the user their walk had taught the app something when nothing read
the result'. This one has a reader — _bankGaitRuns and the session checkpoint
both go through creditedStepsForRun — and no screen may claim otherwise until
that path is built.
Verified against the Python mirror: 71/71.
* refactor(compute): session inputs are what a screen can observe
Rewrites CalibrationSession around a cumulative counter and a clock, because
that is all a screen can read: the live session's raw step total
(AppState.liveStrapRawSteps) and wall time. The previous API took completed
chunks, which no caller has — the screen never sees one, so it could not have
been wired up without inventing them.
* start(bandBaseline, nowMs) — everything the routine is credited with is
measured from the counter's value at that moment, so a walk that happened
before the wearer pressed start cannot land in their calibration.
* observe(bandTotal, nowMs) — the counter is never allowed to go backwards:
it resets when a session ends, and a routine spanning a reconnect must not
be credited with a negative correction.
* bandSeconds is the routine's own span. Documented as wall time, not sampled
time, with why that is tolerable: the live path refuses to commit a window
measuring under kMinLiveSampleRateHz, so a stall shows up as the band count
standing still, and a cadence that still lands outside 60-200 spm merely
produces no gait band.
Also adds AppState.liveStrapRawSteps (public) and
LocalDb.bankLiveStepCalibrationRun, which stores a run and its refitted profile
in one call — the two writes must not be done by hand in two places, or the app
ends up correcting with a gain that does not match the evidence behind it.
Notes a doc bug it had to read past: _liveRaw's docstring calls its value
'gain-applied', but _committedRaw holds pre-gain counts and ana.pedometer
returns a raw count. The new public getter is named for what the value is.
Verified against the Python mirror: 71/71.
* feat(ui): step calibration screen, settings row, and the phone reference read
Adds the screen the calibration needs, wired to the reader that already exists.
THE WEARER COUNTS. A gait routine cannot end when the band reaches the target:
rawSteps would be the target by construction and every wearer would fit exactly
1.0 forever. So the screen asks the wearer to count their own steps and shows the
band's live count as FEEDBACK — labelled as the band's count, never as the
wearer's progress. The gap between the two is the number they came for.
The household routine is timed instead, and its reference is the phone. Adds
PhonePedometer.stepsBetween(from, to), because the reference has to be the steps
inside exactly those three minutes, not a day. Null and intervalNotCovered both
mean 'no reference', and finish() refuses the run on either — a phone on a table
measured nothing, which must not be fitted as a measured zero.
The screen claims nothing before the save returns: LocalDb.bankLiveStepCalibrationRun
stores the run and its refitted profile in one call, AppState.invalidateLiveStepCalibration
clears the per-session cache so a walk in progress banks with the new gain, and
only then does the wearer see a message. app_state.dart records that the previous
guided calibration 'told the user their walk had taught the app something when
nothing read the result'; this one must not repeat that, and the reader
(_bankGaitRuns, the session checkpoint) is already wired.
Every refusal names the routine rather than leaving a silent no-op: no phone
reference, band too quiet, routine too short, band counted far more than the
wearer, unreadable sensor data. A silent band is its own message, not a refusal.
English strings are in app_en.arb. The other six locales fall back to English
through the repo's idiom; they are NOT translated yet —
adding six unreviewed translations of 30 user-facing strings would be worse than
showing English until someone does it properly.
NOT COMPILED. This is the largest unverified change in the series: a new Flutter
widget against theme tokens, Material buttons and a provider lookup, none of
which any check here can execute. Verified: JSON validity of the ARB, bracket
balance of every touched file, and the fit/session logic behind it (71/71
against the Python mirror).
* fix(ui): settings row and screen imports
Three guesses I had to correct by reading rather than assuming:
* _SettingsTile does not exist. The row widget is SetRow(icon, color, title,
{sub, value, onTap}) — a positional signature, declared in
lib/ui2/profile/profile.dart.
* detailScaffold / detailLinkRow are in screens/metric_detail.dart and are NOT
re-exported by ui2.dart (that barrel carries the widgets, not the screens),
so the screen imports them explicitly the way advanced.dart does.
* F.h1 and p.accent do not exist either; corrected earlier to F.display and
the palette's ink/track.
Icon names were checked against the set the repo already uses rather than
against the package (which is not vendored here): footprints, personStanding,
activity and house are all already in use elsewhere in lib/.
* fix(steps): address review findings, and four the review did not find
SOURCERY'S FINDINGS
* #2 incomplete routines: a run whose implied ratio falls outside its
category's gain range is now REFUSED rather than clamped into range. Clamping
turned 'this routine did not measure what it claims' into a value at the edge
of the legal band — and a third-done walk (100 claimed, 30 seen) clamped to
2.0 would DOUBLE every later step.
* #3 walking steps get the household gain: a cadence inside the 60-200 gait
band that no calibrated family owns is now credited EXACTLY as today. The old
fall-through protected nothing — 'a household gain can only damp' is not
reassurance, because damping a real walk to 5-14% is a worse error than the
over-count the calibration exists to fix.
* #4/#8 atomicity and concurrent overwrites: saves are serialised on one lock
and the refit is split out as refitLiveStepGain, so a torn state — a run
durable without the profile it implies — is repairable by re-deriving from
the rows. Both sides are idempotent by key.
* #6 stale load repopulates the cache: loads carry a generation and will not
assign after an invalidation landed while they were in flight.
* #7/#9 phone permission and staleness: the permission is requested BEFORE the
routine (a refusal that blames the wearer's pocket for a grant never asked
for is a wrong reason, which is worse than none), one PhonePedometer is
reused instead of constructing 180 of them, and a final interval read is
awaited before finishing.
* #5 the prorated Android reference is NOT addressed here — it needs a
different read path, and it is a real limitation of fitting a household gain
from a 5-minute bucket.
NOT REPORTED, FOUND HERE
* kAlgoVersion was not bumped (AGENTS.md §3.4: 'without a bump nothing
recomputes'). Now 114, with a changelog entry that states plainly what the
bump can and cannot do: the gain is applied when a run is banked, so it
changes the future and CANNOT repair windows already in live_coverage,
because those rows keep no raw per-run count.
* §4.7 verbatim — the capability was wired into one call path and I never
enumerated the others. addLiveCoverage has three callers; all three are now
listed at the banking site with the decision for each.
* ARB placeholders were undeclared: 516 of 520 existing keys with a placeholder
carry an @-block, and the screen hand-rolled .replaceAll instead of using the
generated getter. Now declared and called properly.
* The screen's refusal copy blamed the wearer's pocket for a missing
permission.
Verified against the Python mirror: 76/76. NOT verified: compilation — no Dart
or Flutter toolchain was available, so flutter analyze has still never run, and
Sourcery's 'does not compile' claim remains open.
* chore(compute): make the three clamp result types explicit
Sourcery flagged `ratio.clamp(...)` and `credited.clamp(...)` as not compiling,
because `num.clamp` returns `num`.
I believe that is a false positive: `lib/data/step_calibration.dart:122` returns
`corrected.clamp(0, (rawTicks * kStepFactorMax).ceil())` straight out of an `int`
function with no cast, in shipped, CI-green code. That only compiles if Dart
applies its static special case for `clamp` when the receiver and both bounds are
the same numeric type.
But I cannot run the analyzer here to settle it, and the two possible readings
have very different costs: if the special case applies, `.toDouble()` /`.toInt()`
is a no-op; if it does not, this is a build failure in someone else's repo. The
conversion is therefore free insurance rather than a claim about whose reading is
right, and the comment beside it says exactly that.
A third site (categoryBand's `half`) was not flagged and had the same shape.
* fix(steps): the two review findings that were only half done
SOURCERY #9 — imported calibration runs stay invisible. I had DESCRIBED this in
a docstring ('also called after a database import') and never wired it: the only
caller of invalidateLiveStepCalibration was the calibration screen itself. A
comment claiming a call that nothing makes is the exact failure this whole round
was about. LocalDb.importFromDbFile has one caller, so the fix is one line in
app_state.dart — and it matters most in the case the feature exists for: a new
phone restoring the backup that holds the wearer's calibration.
SOURCERY #7 — phone reads overwrite newer counts. The previous commit only added
a final awaited read and a reused PhonePedometer; the out-of-order race itself was
untouched. On Android a query can outlive the next tick, so an older interval
landed last and replaced the newer count. Reads now carry a sequence and only the
most recently STARTED one may write.
Still open, and named as such on the PR:
* #5, the prorated Android bucket — the household reference is estimated, which
violates §3.3 on a value used as ground truth. Needs a different read path.
* #4, atomicity — mitigated (one lock, repairable refit), not atomic.
* #1, whether the clamp conversions were ever needed — still unverified.
Verified against the Python mirror: 76/76. Still not compiled.
* fix(steps): CodeRabbit's real findings — a NULL turned back into a reading
CodeRabbit finished its review after the Sourcery round and raised 14 findings,
not 9. Ten are duplicates of things already addressed here; two are not.
**NULL was being turned back into a reading.** live_step_calibration_run's
p2p_g and confidence are nullable precisely so a routine whose caller never
measured them stores an ABSENCE. The reader then did `as num?)?.toDouble() ?? 0`
and converted that absence into 0.0 — a real still reading at the bottom of the
scale, which is exactly the fabrication the nullable column exists to prevent. I
made the WRITE side honest and left the READ side fabricating, in the same commit
series that claimed 'unmeasured features stay absent, never a substituted zero'.
cadence_spm keeps its ?? 0: it is NOT NULL, and 0 there means 'no cadence
measurable', which is a different claim.
**The doc comment was on the wrong getter.** liveStrapRawSteps was inserted
between _liveRaw's doc block and _liveRaw's declaration, so _liveRaw's doc — and
with it the sentence '(real, gain-applied)' — described the new getter, while
_liveRaw had none. Both getters own their own doc now, and the false
'(real, gain-applied)' is corrected in place rather than left standing, because
the calibration reads that value and depends on it being raw.
Also: the settings row said the calibration 'teaches the strap your own walking'.
Nothing is ever written to the strap — the correction lives on the phone — so the
copy said the opposite of the feature's most important property.
Verified against the Python mirror: 76/76. Still not compiled.
* fix(steps): close the remaining CodeRabbit findings
* resetAllData calls LocalDb.wipeAll, which empties live_step_calibration —
and left the cached fit holding the rows it just deleted. A walk banked after
a data reset would still be corrected by the pre-reset gain. Same omission as
the import path, same one line.
* The household routine now STOPS when phone step access is denied, instead of
running for three minutes towards a refusal it cannot avoid. The wearer finds
out while standing there, and the message names the real cause rather than
their pocket.
* _finish locks the screen BEFORE the awaited final phone read. The read leaves
the buttons live, and a second tap would finish an already-finished session.
_busy is released again on both early returns.
* Copy: 'the band saw {band}' now says 'the band saw {band} steps'.
Left as is, deliberately: the counter-reset case. observe() floors a reading
below the baseline to the baseline, which makes the routine void rather than
crediting a negative count — the conservative outcome. Detecting the reset and
splitting the routine around it would mean deciding which side of the break the
steps belong to, and the pre-start side is a walk the wearer did not calibrate.
Voiding is the answer that cannot produce a wrong gain.
Verified against the Python mirror: 76/76. Still not compiled.
* make the calibration screen compile and pass the ui checks
the refusal switch did not cover the two refusals added later, which did not compile. both now have strings, every new string has a russian translation, the screen uses the radius and tick tokens, and the gallery test scrolls to its row now that settings is longer.
* save calibration atomically and refit it after a restore
a run could be saved while the refit or the profile write after it failed, so the screen reported a failure that later fits quietly used. a restore merged the backup's runs and profiles without refitting, so the stored gain no longer matched the runs. a calibration read that was invalidated while in flight now reads again instead of returning the old rows, and the new tables are classified for issue reports.
* put the permission docs back on requestPermission
stepsBetween had been inserted between requestPermission and its doc comment.
* take kAlgoVersion 118 for step calibration
114 to 117 are claimed by other open changes. the freeze goldens differ only in algo_version.
* ask what the wearer counted, and flush android's step batch around a routine
done is live from the first band step, so a wearer who stopped at 60 was
fitted as if they had walked 100. android delivers step counts up to a
minute late, so the household reference missed the end of a three-minute
routine.
* wait for the flushed step batch in the pedometer, and pin the early-stop fit
* drop the cached calibration after an import even if the refit fails
* refuse a gait routine whose cadence is outside the walking range
such a run was saved and counted toward the fit but could never give the
family a band to own its walks.
* take the household reference from the step sensor's own total on android
a range query prorates five-minute bins, so a three-minute routine could pick
up steps from before it or miss its own. the reference is now the running
total at the end minus at the start, each after a flush. a failed final read
refuses the routine instead of keeping an earlier partial count.
* inflect the calibration counts in russian and keep the gait result neutral
* end a walk routine at the done tap, not when the count is answered
the clock kept running while the count question was open, so a slow but
valid walk could fall below the gait cadence range.
* refuse a household routine on android without two flushed totals
the final read fell back to the prorated range query when the starting total
was missing, and a refused flush was waited out and read anyway. on android
the reference is now only a difference of two totals taken after accepted
flushes; anything less refuses the routine.
* make the russian band counter label unambiguous
* wait for a step-sensor flush to complete instead of a fixed pause
the flush call now answers from onFlushCompleted, after the batched counts
are delivered, and a flush that is refused or does not complete in time
reads as failed.
* start a household routine only once the phone's starting total is in
the band baseline and the clock were taken before the phone's starting
total settled, so the two counts covered different spans. access, then the
phone total, then the band; no starting total means the routine does not
start.
* ask for phone step access before a household routine every time
ios reports an undecided permission as allowed, so the request was skipped
on a first-use phone and the prompt came up mid-routine.
* stop the band count where the household phone reference stops
the band's count stopped at the last tick before done while the phone's
reference ran on through the final flush or query. both now end at one
instant: after the completed flush on android, at the query's end time
elsewhere.
* report a stored calibration as saved even if the reload after it fails
the save failure message invited a retry that stored the same routine twice.
* drop the cached calibration even when an import fails part way
the import commits page by page, so a page that throws can leave merged
calibration runs behind; the invalidation now sits in a finally around the
import itself.
* show the saved category from the save itself when the reload fails
---------
Co-authored-by: Minis Agent <minis@agent.local>
Co-authored-by: abdulsaheel <abdulsaheel@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix background sync: hold BLE connection alive + persistent flusher + honest upload count