Skip to content

fix(#412): one NaN reading no longer fails an SD import — dropped and reported, as the live path does (#294) - #413

Merged
tylerkron merged 2 commits into
mainfrom
fix/412-sd-import-nan
Sep 20, 2026
Merged

tylerkron merged 2 commits into
mainfrom
fix/412-sd-import-nan

Conversation

@tylerkron

@tylerkron tylerkron commented Sep 19, 2026 •

Copy link
Copy Markdown
Contributor

Closes #412. Not merging — for review.

What was wrong

If an SD card log held even one NaN reading, the whole import failed with Import Failed. Microsoft.Data.Sqlite refuses to bind a NaN (Cannot store 'NaN' values.), so the 1,000-sample batch holding it failed, along with everything the parser had not reached yet. A NaN in the first batch meant nothing was imported. A later NaN left an ImportFailed fragment with the rest of the file dropped. Either way the failure was logged at Error, which makes it a Sentry event. Core's parsers pass NaN through: the CSV parser accepts NaN/nan, and the .bin float leg takes the float as-is.

The fix

SdCardSessionImporter.WriteSamplesAsync now applies the live path's rule from #294 (SessionSampleWriter.Add): it skips a NaN reading, counts it, and imports everything else.

Diffstat: +61/−12 in SdCardSessionImporter.cs, plus a new 257-line test file.

Verification

  • 8 new tests in SdCardImportNaNTests. I ran them against the unchanged code, with only the logger seam added. 6 of 8 fail, all with InvalidOperationException: Cannot store 'NaN' values.:

    • NaN in the first batch
    • NaN in a later batch (past the first 1,000-sample commit)
    • user-facing guidance
    • Warning-not-Error
    • every reading NaN
    • a firmware CSV with lower-case nan through ImportFromFileAsync

    The other 2 pass on main, as they should: a clean log produces no guidance, and infinities store. They pin behaviour that must stay the same.

  • Mutation control: I changed the NaN summary from Warning to Error. That killed 3 tests, all on the Assert.Empty(_log.Errors) line, so the no-Sentry assertion does catch a regression.

  • Build gate green locally under a build.N lock: desktop, iOS restore, iOS Release iossimulator-arm64. Full suite 1065/1065 passed. Lock files unchanged. Android is left to CI.

  • No board attached. This change can't be observed on the device anyway: the import path runs from a file, and the tests go through the real parser.

  • Warnings, taken from CI's own logs at both ends (Library + Desktop head job): merge-base 858d143 166, this head 789d566 166. The importer has the same 3 pre-existing warnings on both sides (CS8625 plus two CS8601), each shifted +48 lines. The new test file adds none. Tests went from 1057 to 1065 (+8).

🤖 Generated with Claude Code

tylerkron and others added 2 commits September 19, 2026 10:48
…ain with 'Cannot store NaN values'

Adds a logger seam to the importer's internal constructor so a test can see which level an
outcome was logged at. WIP: the guard follows.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… whole import, and says so

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tylerkron

Copy link
Copy Markdown
Contributor Author

/agentic_review

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Skip NaN readings during SD card imports

🐞 Bug fix 🧪 Tests 🕐 40+ Minutes

Grey Divider

AI Description

• Skip NaN SD readings so valid samples continue importing.
• Report omissions through user guidance and one warning, without Sentry errors.
• Cover batch boundaries, all-NaN logs, CSV parsing, logging, and infinity persistence.
Diagram

graph TD
  P["SD Log Parser"] --> I["Session Importer"] --> N{"Reading NaN?"}
  N -- No --> B["Sample Batch"] --> D["SQLite Session"]
  N -- Yes --> C["Dropped Counter"] --> R["Import Outcome"]
  D --> R
Loading
High-Level Assessment

The focused NaN-only guard is the best approach because it mirrors the established live-sample behavior and filters exactly what Microsoft.Data.Sqlite rejects. Using IsFinite would unnecessarily discard supported infinities, while extracting a shared helper for a single double.IsNaN check would add indirection without unifying the distinct reporting lifecycles.

Files changed (2) +318 / -12

Bug fix (1) +61 / -12
SdCardSessionImporter.csDrop and report NaN readings during SD imports +61/-12

Drop and report NaN readings during SD imports

• Filters NaN analog readings before SQLite batching, counts omissions, and emits one warning per import. Adds user-facing guidance for partial and all-NaN imports and introduces injectable logging through the internal test constructor.

Daqifi.Avalonia/Daqifi.Desktop/Loggers/SdCardSessionImporter.cs

Tests (1) +257 / -0
SdCardImportNaNTests.csAdd end-to-end coverage for NaN-tolerant SD imports +257/-0

Add end-to-end coverage for NaN-tolerant SD imports

• Adds database-backed tests proving NaN readings are omitted without losing valid samples across batch boundaries. Covers user guidance, warning-level logging, all-NaN logs, real CSV parsing, and preservation of positive and negative infinity.

Daqifi.Avalonia.Tests/Loggers/SdCardImportNaNTests.cs

@qodo-code-review

qodo-code-review Bot commented Sep 19, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

Grey Divider

Great, no issues found!

Qodo reviewed your code and found no material issues that require review

Grey Divider

Tip of the day
💡 Did you know, you can describe a rule in plain language on the Rules page and Qodo drafts it for you

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Previous reviews

Review updated until commit 789d566 ⚖️ Balanced

Results up to commit 789d566 ⚖️ Balanced


No changes from previous review

Grey Divider

Qodo Logo

@tylerkron

Copy link
Copy Markdown
Contributor Author

/agentic_review

@qodo-code-review

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit 789d566

@tylerkron

Copy link
Copy Markdown
Contributor Author

Qodo-clean, CI green — ready for review. Two clean rounds on head 789d566, the second graded Balanced (fired 21:00:57Z, summary updated 21:03:45Z), both settled with 0 unresolved qodo threads. CI passed all heads on that same SHA, including 1065/1065 tests. Not merging.

@tylerkron

Copy link
Copy Markdown
Contributor Author

7-way sweep — main 858d143 + #405 + #406 + #407 + #408 + #411 + #413 + #414 built together (2026-09-19 21:20Z).

@tylerkron

Copy link
Copy Markdown
Contributor Author

Sweep: the combination now includes #415 — 8-way merge verified. Nothing merged, nothing pushed.

The earlier sweep built #405 #406 #407 #408 #411 #413 #414 onto main 858d143 → tree fb11889, 1096/1096. #415 opened afterwards and had been built with nothing. This run re-derived the state and rebuilt the set with #415 in it, so this PR is now verified against the full combination the user is about to merge.

State re-derived

origin/main is still 858d143 — unchanged. Re-running the seven-way octopus in a fresh worktree reproduced tree fb11889 byte-for-byte, which independently confirms both that main has not moved and that the earlier result stands. All seven heads unchanged; this PR is still MERGEABLE/CLEAN with CI green on its ready-note SHA and 0 unresolved qodo threads (author filter validated against the #196 control = 3 threads).

#415 on its own

92db2b6 — ready note names the current head, CI run 35470171059 green on that SHA, MERGEABLE/CLEAN, 0 qodo threads, and its green does not predate main's head or the newest .github/ change (94fcac3, 2026-09-08). It adds no top-level type and removes no member: the whole diff is nullable annotations (DataSample?, Expression?, CancellationTokenSource?, LoggingSession?, string?) across four production files, +8/−8.

8-way merge

Two orders — octopus main ← 405…415, and sequential 415, 414, 413, 411, 407, 408, 406, 405 — both giving tree c15f6937439e15485867aac96a4c6c7e8f4d98f9. And git diff fb11889 c15f693 is exactly #415's four files: the 8-way tree is the already-verified 7-way tree plus eight lines, so nothing about this PR's position in the set changed.

Full gate under one build.1 lock, judged by exit code:

leg result
desktop --no-incremental ✅ exit 0, 155 warning(s)
iOS restore (no -r) ✅ exit 0
iOS Release -r iossimulator-arm64 --no-restore ✅ exit 0, 156 warning(s), 112.0 s
HeadlessBench Release --no-incremental ✅ exit 0, 156 warning(s), 3.5 s — it emitted Program.cs(2060,13) CS0618, so it really compiled #406's Program.cs (the previous sweep caught this leg "succeeding" in 0.77 s against a stale assembly)
dotnet test ✅ 1096 / 1096, 0 failed, 0 skipped

git status clean, no packages.lock.json drift.

Warnings improve, and nothing is added

Both ends re-measured here with --no-incremental, not carried over: desktop 166 → 155, iOS Release 167 → 156. The set diff is identical on both heads — 11 removed, 0 added, no line-number shift (6 × CS8625 + 5 × CS8618, all #415's); 8 of the 10 other warning codes bit-identical. The reconciliation identity closes at all four ends.

The one cross-PR hazard checked, and cleared

#415 annotates production members nullable while #407/#408/#413 own the IAppLogger test fakes, and #415's body says two Error(null, …) sites were left because of those PRs. In the merged tree those sites are still exactly two (DatabaseMigrator.cs:739, SdCardSessionImporter.cs:751 — #413 shifted the latter but neither added nor removed one), no test type implements IChannel, and #414's vm.SelectedDevice?.DeviceSerialNo is IStreamingDevice's member, not the Notifications one #415 touches. The decisive check is that the test project compiles at all: it runs TreatWarningsAsErrors, so a CS8602/CS8604/CS8767 cascade into the merged test files would have failed the build rather than passed quietly.

Test count reconciles

1057 (main) + 19 (#405) + 0 (#406) + 0 (#407) + 0 (#408) + 3 (#411) + 8 (#413) + 9 (#414) + 0 (#415) = 1096 ✅. #415's +0 was re-derived from its diff — it touches no file under Daqifi.Avalonia.Tests/.

Not covered

#363 (dependabot, BLOCKED) has never been built with any of these eight. Android was not built locally (workload absent on this Mac); CI covers it. No visual re-capture — none of the eight touches tools/parity-audit/baselines/macos-arm64.sha256, so the combination's visual state is unchanged from the 7-way's, but the combination itself was not re-captured.

Throwaway branches deleted. Nothing merged, nothing pushed.

@tylerkron

Copy link
Copy Markdown
Contributor Author

Sweep: #413 re-verified and built with the other five, onto today's main

Result: green. The previous sweep built this PR onto main 858d143; main has since moved to d8ef0f9 (#405, #406, #407 merged), so that verification needed redoing.

check result
ready-note SHA = head ✅ note at 21:08:15Z names 789d566 = current head, no push since
CI green on that SHA ✅ run 35468880064 "Build", head_sha 789d566, success
mergeable ✅ MERGEABLE / CLEAN
qodo threads ✅ 0 threads, 0 unresolved (author-leg control #196 returns 3, so the filter is live)
stale pipeline ✅ newest .github/ change on main is 2026-09-08
stale base ⚠️ yes — see below

⚠️ This PR's green predates all three merges

#405 merged 04:22:46Z, #406 04:25:27Z, #407 05:00:43Z (2026-09-20). This PR's newest Build run was created 2026-09-19T20:55:46Z — before all three. The badge is displayed against a main that no longer exists (strict: false), so the local build below is the only current evidence for this PR. Same is true of #408, #411, #414 and #415; only #416's run postdates the merges.

Note also that the ready note quotes 1065/1065, measured on the older base. On today's main the six-way tree runs 1116/1116, and this PR's own contribution re-derives from its diff as +8 (SdCardImportNaNTests, eight [Fact]).

Specific to this PR

#413 edits SdCardSessionImporter.cs, a production file inside the scope of the repo-wide BindingFacts source scanner that #405 tightened. That guard was run explicitly on the merged tree (--filter "FullyQualifiedName~Binding" → 175 passed, 0 failed). The new type SdCardImportNaNTests does not exist on main and is not introduced by any other PR in the set.

The 6-way merge

origin/main d8ef0f9 ← #408 + #411 + #413 + #414 + #415 + #416, built in two orders
(octopus, and sequential 416→415→414→413→411→408). Both land on the same tree
ebe16170827e2f2c2cdb745a2ec41885a8921f40
. One auto-merge, SerialPortPollingDeviceWatcherTests.cs.

git diff c15f693 ebe1617 — against the tree the 8-way sweep verified — is one file, +650/−0: BootloaderHoldServiceTests.cs. So #405/#406/#407 merged as squashes into exactly the content that sweep predicted, and #416 is textually orthogonal to the other five.

Full gate on ebe1617 — judged by exit code, not by grepping for Build succeeded

leg result
desktop --no-incremental exit 0, 155 warning(s)
iOS restore (no -r) exit 0
iOS Release -r iossimulator-arm64 --no-restore --no-incremental exit 0, 156 warning(s)
HeadlessBench restore exit 0
HeadlessBench Release --no-incremental exit 0, 156 warning(s)
dotnet test 1116 passed / 0 failed / 0 skipped

Working tree clean, no packages.lock.json drift. The HeadlessBench leg demonstrably compiled — its output carries Program.cs(2060,13): warning CS0618, not a stale-assembly no-op.

Test reconciliation — each delta re-derived from that PR's own diff

main d8ef0f9 = 1057 (at 858d143) + 19 (#405/#406/#407 together) = 1076.

#408 #411 #413 #414 #415 #416
+0 +3 +8 +9 +0 +20

1076 + 0 + 3 + 8 + 9 + 0 + 20 = 1116 ✅

Hazards checked by compilation, not by argument

Not claimed

#363 (dependabot, UNKNOWN) was not built with any of these. Android not built here — CI covers it. No visual re-run and none needed: the merged diff is 15 files, all .cs; no .axaml and no baseline manifest. No bench run — none of the six is device-observable. main's own warning baseline was not re-measured, so no warning delta is claimed.

Full log: ~/.claude-daqifi/avalonia-loop/logs/20260920T053803Z-sweep-6way.md.
Nothing merged, nothing pushed.

@tylerkron
tylerkron added this pull request to the merge queue Sep 20, 2026
Merged via the queue into main with commit ae48adc Sep 20, 2026
7 checks passed
@tylerkron
tylerkron deleted the fix/412-sd-import-nan branch September 20, 2026 06:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SD import: one NaN sample fails the whole import; the live path drops it (#294/#298), the importer has no guard

1 participant