Skip to content

Avoid allocating global offline area grid - #97

Merged
pfeiferj merged 1 commit into
pfeiferj:mainfrom
FrogAi:codex/reduce-mapd-memory
Aug 9, 2026
Merged

pfeiferj merged 1 commit into
pfeiferj:mainfrom
FrogAi:codex/reduce-mapd-memory

Conversation

@FrogAi

@FrogAi FrogAi commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Replace the package-level AREAS = generateAreas() allocation with an O(1) coordinate-to-area calculation.
  • Reuse the calculated area for the offline tile path and unloaded fallback box.
  • Derive the generation grid and runtime lookup from the same areaBox(latitudeIndex, longitudeIndex) helper.

Motivation

Importing the maps package currently creates a global slice containing 1,045,456 Area values before the first tile lookup. On a 64-bit build, each Area is 56 bytes, so the slice accounts for 55.83 MiB before considering other process memory.

FindWaysAroundPosition then linearly scans that slice to locate one 0.25-degree tile. If a tile cannot be loaded, the old implementation allocates another 55.83 MiB slice, constructs the complete grid again, and rescans the entire slice for the fallback box.

That fallback is hit more often than tile crossings alone would suggest. The main loop re-enters FindWaysAroundPosition whenever len(state.Data.Ways()) == 0, and that condition stays true for as long as no tile loads. On a device without tiles downloaded for the current area, the old path therefore ran on every GPS fix inside a loop with a 50 ms delay.

This change calculates the same tile directly from latitude and longitude while preserving the existing first-match behavior at exact tile boundaries.

Memory impact

mapd-memory-comparison

mapd-memory-raw.csv

Metric Baseline (68813e0) Optimized (ad3c618) Difference
Mean PSS 63.67 +/- 0.04 MiB 8.17 +/- 0.05 MiB -55.50 MiB (87.2%)
Mean Go heap allocation 56.01 MiB 0.17 MiB -55.84 MiB
Expected removed slice - - 1,045,456 x 56 bytes = 55.83 MiB

The PSS values are reported as mean +/- sample standard deviation.

Benchmark methodology:

  • Go 1.25.1 on Linux/amd64.
  • The same isolated probe imported pfeifer.dev/mapd/maps, forced runtime.GC() and debug.FreeOSMemory(), and then remained idle.
  • Ten process launches per commit using a repeated ABBA ordering to reduce ordering bias.
  • PSS was read from /proc/<pid>/smaps_rollup; Go heap values came from runtime.MemStats.
  • This isolates package-initialization cost rather than simulating a full on-road workload.

The measurements were collected at optimized commit ad3c618. Final commit 6b8c3ea preserves the same package-initialization behavior; its additional changes share the area-box calculation and size the offline generation grid exactly.

The runtime lookup changes from a linear scan of up to 1,045,456 boxes to constant-time arithmetic, and the fallback grid allocation is removed entirely.

Correctness fixes

Three latent defects fall out of the rewrite:

  • Boundary inconsistency in the unloaded fallback. The old fallback rescan had no break, so the last matching area won, while the filename it had just attempted came from the first match. On an exact 0.25-degree boundary, the fallback box could therefore disagree with the attempted tile path. The new code uses the same calculated area for both.
  • Degenerate box at (0, 0). make([]Area, int((361/0.25)*(181/0.25))) over-allocated 8,656 slots beyond the 1,036,800 real cells. Those trailing zero-value Areas have box (0,0)-(0,0), and the last-match rescan selected one of them at exactly (0, 0).
  • Junk tile during generation. GenerateOffline iterated the whole slice, so generating a region containing the origin could write a degenerate offline/0/0/0.000000_0.000000_0.000000_0.000000 tile. Sizing the slice to latitudeAreas * longitudeAreas removes those slots.

Validation

A temporary comparison harness was used for focused and exhaustive validation without adding test-only code to this commit:

  • Checked values immediately below, exactly on, and immediately above every internal latitude and longitude tile boundary: 6,474 boundary assertions passed.
  • Compared old and new resolved tile filenames and found/not-found results across 2,275,789 positions: 1,036,800 cell centres, 1,038,961 grid corners, 200,000 deterministic random fixes, and 28 world-edge or invalid-coordinate cases. There were zero differences.
  • Confirmed generateAreas() is bit-identical to the previous implementation across all 1,036,800 real cells; the 8,656 removed slots were all zero-valued.
  • Confirmed the centre of every generated cell resolves back to that same cell through areaForPosition.
  • Checked ordinary coordinates, world edges, poles, datelines, NaN, and positive and negative infinity.
  • Confirmed the math.IsNaN guard is required: NaN fails both range comparisons, and converting the result of math.Ceil(NaN) to an integer is implementation-dependent.
  • go test ./... and go vet ./... passed for the final source on Linux/amd64 with Go 1.25.1.

Compatibility

  • No offline map file-format changes.
  • No changes to the naming format or any valid tile path; only the degenerate zero-area artifact is no longer generated.
  • No new dependencies.
  • generateAreas() remains available for offline map generation and now uses the shared areaBox helper.
  • The unloaded fallback box now remains consistent with the area used to select the tile filename.
  • For non-power-of-two AREA_BOX_DEGREES values, repeated addition and indexed multiplication can round differently. Generation and lookup now derive bounds through the same helper.
  • AREAS was exported. Nothing in this repository referenced it, but removing it is a package-level API change for any external consumer that did.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants