Skip to content

perf(query): avoid unused property hydration in COUNT row markers #1249

Closed

Description

@DecisionNerd

Verified cause and scope

lower_aggregate anchors COUNT(*)'s cypher_row_marker to the last input-schema field. A property field can therefore be decoded and materialized even though the query only counts rows. The marker returns non-null true for every row, including null inputs; keep that behavior. Prefer a qualified identity column already in the aggregate input schema, falling back to the existing last scalar field when no identity is available. Do not change explicit nullable count(property), SUM, grouping, optional-row cardinality or the edge-count shortcut.

This is a bounded repair selected by #1207, native child and blocker of #1194. It adds no storage format, compatibility layer or writer policy. #1207 owns the shared assessment fixtures and decision ledger. Implement after that prerequisite merges; no overlapping assessment or generic planner refactor.

Measured selection

On integrated main a3a620c with the frozen public scalar probe (baseline 60f02ce, experimental 5a78be3), three fresh processes each execute five queries on the same compacted/reopened 4,101-node fixture. All seven independent scalar oracles pass.

  • Count-all warm median: 14,318,835→10,852,928.5ns. Process CPU 0.16/0.16/0.17→0.15/0.15/0.15s; this does not meet that case's 0.02 s CPU threshold and is reported as a latency observation.
  • Empty-label process medianCPU 0.17→0.15s meets its preselected 0.02 s saving; warm 14,588,430→10,942,945.5ns.
  • Count-all, label-count and empty-count each avoid 923,923 syscall read bytes over five queries and approximately 308.8 KB writes. Explicit identity count, nullable property count, SUM and selective property count are controls; do not claim their work improves.
  • All seven cases satisfy their preselected RSS ceilings; no permanent representation changes. Collectors distinguish process/native RSS, logical counters, syscall bytes, filesystem blocks and sampled disk peaks.

The experimental implementation and failed first compile are retained honestly in #1207 evidence; no production improvement is claimed until this issue merges.

Acceptance criteria

  • Use an identity already present in the current aggregate input schema, preserve its qualifier and retain scalar-only fallback. Keep the true-for-null row marker; count(identity) is not a replacement for COUNT(*).
  • Direct public execution covers populated/empty scans; nullable and heterogeneous properties; unmatched OPTIONAL rows; WITH entity aliases; scalar-only WITH/UNWIND including null/empty input; grouping including literal-null keys; fixed/variable-length edge counts and filtered frontiers.
  • Deterministic physical/resource assertions prove unrequested property values are not materialized for row counting. Explicit nullable count/SUM retain their property reads. Logical plans alone are insufficient.
  • Preserve current-format authentication and malformed mandatory-key/late-decoder refusal, including a storage-backed COUNT after a retained property file is modified. Do not weaken existing unselected-row or corruption tests to satisfy an optimization.
  • Public construction/mutation, immediate count/query, reopen, export/full verification, clean import and subsequent mutation retain exact identities, properties and aggregate outcomes. Reuse existing lifecycle fixtures where applicable.
  • Record final source and frozen executable, exact scalar oracles, deterministic avoided work, process CPU/RSS and I/O, physical allocation and temporary overlap. Preserve the preselected test(perf): assess practical query and storage efficiency opportunities #1207 ceilings; do not gate CI on noisy latency.
  • Focused PR to main, relevant local gates, exact-head CI Gate, clean merge state, verified review findings, no unresolved threads, closing reference exactly this issue; squash merge and verify closure.

Before v1.0.0 backward compatibility is out of scope. Preserve current-format semantics, authentication, cancellation, retries, recovery and active snapshots. No S24/S26 or new certification workflow is authorized.

Warm medians use the median of three per-process medians, each over executions 2–5; they are not pooled medians. Frozen FD samples include unlinked open files and globally deduplicate physical owners; sampled peaks are not hard bounds.

Final implementation and resource disposition — PR #1255

Production source e8cb7dcb5d3f65e207df3edca72d7f2e7611d7a8 is frozen against a baseline tree identical to merged b9765e8e. The seven scalar oracles and direct public lifecycle/corruption/nullable-count regressions pass. The 129-row resource test avoids 34,281 logical spill bytes while authentication and physical rows remain equal; no property validation is skipped.

The final paired count-all/label/empty cases meet their original CPU thresholds (0.03/0.02/0.03 s median savings) and avoid 923,911 syscall read bytes each per five queries. Physical source and whole-fixture allocation are unchanged. Logical counters, process RSS, filesystem blocks and sampled allocation remain separate.

Resource limitation: the explicit identity-count control RSS ceiling remains failed, at 70,184 KiB against the unchanged 69,952 KiB cap. No ceiling was raised and no result was replaced. A predeclared fixed eight-pair alternating diagnostic crosses that cap on unchanged baseline 3/8 and candidate 1/8. Identical explicit-count plans/work and variable executable residence support accepting the targeted improvement with this disclosed control-budget limitation; they do not prove RSS equivalence, rule out every allocation effect or establish the cause of the original unsampled maximum. All affected row-count cases meet their resource ceilings. This is the evidence-based disposition of the control observation, not a claim that all seven RSS checks pass.

Required exact-head CI, squash merge and verified closure remain pending. Full local pre-push retains the known #1192 hardcoded-/tmp admission failure; relevant focused tests and local quality checks passed. Evidence and collector sources: docs/development/evidence/count-row-marker-1249.json in PR #1255. No S24/S26 or new certification workflow has started.

Verified merge and closure

PR #1255 squash-merged as 9312a47096d16af66313fee2aeeb766c892b906d on 2026-09-11 at 09:09:56Z; this issue closed at 09:09:57Z. Required Test Suite/CI Gate run 34581117389 passed at exact head fbe020fe58a203ec8c7960ee477069d49ad7294e, including authoritative Bazel, native durability, both bindings and concurrency. CLEAN state, no unresolved review threads and closing reference exactly #1249 were verified before merge. Branches were deleted while preserving the worktree.

The resource-recording criterion is complete with the explicit failed-control-budget disposition above; its checkbox does not claim the original identity RSS ceiling passed. Frozen measurements and all failed/superseded observations remain in the merged evidence. Earlier pending-merge statements are historical.

Activity

  1. DecisionNerd commented on Sep 29, 2026

    @DecisionNerd
    ContributorAuthor

    Evidence relocated from the docs tree (#1625). The 1 file(s) below were committed under docs/development/ as measurement evidence for this issue. They are removed from the tree by #1628 (the #1625 pull request); the repository keeps method and content digests, results live here. Every file remains available at its permalink on commit 29a7b34ebe44, and its SHA-256 is recorded next to it.

    Files (1 total, 12,578 lines)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions