[johntaylor-cell]

Why are these changes needed?

Follow-up to #64281 (codec), #66104 (kernels), #66105 (extraction) and #66106 (ingest metrics). This is the consumer side: the controller decodes columnar handle reports and aggregates them as arrays.

The ServeController aggregates autoscaling metrics in a single-threaded control loop. Each handle report covers every replica that handle routes to, so at high fan-in the controller spends much of each loop cloudpickle-decoding reports, rebuilding one Python object per timestamped point. This decodes the SCR1 format into flat float64 arrays instead, and merges them in a single vectorized pass.

What changed

The controller wire-detects the frame magic per report, so it reads whatever each sender actually emitted rather than relying on config agreement. Columnar frames decode straight into a per-handle array store; everything else takes the existing path unchanged.

get_total_num_requests routes three ways: all-columnar, all-object, or one merged pass when both stores hold sources. The mixed case is a steady state, not a rollout artifact — replica reports are never columnar, and direct-ingress deployments report from replicas even when handle collection is on, so both stores are live simultaneously. It has to merge every source in one pass because summing two separate aggregations is correct for MEAN but wrong for MAX and MIN.

Decode duration is recorded separately from decompress duration, since keeping the two wire formats comparable is the point of that metric.

Scoped to correctness. The tick optimizations for wide handles — a vectorized gather and a memo on the running-set generation — ride with the producer switch, because nothing exercises them until frames actually flow. The masking here is the straightforward per-replica form.

This commit is inert

Nothing produces columnar frames yet. The producer switch is six lines in router.py and lands in the final PR, so merging this cannot change behaviour: the decode branch is never entered. That also makes the ordering safe by construction — a router cannot emit SCR1 to a controller that does not yet understand it.

Test Plan

test_columnar.py runs 34 cases covering the array store, the three aggregation paths, controller dispatch, mixed-format fleets, cross-format staleness, stale-replica masking and pruning, and drop logging.

Also ran every suite touching the changed modules: test_autoscaling_policy (47), test_deployment_state (256), test_controller (71), test_application_state (122), test_schema (116).

[gemini-code-assist]

Code Review

This pull request introduces an array-native, columnar format for autoscaling metrics in Ray Serve, optimizing the decision-time merge and aggregation paths using NumPy. It adds a new autoscaling_metrics_merge module, integrates columnar and mixed-mode aggregations into AutoscalingStateManager, and adds performance tracking for metrics ingestion. The review feedback focuses on correcting a misleading comment about dropping columnar replica entries, replacing platform-specific "<i8" dtype literals with standard np.int64, and adhering to PEP 8 naming conventions by removing leading underscores from actively used loop variables.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{Reviewed by Cursor Bugbot for commit c59920d. Configure here.}

[gemini-code-assist]

Code Review

This pull request introduces a columnar wire format for autoscaling metrics in Ray Serve to improve performance by enabling efficient array-based processing. Key changes include adding numpy-based aggregation logic, updating the autoscaling state manager to handle columnar metrics with memoization, and implementing a unified staleness gate across wire formats. The controller is updated to wire-detect the format and track decoding metrics separately from decompression. I have kept the review comment regarding a potential type mismatch in numpy array indexing, as it identifies a critical issue that could lead to runtime errors.

[abrarsheikh]

High-level comments

1. code is currently hard to read and requires refactoring. I think it requires some more revisions.
2. offs / csr / run_offsets, which are one concept under three names — pick source_offsets and use it everywhere.
3. ts now means two types in this file: a TimeSeries object list, a timestamps ndarray in _running_csr method and one other place, in the autoscaling state file.

[johntaylor-cell]

High-level comments

1. code is currently hard to read and requires refactoring. I think it requires some more revisions.
2. offs / csr / run_offsets, which are one concept under three names — pick source_offsets and use it everywhere.
3. ts now means two types in this file: a TimeSeries object list, a timestamps ndarray in _running_csr method and one other place, in the autoscaling state file.

Refactored, clarified variable names