Repository navigation
fix: convert anonymous classes to named inner classes for Jackson serialization - #20070
zhang-arvin wants to merge 2 commits into
Conversation
…ialization Anonymous subclasses in Long/Double/Float First/Last aggregator factories were being serialized as LongLastAggregatorFactory$1 instead of the registered @JsonTypeName. This caused 'Could not resolve type id' errors during Jackson deserialization. Converted all NIL_AGGREGATOR and NIL_BUFFER_AGGREGATOR anonymous classes to private static named inner classes in all 6 factory classes. Closes apache#7599
FrankChen021
left a comment
There was a problem hiding this comment.
| Severity | Findings |
|---|---|
| P0 | 0 |
| P1 | 1 |
| P2 | 2 |
| P3 | 0 |
| Total | 3 |
Reviewed 6 of 6 changed files.
The review found one unresolved Jackson-type risk, two remaining anonymous NIL paths, and missing serialization regression coverage.
This is an automated review by Codex GPT-5.6-Luna(max)
| ) | ||
| private static final Aggregator NIL_AGGREGATOR = new NilLongLastAggregator(); | ||
|
|
||
| private static class NilLongLastAggregator extends LongLastAggregator |
There was a problem hiding this comment.
[P1] Named NIL class does not fix the Jackson type ID
NilLongLastAggregator extends LongLastAggregator, not AggregatorFactory. The Jackson subtype registration applies to the factory, while the static NIL field is not serialized. If the NIL object is serialized directly, it still has no registered longLast type, so the change does not demonstrably fix the reported serialization failure.
| NilColumnValueSelector.instance(), | ||
| false | ||
| ) | ||
| private static final Aggregator NIL_AGGREGATOR = new NilDoubleFirstAggregator(); |
There was a problem hiding this comment.
[P2] Anonymous NIL implementations remain elsewhere
The conversion is incomplete: anonymous NIL implementations remain in StringFirstAggregatorFactory, StringLastAggregatorFactory, and the numeric Any factories. The same class-name serialization risk therefore remains for those absent-column paths if runtime aggregators are serialized.
| NilColumnValueSelector.instance(), | ||
| false | ||
| ) | ||
| private static final Aggregator NIL_AGGREGATOR = new NilLongLastAggregator(); |
There was a problem hiding this comment.
[P2] No serialization regression test covers the fix
Existing serde tests only deserialize hard-coded factory JSON; they do not serialize a factory or exercise NIL factorization. Add round-trip coverage for the affected types and the missing-column branch.
The existing testSerde never serializes, so it cannot observe the type id Jackson actually writes. A regression that put an anonymous class on the jackson-facing surface would emit an id like LongLastAggregatorFactory$1 and the old test would still pass. Adds: - testSerdeRoundTrip: serialize -> assert no generated $N class name in the JSON -> deserialize -> assert equality, name, cache key, and that a second round trip is stable. - testNilAggregatorIsUsedForMissingColumn: exercises the NilColumnValueSelector branch of factorize(), asserts the shared instance is reused, that aggregate() is a no-op, and that serializing the factory never surfaces the NIL aggregator's class. Addresses review feedback on apache#20070.
|
Thanks @FrankChen021 — the review is right on all three counts, and the P1 in particular goes deeper than the diff does. Here is where I landed after tracing it. P1 — the change does not fix the reported failureConfirmed.
This also means the type id in the report — So this PR's premise does not hold. I did not want to silently drop the work, though, because the reviewability problem you identified is real, so I kept the mechanical rename (it is harmless and arguably clearer) and added the thing that was actually missing: P2 #3 — added the regression coverageThe existing
P2 #2 — confirmed, and I left it out deliberatelyYou are right that I deliberately did not extend the rename to them, because on the analysis above those anonymous classes are equally unreachable from Jackson — extending the rename would multiply the diff by ~3x while fixing nothing verifiable. If you'd rather have the sweep for uniformity, say so and I'll do it in one commit; I just don't want to justify it with a bug that does not exist. Happy to close this PR instead if you judge that the right call — the test coverage is the only part with real value, and I can carry that into a fresh PR titled for what it actually does. |
FrankChen021
left a comment
There was a problem hiding this comment.
🟡 Changes recommended
The current head still does not address the failure described in the PR. The six renamed NIL classes are runtime Aggregator/BufferAggregator implementations returned by factorization; the Jackson subtype registration is for AggregatorFactory, and the factory serialization path exposes only its configured properties. Renaming these classes therefore cannot change the reported AggregatorFactory type id. The new tests serialize the factory itself, so they pass without exercising the production change.
Reviewed 7 of 7 changed files:
- processing/src/main/java/org/apache/druid/query/aggregation/firstlast/first/DoubleFirstAggregatorFactory.java
- processing/src/main/java/org/apache/druid/query/aggregation/firstlast/first/FloatFirstAggregatorFactory.java
- processing/src/main/java/org/apache/druid/query/aggregation/firstlast/first/LongFirstAggregatorFactory.java
- processing/src/main/java/org/apache/druid/query/aggregation/firstlast/last/DoubleLastAggregatorFactory.java
- processing/src/main/java/org/apache/druid/query/aggregation/firstlast/last/FloatLastAggregatorFactory.java
- processing/src/main/java/org/apache/druid/query/aggregation/firstlast/last/LongLastAggregatorFactory.java
- processing/src/test/java/org/apache/druid/query/aggregation/firstlast/last/LongLastAggregationTest.java
Review scope included the incremental test commit, the full current merge-base diff, the prior review findings and author response, the AggregatorFactory Jackson mixin, the factory property accessors, the NIL factorization paths, and the numeric aggregator base classes.
Validation: static review only; git diff --check passed for the current diff.
| Severity | Findings |
|---|---|
| P0 | 0 |
| P1 | 1 |
| P2 | 1 |
| P3 | 0 |
| Total | 2 |
This is an automated review by Codex GPT-5.6-Luna(max)
After addressing the findings or replying to the comments, you can request another review from me to trigger a new automated review.
| NilColumnValueSelector.instance(), | ||
| false | ||
| ) | ||
| private static final Aggregator NIL_AGGREGATOR = new NilLongLastAggregator(); |
There was a problem hiding this comment.
[P1] Named NIL class does not fix the Jackson type ID
Finding: The new NilLongLastAggregator is still an Aggregator, while Jackson's subtype registration is on AggregatorFactory; the only references to this static instance are the factorize/factorizeBuffered return paths, and the factory serializer exposes only its name, fieldName, and timeColumn properties. Consequently, this rename cannot change the AggregatorFactory type id or resolve the reported LongLastAggregatorFactory$1 failure; if the runtime aggregator is serialized through another path, the new private class is still not registered.
Suggestion: Trace the actual object being serialized and fix or register that Jackson-facing type; otherwise retitle or remove this no-op production change and keep only a regression test for the real failure.
| public void testSerdeRoundTrip() throws Exception | ||
| { | ||
| final DefaultObjectMapper mapper = new DefaultObjectMapper(); | ||
| final String serialized = mapper.writeValueAsString(longLastAggFactory); |
There was a problem hiding this comment.
[P2] Round-trip tests do not exercise the changed NIL classes
Finding: Both added assertions serialize longLastAggFactory, not the Aggregator or BufferAggregator returned for a missing column, so the JSON only contains factory properties and would be identical before this refactor. The only missing-column test calls the on-heap factorize path for LongLast; the five other factories and all six NIL_BUFFER_AGGREGATOR paths remain uncovered. This lets regressions in the changed classes pass unnoticed and does not validate the claimed fix.
Suggestion: Add a focused test for the actual Jackson-facing value and cover the buffer and remaining factory variants, or narrow the production change to code that the test demonstrably protects.
Purpose
Fix Jackson serialization failure for Long/Double/Float First/Last aggregator factories.
Anonymous inner classes used as
NIL_AGGREGATORandNIL_BUFFER_AGGREGATORwere being serializedas
LongLastAggregatorFactory$1instead of the registered@JsonTypeName. This caused the error:Changes
Converted all anonymous subclasses of
AggregatorandBufferAggregatorin the following 6 factoryclasses to private static named inner classes:
LongLastAggregatorFactoryDoubleLastAggregatorFactoryFloatLastAggregatorFactoryLongFirstAggregatorFactoryDoubleFirstAggregatorFactoryFloatFirstAggregatorFactoryEach factory previously had two anonymous classes (
NIL_AGGREGATORandNIL_BUFFER_AGGREGATOR)that overrode
aggregate()to be no-ops. These are now named inner classes (e.g.,NilLongLastAggregator,NilLongLastBufferAggregator) that correctly extend their parent classeswith explicit constructors.
Closes #7599