fix: emit array-of-primitive entries in JSONToMap flatten mode#5045
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #5045 +/- ##
============================================
- Coverage 43.06% 42.87% -0.20%
+ Complexity 2211 2207 -4
============================================
Files 1045 1045
Lines 40216 40068 -148
Branches 4243 4237 -6
============================================
- Hits 17321 17181 -140
+ Misses 21829 21816 -13
- Partials 1066 1071 +5
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
/request-review @aglinxinyuan |
There was a problem hiding this comment.
Pull request overview
This PR fixes JSONUtils.JSONToMap so that, in flatten=true mode, primitive elements inside arrays are no longer silently dropped. The implementation is brought into alignment with the method’s docstring example (e.g., {"E":["X","Y"]} → Map("E1" -> "X", "E2" -> "Y")), and the unit tests are updated to assert the intended contract.
Changes:
- Emit
parentName -> node.asText()when the recursive flattening reaches a value node with a non-emptyparentName. - Update existing specs that previously pinned the buggy behavior to now assert the documented output.
- Add a new spec covering a mixed array containing both objects and primitives.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| common/workflow-core/src/main/scala/org/apache/texera/amber/util/JSONUtils.scala | Adds a value-node emission branch so primitives inside arrays are included during flatten recursion. |
| common/workflow-core/src/test/scala/org/apache/texera/amber/util/JSONUtilsSpec.scala | Updates/extends tests to validate flattening arrays of primitives (including top-level arrays) and mixed arrays. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
What changes were proposed in this PR?
JSONUtils.JSONToMapwithflatten=truesilently dropped primitive elements that lived inside an array. The docstring's worked example claimed{"E":["X","Y"]}would flatten to{"E1":"X","E2":"Y"}, but the implementation only emitted entries from inside itsisObject/isArraybranches, so a recursive call landing on a value node returned an empty mapand the data vanished. This PR adds a third top-level branch that emitsparentName -> node.asText()wheneverJSONToMapis called on a value node with a non-emptyparentName, bringing the behavior into line with the docstring. The twoJSONUtilsSpeccases that previously pinned the buggy behavior (with comments inviting exactly this tightening) are flipped to assert the documented contract, and a new case covers a mixed array of objects and primitives.Any related issues, documentation, or discussions?
Closes: #4729
How was this PR tested?
JSONUtilsSpec: flipped the two pinning cases (array-of-primitives, top-level array of primitives) to assert the documented output, added a mixed-array case.sbt "WorkflowCore/testOnly org.apache.texera.amber.util.JSONUtilsSpec"passes (15/15).sbt scalafmtAllis clean.Was this PR authored or co-authored using generative AI tooling?
Co-authored with Claude Opus 4.7 in compliance with ASF