Is your feature request related to a problem or challenge?
variant_kernels.rs benchmarks json_to_variant and variant_get, but nothing covers the other direction, so changes to variant_to_json perf go unnoticed.
#10810 rewrote it on top of VariantArray so it could handle shredded input and dictionary/run-end-encoded metadata. Good change, but the row loop now goes through try_value(i), which re-matches the data type and re-downcasts both the metadata and value columns on every row — the old loop hoisted those out. On my machine (M4 Pro) that's ~1.4x slower on batches of scalars, where there's barely any JSON to write and the dispatch dominates, and ~1.1x on objects.
Describe the solution you'd like
Two things, and the first is worth doing on its own:
Add variant_to_json benchmarks to variant_kernels.rs — scalars, small objects, wide objects, plus a shredded array and one with dictionary metadata.
Use them to decide how far to take the fast path.
Most of the regression comes back by resolving the column dispatch once and keeping try_value only as the fallback for shredded and encoded-metadata arrays — that gets to ~1.05x. The rest is a per-row enum match that would need monomorphizing over the concrete array types, which may or may not be worth the code.
Describe alternatives you've considered
No response
Additional context
Whatever lands needs to keep try_value's edge cases: a null value in an unshredded row is Variant::Null rather than an error, and a null metadata row that isn't masked by the struct's null buffer must stay an error instead of feeding garbage to Variant::new.
Is your feature request related to a problem or challenge?
variant_kernels.rs benchmarks json_to_variant and variant_get, but nothing covers the other direction, so changes to variant_to_json perf go unnoticed.
#10810 rewrote it on top of VariantArray so it could handle shredded input and dictionary/run-end-encoded metadata. Good change, but the row loop now goes through try_value(i), which re-matches the data type and re-downcasts both the metadata and value columns on every row — the old loop hoisted those out. On my machine (M4 Pro) that's ~1.4x slower on batches of scalars, where there's barely any JSON to write and the dispatch dominates, and ~1.1x on objects.
Describe the solution you'd like
Two things, and the first is worth doing on its own:
Add variant_to_json benchmarks to variant_kernels.rs — scalars, small objects, wide objects, plus a shredded array and one with dictionary metadata.
Use them to decide how far to take the fast path.
Most of the regression comes back by resolving the column dispatch once and keeping try_value only as the fallback for shredded and encoded-metadata arrays — that gets to ~1.05x. The rest is a per-row enum match that would need monomorphizing over the concrete array types, which may or may not be worth the code.
Describe alternatives you've considered
No response
Additional context
Whatever lands needs to keep try_value's edge cases: a null value in an unshredded row is Variant::Null rather than an error, and a null metadata row that isn't masked by the struct's null buffer must stay an error instead of feeding garbage to Variant::new.