[Variant] Match Spark physical Parquet semantics for projected Variant columns

[peterxcli]

Problem

Arrow-rs and Spark can derive different Arrow types and name matches from the same Parquet file. For projected Variant storage this can change the reconstructed value, reject a file Spark reads, or select the wrong physical column.

Parent: #5546
Depends on: #5549
Cleanup trackers: #5474, #5477, #5495

Solution

At the projected, explicitly marked Variant boundary only:

• Prevent advisory ARROW:schema types from overriding Spark's physical Parquet interpretation.
• Preserve Parquet ENUM as a Variant string while leaving raw BINARY as binary.
• Match Spark's physical decimal, date/timestamp, fixed binary/list, and encoded child behavior needed by whole-value reconstruction.
• Preserve row-group metadata and page/offset indexes when rebuilding reader metadata.
• Keep encrypted affected scans on Spark when reader metadata cannot be reconstructed safely.
• Match Spark field-ID precedence and fail closed for unresolved Unicode/name ambiguity without duplicating the generic work in Support Spark-compatible Unicode case-insensitive Parquet field matching #5495.

The JVM Variant scan gate remains closed in this task.

Tests

Use generated Parquet inputs rather than committed fixtures for:

• encoded storage children and supported unsigned widths;
• timestamp, fixed binary, and fixed-size-list storage;
• positive and negative wide DECIMAL values;
• ENUM versus UTF8 versus raw BINARY;
• advisory Arrow-schema hints;
• field-ID shadowing and Unicode/name ambiguity;
• encrypted fallback; and
• pruning that does not activate compatibility work for an unread Variant column.

Each compatibility branch must link its upstream owner and the corresponding cleanup tracker.

[peterxcli]

take