[Feature](lance) Support time travel and REST managed versioning #66491

Parent

Tracked by #66340. The baseline Lance read integration was introduced by #65730.

Completion status

Updated on 2026-09-29. Implementation #68453 and documentation apache/doris-website#4172 have both merged into their respective target branches (Doris branch-4.1 and the 4.x documentation). This records implementation delivery, not availability in an already published release.

The four functional scope items below are implemented for catalog table scans under the documented read contract. Search TVF snapshot selection is tracked separately in #68625.

Scope

• Support time travel and explicit dataset version selection.
• Support REST Namespace managed versioning in the BE reader.
• Keep schema binding, fragment planning, and scan execution on the selected version.
• Define errors for missing, expired, or inaccessible versions.

Completion criteria

• FE and BE implementation is complete.
• Unit and regression tests cover explicit versions, timestamps, and REST-managed versions.
• User documentation describes syntax, semantics, and limitations.
• The capability is verified with representative filesystem and REST Namespace datasets.

Delivered behavior

• FOR VERSION AS OF and FOR TIME AS OF select catalog snapshots; @tag(...) and @branch(...) are also supported, including explicit versions and timestamps within a branch.
• For managed tables, the namespace determines the visible versions. FE resolves the selected version and passes the dataset URI, version, and storage options to the existing BE reader; this did not require a new BE/thrift implementation.
• Each table reference uses its selected snapshot consistently for schema binding, fragment planning, predicate pushdown, metadata COUNT, and BE scanning. Separate references can intentionally select different versions; independently resolved branch heads are not a statement-wide snapshot-isolation guarantee.
• Missing versions, tags and branches, invalid selectors, empty managed histories, and unreadable versions report explicit errors. For storage-versioned tables, a version removed by cleanup cannot be distinguished from one that never existed.
• Filesystem and REST time-travel regression suites passed on the reviewed implementation head 5a682a80bbcafa1b2d709d7830335ec007b866cb; the PR's External Regression and FE UT checks subsequently completed successfully.

Documented boundaries

• Managed reads require the selected manifest to be available at its canonical location. A namespace record pointing to the adjacent staged path is accepted when the canonical manifest exists. Doris does not finalize incomplete commits or expose staged-only versions; these reads fail explicitly. This is a read-only boundary, not a requirement to read uncommitted data.
• Arbitrary manifest layouts and managed s3+ddb access are outside the supported read contract. Timestamp selection follows the documented history-gap/cleanup rules.
• vector_search() and full_text_search() still select the latest main snapshot; version/time/tag/branch selection for these functions is a separate enhancement: [Feature](lance) Support snapshot selection in vector and full-text search #68625.
• Historical/tag/branch index inspection is also not provided by this change and is outside [Feature](lance) Support snapshot selection in vector and full-text search #68625's search-only scope.

Remaining validation follow-up

• Re-enable LanceManagedVersioningTest after resolving its documented Arrow C Data JNI / CI runtime compatibility prerequisite, and run its native managed-versioning edge cases in CI. The class is currently disabled; successful enabled unit and external regression suites do not imply that these disabled cases ran.