Skip to content

View's VersionLog uses wrong timestamp if ViewVersions are reused聽#12809

Closed

Description

@c-thiel

Apache Iceberg version

1.8.1 (latest release)

Query engine

Spark

Please describe the bug 馃悶

Lets assume the following changes:

  1. ViewVersion 1 is added and set active
  2. View Version 2 is added and set active
  3. View Version 1 is set active.

This resulted in the following version_log:

    "version-log": [
        {
            "version-id": 1,
            "timestamp-ms": 1744716416168
        },
        {
            "version-id": 2,
            "timestamp-ms": 1744716449754
        },
        {
            "version-id": 1,
            "timestamp-ms": 1744716416168
        }
    ],

Note that the last and first entries have the same timestamp, namely the timestamp of the initial ViewVersion creation.

I believe this is undesired in a history, where we are interested when a certain change became active. It makes sense to use the exact timestamp of the ViewVersion if it was added in the same set of changes, but re-enabling a previously used view version (maybe years ago) should not add a history for this past timestamp.

Fix in Rust: apache/iceberg-rust#1218

The problematic line in Java is here:

.timestampMillis(version.timestampMillis())

Willingness to contribute

  • I can contribute a fix for this bug independently
  • I would be willing to contribute a fix for this bug with guidance from the Iceberg community
  • I cannot contribute a fix for this bug at this time

Activity

  1. RussellSpitzer commented on Apr 16, 2025

    @RussellSpitzer
    Member

    I agree this is a bug but i'm not sure about the fix in the rust lib. Maybe I haven't looked into this enough but the time of the creation of the metadata object seems wrong? Shouldn't it be the commit time? If that's just what we are doing in any case then I guess it's fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions