Skip to content

Fallback for skip_page/peek_page without OffsetIndex in SerializedPag… - #2490

Closed
thinkharderdev wants to merge 3 commits into
apache:masterfrom
coralogix:issue-2459
Closed

Fallback for skip_page/peek_page without OffsetIndex in SerializedPag…#2490
thinkharderdev wants to merge 3 commits into
apache:masterfrom
coralogix:issue-2459

Conversation

@thinkharderdev

@thinkharderdev thinkharderdev commented Aug 17, 2022

Copy link
Copy Markdown
Contributor

…eReader

Which issue does this PR close?

Closes #2459
Closes #2434

Rationale for this change

Need to be able to peek/skip pages even if we do not have an OffsetIndex

What changes are included in this PR?

Implement skip_next_page and peek_next_page in SerializedPageReader

Are there any user-facing changes?

@github-actions github-actions Bot added the parquet Changes to the parquet crate label Aug 17, 2022
}

/// Given an offset index, return the offset ranges for all data pages selected by `self`
#[allow(unused)]

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clippy was unhappy because this method is only used with the async feature

Ok(result)
}

#[allow(clippy::large_enum_variant)]

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Boxing buffered_header seemed counterproductive here so I just disabled the lint warning

@tustvold

Copy link
Copy Markdown
Contributor

I believe @Ted-Jiang is already working on this in #2460... Perhaps we could keep some of the tests from this?

@thinkharderdev

Copy link
Copy Markdown
Contributor Author

I believe @Ted-Jiang is already working on this in #2460... Perhaps we could keep some of the tests from this?

Ah, sorry about that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

parquet Changes to the parquet crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support SerializedPageReader::skip_page without OffsetIndex Error Reading Page Index When Not Available

2 participants