Skip to content

fix(storage): compress permanent construction Parquet with Zstd - #1214

Merged
DecisionNerd merged 3 commits into
mainfrom
fix/1202-parquet-zstd
Sep 10, 2026
Merged

DecisionNerd merged 3 commits into
mainfrom
fix/1202-parquet-zstd

Conversation

@DecisionNerd

@DecisionNerd DecisionNerd commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor

Construction published uncompressed topology, properties and runtime catalogs. Use Zstd level 1 for these permanent payloads, preserving schema, dictionary/row-group defaults, hashing, cache release and durable publication. Runtime catalog encoding during shaping uses the same policy because those bytes become permanent.

Random-ID facade fixtures now publish 4,015,814 / 7,845,372 / 6,398,839 Parquet bytes, each below 80% of its same-layout uncompressed control. Exact reopen/query/export/full-verify/clean-import oracles pass. Source-bound raw allocation, timing, RSS and I/O evidence is committed. The lifecycle peak test verifies shaping-bound ties against the compressed control and strict improvement against the frozen uncompressed baseline.

Validation: 99 construction tests, 6 API resumable tests, 29 ladder tests and all 5 permanent-storage tests pass. Workspace clippy, formatting, make pre-push-fast and make gate-registry-check pass. Independent review found no blocking issues. Full pre-push on the preceding lifecycle work ran native/binding suites but failed the existing coverage floor tracked by #360; no full-pre-push green claim is made here.

Permanent encoding policy across replay, mutation, compaction, projections and other production publishers is separately required by #1213, including per-path resource evidence. This PR remains construction-scoped.

Closes #1202


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

@coderabbitai

coderabbitai Bot commented Sep 10, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: df14b293-c7ef-47c0-8dbd-ea871528ef4c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Warning

Billing warning: we have not been able to collect payment for this subscription for more than 72 hours. Please update the payment method or pay any pending invoices in Billing to avoid service interruption.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added core Core source code changes documentation Improvements or additions to documentation labels Sep 10, 2026
@DecisionNerd
DecisionNerd merged commit 2acc568 into main Sep 10, 2026
23 checks passed
@DecisionNerd
DecisionNerd deleted the fix/1202-parquet-zstd branch September 10, 2026 01:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core source code changes documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(storage): compress permanent construction Parquet with Zstd

1 participant