Repository navigation
fix(storage): compress permanent construction Parquet with Zstd - #1214
Conversation
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Warning Billing warning: we have not been able to collect payment for this subscription for more than 72 hours. Please update the payment method or pay any pending invoices in Billing to avoid service interruption. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Construction published uncompressed topology, properties and runtime catalogs. Use Zstd level 1 for these permanent payloads, preserving schema, dictionary/row-group defaults, hashing, cache release and durable publication. Runtime catalog encoding during shaping uses the same policy because those bytes become permanent.
Random-ID facade fixtures now publish 4,015,814 / 7,845,372 / 6,398,839 Parquet bytes, each below 80% of its same-layout uncompressed control. Exact reopen/query/export/full-verify/clean-import oracles pass. Source-bound raw allocation, timing, RSS and I/O evidence is committed. The lifecycle peak test verifies shaping-bound ties against the compressed control and strict improvement against the frozen uncompressed baseline.
Validation: 99 construction tests, 6 API resumable tests, 29 ladder tests and all 5 permanent-storage tests pass. Workspace clippy, formatting, make pre-push-fast and make gate-registry-check pass. Independent review found no blocking issues. Full pre-push on the preceding lifecycle work ran native/binding suites but failed the existing coverage floor tracked by #360; no full-pre-push green claim is made here.
Permanent encoding policy across replay, mutation, compaction, projections and other production publishers is separately required by #1213, including per-path resource evidence. This PR remains construction-scoped.
Closes #1202
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.