Skip to content

docs: [branch-1.1] regenerate release docs for 1.1.0-rc2 - #6568

Merged
andygrove merged 1 commit into
apache:branch-1.1from
andygrove:release-docs-branch-1-1-rc2
Oct 3, 2026
Merged

andygrove merged 1 commit into
apache:branch-1.1from
andygrove:release-docs-branch-1-1-rc2

Conversation

@andygrove

Copy link
Copy Markdown
Member

Which issue does this PR close?

N/A. This reruns the "Generate Release Documentation" step of the 1.1.0 release (#5327) for rc2. The first run, #6344, came before the backports that merged after rc1.

Rationale for this change

A release branch commits the generated docs under docs/source/user-guide/latest/, so the archived 1.1.0 docs render real tables. Some backports since #6344 change what the generator writes. #6488 added a config key, and #6489 and #6561 made the regr_* aggregates, array_distinct and array_union fall back to Spark by default. Without this, the 1.1.0 docs would leave out the new key and the reasons those expressions fall back.

What changes are included in this PR?

Output of ./dev/generate-release-docs.sh on branch-1.1 at 4f5cf2d, with no hand edits:

The four Spark versions get the same additions.

How are these changes tested?

  • ./dev/generate-release-docs.sh completed for all four Spark profiles on JDK 17.
  • npx prettier@latest --check passes on the changed files.

Output of ./dev/generate-release-docs.sh on branch-1.1 after the
backports that merged since rc1: the skipPartial config row (apache#6488),
the regr_* fallback reasons (apache#6489) and the array_distinct/array_union
fallback reasons (apache#6561) on every Spark version's compatibility pages,
and the regr_sxx/regr_syy Implementation cells back to the generator's
placeholder.
@andygrove andygrove added the documentation Improvements or additions to documentation label Oct 3, 2026

@sunchao sunchao left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

  • Prior state and problem: The release docs omitted the partial-aggregation config and compatibility warnings added by recent backports.
  • Design approach: Refresh committed documentation through the existing release-doc generator.
  • Correctness / compatibility analysis: All 28 added compatibility sections match their source declarations. Checked relevant Spark sources for 3.4.3, 3.5.9, 4.0.4 and 4.1.3, plus the cited 4.2.0 normalization rule. No introduced P1/P2 issues found within this review.
  • Key design decisions: Preserve explicit incompatible-expression opt-ins. The regr_sxx and regr_syy placeholders agree with the documented convention for expressions Spark rewrites. No new abstraction is introduced.
  • Implementation sketch: Update eight compatibility pages, add one config row and refresh two implementation cells.
  • Behavioral changes worth calling out: Documentation only, with no runtime or performance overhead. The config row accurately records the false default and potential shuffle amplification.
  • Suggested improvements: None at P1/P2 priority.

Reviewed the full ten-file diff from 4f5cf2db1e80bf94db8a4b79a870738d621f1262 to bcf1a5f6c0abb554a45436832e444b906ee62495. Confirmed the PR is not a draft. The snapshot and live discussion endpoints contain no existing reviews or comments.

Routed skills: review-comet-pr. No sibling skill applies to this documentation-only change.

Exact-head CI: Label run / Preflight, label and check-pr-title succeeded. Preflight remains in progress and Label run / Detect changes is queued. No failures reported so far.

Validation: git diff --check and focused source-to-document consistency checks passed. Full Maven doc generation, Sphinx rendering and runtime suites were not rerun. Local Prettier validation was unavailable because npx is absent. CI is not yet complete.

@andygrove
andygrove merged commit 6841609 into apache:branch-1.1 Oct 3, 2026
38 checks passed
@andygrove
andygrove deleted the release-docs-branch-1-1-rc2 branch October 3, 2026 16:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants