Skip to content

[GLUTEN-12456][BOLT] Add bolt backend in gluten - #12454

Merged
taiyang-li merged 48 commits into
apache:mainfrom
taiyang-li:add_bolt_backend_new
Sep 10, 2026
Merged

taiyang-li merged 48 commits into
apache:mainfrom
taiyang-li:add_bolt_backend_new

Conversation

@taiyang-li

@taiyang-li taiyang-li commented Jul 6, 2026 •

Copy link
Copy Markdown
Contributor

What changes are proposed in this pull request?

Add Bolt backend in Gluten.

This PR depends on the following common patches first:

For more details, please refer to #12456.

How was this patch tested?

WIP stage.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: TraeCli openrouter-3o

@taiyang-li
taiyang-li force-pushed the add_bolt_backend_new branch from 23d492b to 8d197c4 Compare July 6, 2026 09:47
@github-actions github-actions Bot added CORE works for Gluten Core BUILD VELOX DOCS labels Jul 6, 2026
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

1 similar comment
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@taiyang-li taiyang-li changed the title [WIP] try to add bolt backend in gluten [WIP] add bolt backend in gluten Jul 6, 2026
@taiyang-li
taiyang-li force-pushed the add_bolt_backend_new branch from 8d197c4 to 2b6f49a Compare July 6, 2026 11:14
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@taiyang-li
taiyang-li force-pushed the add_bolt_backend_new branch from 2b6f49a to e04cc1c Compare July 6, 2026 11:21
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@taiyang-li
taiyang-li force-pushed the add_bolt_backend_new branch from e04cc1c to 9cb20f4 Compare July 6, 2026 11:26
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@taiyang-li
taiyang-li force-pushed the add_bolt_backend_new branch from f74ab91 to a148383 Compare July 8, 2026 09:19
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

4 similar comments
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@taiyang-li taiyang-li added BOLT and removed VELOX labels Jul 9, 2026
@github-actions github-actions Bot added the VELOX label Jul 10, 2026
@github-actions

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

1 similar comment
@github-actions

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions github-actions Bot removed the INFRA label Sep 7, 2026
@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@taiyang-li taiyang-li changed the title [WIP] add bolt backend in gluten [GLUTEN-12456][BOLT] Add bolt backend in gluten Sep 8, 2026
@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

The cpp tests had not been built since BUILD_TESTS was never enabled in CI,
so several things had rotted:

- cpp/bolt/tests/CMakeLists.txt was mangled by the C++ formatter run in
  b436c03 (source names split like `MemoryManagerTest .cc`); restore it.
- Core tests linked bolt::bolt although they include no Bolt headers, and the
  target does not even exist yet when core/ is configured; drop it. Link glog
  before gluten so the test resolves gflags from the static archives instead
  of the copy exported by libgluten.so.
- Bolt tests use -export-dynamic; hide the statically linked gflags with
  --exclude-libs so libgluten.so's own glog does not register its flags into
  the executable's registry a second time.
- MemoryManagerTest / RuntimeTest were out of date with the 3-argument
  BoltMemoryManager constructor, Metrics(unsigned, std::string) and the
  createShuffleReader signature.
- CreateBoltRuntime runs without a JVM, so keep spark.gluten.useBoltMemoryManager
  off there; enabling it makes BoltRuntime fetch the task attempt id via JNI.

The shared-library build no longer runs `cmake --target install`: nothing
consumes /usr/local/lib/libgluten.so, and Maven packages the natives from
cpp/build/package/bolt/<platform>/<arch>/ which the build stages itself. The
staging step now also drops the three libraries into cpp/build/releases/ so
that directory keeps working without an install step.
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86

@taiyang-li
taiyang-li merged commit 53ad31b into apache:main Sep 10, 2026
60 checks passed
@prestodb-ci

prestodb-ci commented Sep 10, 2026 •

Copy link
Copy Markdown

Rebase job

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants