Skip to content

Update build.yml to support Linux ARM 64 on Github CI#441

Closed
metesynnada wants to merge 2 commits into
apache:mainfrom
metesynnada:ci/linux-arm-64-emulation
Closed

Update build.yml to support Linux ARM 64 on Github CI#441
metesynnada wants to merge 2 commits into
apache:mainfrom
metesynnada:ci/linux-arm-64-emulation

Conversation

@metesynnada
Copy link
Copy Markdown

Which issue does this PR close?

Closes #426

Rationale for this change

The need for this change arises due to the prolonged build times experienced with linux arm64 docker, especially on macOS during the prototyping phase.

What changes are included in this PR?

In this PR, an emulator for Linux arm64 is included, as GitHub does not natively provide one.

BTW, I am not sure how to test this, if a PMC could help, that would be perfect.

Are there any user-facing changes?

No

@metesynnada metesynnada marked this pull request as draft July 27, 2023 07:30
@metesynnada metesynnada force-pushed the ci/linux-arm-64-emulation branch from 1cb0490 to eb309ef Compare July 27, 2023 07:38
@metesynnada metesynnada force-pushed the ci/linux-arm-64-emulation branch from eb309ef to 4d5fedb Compare July 27, 2023 07:39
@metesynnada metesynnada force-pushed the ci/linux-arm-64-emulation branch from 6cc843d to 62c9637 Compare July 28, 2023 14:23
@gokselk
Copy link
Copy Markdown
Contributor

gokselk commented Aug 1, 2023

I made an attempt to address this, but it seems that manylinux2014-cross relies on GCC 4.8.5, causing compilation issues with the libmimalloc-sys crate. At the moment, I haven't found a straightforward solution to this problem. However, I will closely monitor the status of manylinux2014-cross and plan to submit a pull request once they update GCC to version 4.9 or higher. It's worth noting that the compilation occurs inside the manylinux container, so installing dependencies before building won't resolve the current issue. Any suggestions or assistance on this matter would be greatly appreciated.

@gokselk
Copy link
Copy Markdown
Contributor

gokselk commented Aug 1, 2023

I've made several attempts to build the aarch64 wheel using different approaches:

  1. manylinux-cross 2014: Unfortunately, this build failed because it relies on GCC 4.8.5, which isn't sufficient to build some of the required dependencies.

  2. manylinux-cross 2_28: Good news! The build was successful, and it took only 30 minutes.

  3. manylinux 2014 with QEMU: The build process here took a substantial 5 hours and is still ongoing.

Based on the results, it seems that the most feasible solution for building the Linux aarch64 wheel, at least for now, is to use manylinux 2_28. I'll create another PR to discuss this further and explore other potential solutions.

@andygrove
Copy link
Copy Markdown
Member

@metesynnada The PR that @gokselk mentioned (#443) has now been merged

@metesynnada
Copy link
Copy Markdown
Author

@metesynnada The PR that @gokselk mentioned (#443) has now been merged

Acknowledged. Proceeding to close the PR now. Thank you for the reminder. I appreciate the effort you put in, @gokselk.

@metesynnada metesynnada closed this Aug 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GitHub Workflow should build DataFusion wheel for Linux aarch64

3 participants