[FR] macOS & Apple Silicon Support

[jonpryor]

At WWDC 2020, Apple announced that macOS will be migrating to "Apple Silicon" over the next couple of years. While Rosetta 2 will exist as an immediate solution to allow the execution of existing x86_64 code on devices with Apple Silicon CPUs, the mid-to-long term plan should be to distribute all native binaries as Universal 2 binaries: executables which contain both x86_64 and Apple Silicon slices:

https://developer.apple.com/documentation/xcode/building_a_universal_macos_binary

All native binaries in the Android SDK & NDK, such as clang and the linker, presumably need to be built as Universal 2 binaries.

Are there any plans to distribute the NDK as Universal 2 binaries?

Is your feature request related to a problem? Please describe.

Apple will be migrating macOS to a new hardware architecture. It would be nice if it were properly supported before x86 support is dead. (Admittedly not "soon"…)

Describe the solution you'd like

Ship NDK binaries as Universal 2 binaries, containing both x86_64 and Apple Silicon support.

Describe alternatives you've considered

Crying.

Components that need to be made fat

• M1 support in ndk-build script #1410
• M1 LLVM toolchain #1544
• M1 CMake #1545
• M1 Make #1546
• M1 Python #1547
• M1 Ninja #1548
• M1 yasm #1549
• M1 shader-tools #1577
• M1 simpleperf #1584

Won't fix:

• GDB
• GAS
• renderscript

[DanAlbert]

Thanks for filing. We've been planning for this since the rumors started being published, but hadn't filed a tracking bug.

We're working on getting some of the development devices. We also need to get buildbots that have a new enough xcode (which could possibly require a new version of macOS, I don't know). Some test infra work probably needs to be done, since I'm assuming that the new OS will be cross compiled for rather than natively compiled for.

After we have all the infrastructure we can finally start actually working on this. There may be some porting work, but for the most part we're shipping third-party software and will be dependent on those projects being ready.

In other words, there's a pretty long tail of work here, but we have started on it. There's still a non-zero chance that we'll need to rely on Rosetta at first. There is an outstanding question of whether or not Rosetta will support bare executables or just app bundles.

Crying.

Same.

[alexcohn]

The chances are pretty high that x86_64 NDK will run decently in Rosetta-2 (translation) mode even on the ARM hardware.

[jonpryor]

We can confirm that the x86_64 NDK tools that Xamarin.Android uses run on Rosetta 2 on the Apple DTK. I can't say how "well" they run, but they do run, and we're able to use Rosetta 2-translated SDK & NDK utilities to build & install a Xamarin.Android application.

[DanAlbert]

Great, thanks for that info. Just to check, they run outside your app bundle as well?

[jonpryor]

Great, thanks for that info. Just to check, they run outside your app bundle as well?

No, we re-package and redistribute bits of the Android NDK toolchain such that they are inside our .pkg, e.g.:

/Library/Frameworks/Xamarin.Android.framework/Versions/10.2.0.100/lib/xamarin.android/xbuild/Xamarin/Android/Darwin/ndk/aarch64-linux-android-ld

I don't have the DTK machine, so I can't verify that they run outside of our default installation location. I don't see why they wouldn't…

[DanAlbert]

I would hope they do as well, but so far I haven't seen anything definitive about whether Rosetta will work for plain executables outside an app package. Our recent experience with notarization has me worried that it won't.

We should have our hands on the hardware soon enough and be able to find out though.

[enh-google]

going back to the original question:

Are there any plans to distribute the NDK as Universal 2 binaries?

we're still thinking about whether we'll have universal binaries or two separate NDKs for darwin/x86-64 and darwin/arm64, so if you have a strong argument why you'd rather have universal binaries than a separate NDK, let us know while we/Studio are still trying to decide! we don't have any timeline to announce yet, but we have started work on building everything for darwin/arm64. (and in the meantime, everything works with Rosetta2.)

[alexcohn]

If you ask me, I'd prefer the smaller separate packages. These days, we mostly download NDK via SDK Manager, and it should know well which flavor of NDK is correct for the given machine.

BTW, same applies to Windows and Linux.

[oovz]

(and in the meantime, everything works with Rosetta2.)

How do you run ndk using rosetta2?

[DanAlbert]

Nothing special I don't think. The whole point of Rosetta is to allow binaries to run as if nothing were different.

[enh-google]

yeah, that's my understanding too (and how it was for "classic" Rosetta back in the PowerPC transition days). @kongy has actually tried it...

[osrl]

hi, I get this error while trying to build a project with ndk

Error while executing process /Users/osrl/Library/Android/sdk/ndk/21.3.6528147/ndk-build with arguments {NDK_PROJECT_PATH=null APP_BUILD_SCRIPT=/Users/osrl/Documents/Android Studio/hesabim-android/app/src/main/jni/Android.mk NDK_APPLICATION_MK=/Users/osrl/Documents/Android Studio/hesabim-android/app/src/main/jni/Application.mk APP_ABI=armeabi-v7a NDK_ALL_ABIS=armeabi-v7a NDK_DEBUG=1 APP_PLATFORM=android-18 NDK_OUT=/Users/osrl/Documents/Android Studio/hesabim-android/app/build/intermediates/ndkBuild/debug/obj NDK_LIBS_OUT=/Users/osrl/Documents/Android Studio/hesabim-android/app/build/intermediates/ndkBuild/debug/lib APP_SHORT_COMMANDS=false LOCAL_SHORT_COMMANDS=false -B -n}
ERROR: Unknown host CPU architecture: arm64

I also tried to build with arch -x86_64 but no luck. How does "everything work with Rosetta2"? Can you help @enh-google?