Repository navigation
Cross-compile NSS for macOS instead of fetching NSS CI's build - #7814
rvandermeulen wants to merge 3 commits into
Conversation
6fda583 to
dc4bba2
Compare
bendk
left a comment
There was a problem hiding this comment.
This is pretty interesting. I'm feeling a combination of fear and excitement. On the one hand I'd love to be able to build NSS ourselves. On the other hand, there's a good deal of complexity in this code that I don't want to have to maintain. On the third hand, I guess if it builds it builds and the CI is green.
As a compromise, I'd say try to cut down some of the big sources of complexity. Is there some package we can download that would eliminate/simplify macpkg.py? Could the NSS team review/take the patch? Even if we could only do one of those I think it would make a big impact. @mhammond what's your opinion on this?
|
To be clear, this is very much PoC level at the moment. I absolutely plan to get the applicable parts upstreamed into NSS first before trying to do anything in this repo. I also want to see about getting the right scopes to access internal tooltool artifacts from this repo. Doing both of those should eliminate the vast majority of what's currently living on this stack. I just needed it all in one place to be able to test in this repo. |
|
And https://bugzilla.mozilla.org/show_bug.cgi?id=2095405 is the NSS bug I've filed. |
1d7af45 to
2ddc921
Compare
Ubuntu 22.04's gyp package is the old Chromium gyp, whose Xcode emulation breaks on Python 3 bytes output and reports "No Xcode or CLT version detected!" for the macOS cross-compile. Ubuntu only ships gyp-next from 25.04.
The desktop-macos toolchain and the module builds now fetch the MacOSX27.2.sdk that mozilla-central's macosx64-sdk-27.2 toolchain task publishes, instead of MacOSX11.0.sdk from tooltool. The 11.0 SDK has no libc++ headers, which an NSS cross-compile needs for mozpkix, and Gecko's clang doesn't ship them. cross-compile-macos-on-linux-desktop-libs.sh only fetched that SDK and has no callers, so it goes with the manifest. The SDK is a private artifact, and fxci-config grants queue:get-artifact:project/gecko/mac-sdk/* only to trusted jobs. Pull requests from non-collaborators therefore run the module builds without the darwin Rust target and the macOS libs, and don't schedule the desktop-macos toolchain. android-build sources cross-compile-setup.sh but builds no darwin target, so the SDK fetch is a separate script that only the darwin builds run. The x86_64 Rust build now targets macOS 11.0 rather than Rust's default of 10.12, the oldest release the 27.2 SDK's libc++ headers support, matching the NSS build.
NSS's build.sh has no macOS target for a non-macOS host, so libs/macos-cross-compile-nss.patch, a copy of the fix for bug 2095405 that ships in NSS 3.135, adds --target-os=macos, which builds with clang, the LLVM tools, lld and an unpacked SDK, and shims the libtool, nm, otool, xcodebuild and xcrun calls of gyp's ninja-mac generator. build-nss-desktop.sh then runs build.sh with it against the SDK that fetch-macos-sdk.sh unpacks into MACOS_SDK, and the fetch kind, whose only task was nss-artifact, goes away. check-nss-libs.sh now runs on the result too, with llvm-nm as on Android. The macOS libraries now get the same OpenSSL symbol renames as the other platforms, and target macOS 11.0, the oldest release the SDK's libc++ headers support, rather than the 15.0 of NSS CI's build.
2ddc921 to
9307aca
Compare
|
Both of those are done now, and the PR is a lot smaller for it:
What I've checked on the output of this push, compared with main:
None of that covers actually running it on a Mac, and I don't have one. @bendk, could you review this and verify it locally on your Mac? Two things would cover it:
|
NSS's build.sh cross-compiles for Android and iOS but has no macOS target for a non-macOS host, so the desktop-macos toolchain task links NSS CI's prebuilt build-mac-x64/opt-static dist, pinned by task id, sha256 and size, and repinned on every NSS bump. This builds it instead: libs/macos-cross-compile-nss.patch, a copy of the fix for bug 2095405 that ships in NSS 3.135, adds --target-os=macos to build.sh (clang, the LLVM tools and lld against an unpacked SDK, with shims for the libtool, nm, otool, xcodebuild and xcrun calls that gyp's ninja-mac generator makes). build-nss-desktop.sh runs it, and the fetch kind, whose only task was nss-artifact, goes away. The patch drops out again with the NSS 3.135 bump.
The tooltool MacOSX11.0.sdk has no libc++ headers, which the NSS build needs for mozpkix, so the macOS builds now fetch the MacOSX27.2.sdk that mozilla-central's macosx64-sdk-27.2 toolchain task publishes. It is a private artifact, and mozilla-releng/fxci-config#1418 grants it to trusted jobs only, so pull requests from non-collaborators run the module builds without the darwin target and the macOS libs, and don't schedule the desktop-macos toolchain.
The macOS libraries now get the same OpenSSL symbol renames as the other platforms, and both they and the Rust build target macOS 11.0, the oldest release the 27.2 SDK's libc++ headers support, rather than the 15.0 of NSS CI's build and Rust's default of 10.12.
Pull Request checklist