Cflags in openblas.pc are not compatible with Apple Clang

[dimpase]

brew gist-logs <formula> link OR brew config AND brew doctor output

HOMEBREW_VERSION: >=4.3.0 (shallow or no git repository)
ORIGIN: (none)
HEAD: (none)
Last commit: never
Branch: (none)
Core tap JSON: 01 Nov 13:19 UTC
Core cask tap JSON: 01 Feb 22:25 UTC
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_MAKE_JOBS: 10
Homebrew Ruby: 3.3.7 => /opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.3.7/bin/ruby
CPU: deca-core 64-bit arm_firestorm_icestorm
Clang: 16.0.0 build 1600
Git: 2.39.5 => /Library/Developer/CommandLineTools/usr/bin/git
Curl: 8.7.1 => /usr/bin/curl
macOS: 15.3.1-arm64
CLT: 16.2.0.0.1.1733547573
Xcode: 16.1
Rosetta 2: false

(I'm not root on the machine, so I can't meaningfully run brew doctor)
I asked a root for the output of brew gist-logs openblas, still waiting.

EDIT: the root replied with

% brew gist-logs openblas
> Error: No logs.

Verification

• My brew doctor output says Your system is ready to brew. and am still able to reproduce my issue.
• I ran brew update and am still able to reproduce my issue.
• I have resolved all warnings from brew doctor and that did not fix my problem.
• I searched for recent similar issues at https://github.com/Homebrew/homebrew-core/issues?q=is%3Aissue and found no duplicates.

What were you trying to do (and why)?

I am trying to use Homebrew's openblas with XCode C compiler, Apple's clang:

Apple clang version 16.0.0 (clang-1600.0.26.6)
Target: arm64-apple-darwin24.3.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

Openblas is located in /opt/homebrew/Cellar/openblas/0.3.29/ and its openblas.pc produces the following CFLAGS:

% PKG_CONFIG_PATH="/opt/homebrew/Cellar/openblas/0.3.29/lib/pkgconfig"  pkg-config -cflags openblas 
-I/opt/homebrew/Cellar/openblas/0.3.29/include -fopenmp

Such flags are not accepted by Apple's clang.
Specifically, on gets:

% clang -c foobar.c -fopenmp
   clang: error: unsupported option '-fopenmp'

The correct format for the OpenMP option is -Xclang -fopenmp,
as can be checked directly, or read on the R Project website: https://mac.r-project.org/openmp/

Thus, openblas.pc has to be fixed, but it's probably only opening a can of worms, as I have no idea whether
libgomp, against which libopenblas is linked, is not compatible with Apple clang's support of OpenMP.

What happened (include all command output)?

% export PKG_CONFIG_PATH="/opt/homebrew/Cellar/openblas/0.3.29/lib/pkgconfig"
% clang -c foobar.c  `pkg-config -cflags openblas`

      clang: error: unsupported option '-fopenmp'

What did you expect to happen?

It should work.

Step-by-step reproduction instructions (by running brew commands)

let `foobar.c` contain valid C code.

brew install openblas
export PKG_CONFIG_PATH=<path to libopenblas.dylib location>/pkgconfig
clang -c foobar.c  `pkg-config -cflags openblas`


One needs a recent XCode on arm64 (I tested on M1, other people tell me that on M4 it's the same).

[SMillerDev]

Homebrew only runs make for openblas, so if something is generated wrong there it should be reported upstream to be fixed.

[carlocab]

This is probably a consequence of our using GCC to build on macOS. Using -Xclang -fopenmp won't work for GCC, of course. Not convinced upstream will want to do anything about this (or that they even can).

[dimpase]

Upstream doesn't seem to be aware of a possibility to use -Xclang -fopenmp, there is nothing in their repo or github that indicates otherwise. Should they be told? Perhaps what R project is doing makes sense for openblas.

I tried building the current git version of openblas with just make, with Homebrew gfortan and Apple clang (i.e. their fake gcc), the result doesn't use openmp or gopenmp, and there is no -fopenmp in the resulting openblas.pc

I also tried to remove -fopenmp from Homebrew's openblas.pc, and the result is usable for us, it doesn't seem to affect the resulting application using Homebrew openblas in a bad way (but that's with limited testing). An openmp expert might be needed to confirm that this is OK as a solution for this bug.

[carlocab]

I tried building the current git version of openblas with just make, with Homebrew gfortan and Apple clang (i.e. their fake gcc), the result doesn't use openmp or gopenmp, and there is no -fopenmp in the resulting openblas.pc

You need to enable OpenMP for a comparable build with:

export USE_OPENMP=1

[dimpase]

I tried building the current git version of openblas with just make, with Homebrew gfortan and Apple clang (i.e. their fake gcc), the result doesn't use openmp or gopenmp, and there is no -fopenmp in the resulting openblas.pc

You need to enable OpenMP for a comparable build with:

export USE_OPENMP=1

this leads to an error as above in such a configuration, almost immediately.

EDIT2. I was slightly sloppy again. Now, with checkout of 0.3.29 (or with today's main), I get

make[1]: warning: -jN forced in submake: disabling jobserver mode.
cc -O2 -DSMALL_MATRIX_OPT -DGEMM_GEMV_FORWARD -DMAX_STACK_ALLOC=2048 -fopenmp -Wall -DF_INTERFACE_GFORT -fPIC -DSMP_SERVER -DUSE_OPENMP -DNO_WARMUP -DMAX_CPU_NUMBER=10 -DMAX_PARALLEL_NUMBER=1 -DBUILD_SINGLE=1 -DBUILD_DOUBLE=1 -DBUILD_COMPLEX=1 -DBUILD_COMPLEX16=1 -DVERSION=\"0.3.29\" -march=armv8.3-a -UASMNAME -UASMFNAME -UNAME -UCNAME -UCHAR_NAME -UCHAR_CNAME -DASMNAME=_saxpy -DASMFNAME=_saxpy_ -DNAME=saxpy_ -DCNAME=saxpy -DCHAR_NAME=\"saxpy_\" -DCHAR_CNAME=\"saxpy\" -DNO_AFFINITY -I.. -I. -UDOUBLE  -UCOMPLEX -c axpy.c -o saxpy.o
cc -O2 -DSMALL_MATRIX_OPT -DGEMM_GEMV_FORWARD -DMAX_STACK_ALLOC=2048 -fopenmp -Wall -DF_INTERFACE_GFORT -fPIC -DSMP_SERVER -DUSE_OPENMP -DNO_WARMUP -DMAX_CPU_NUMBER=10 -DMAX_PARALLEL_NUMBER=1 -DBUILD_SINGLE=1 -DBUILD_DOUBLE=1 -DBUILD_COMPLEX=1 -DBUILD_COMPLEX16=1 -DVERSION=\"0.3.29\" -march=armv8.3-a -UASMNAME -UASMFNAME -UNAME -UCNAME -UCHAR_NAME -UCHAR_CNAME -DASMNAME=_sswap -DASMFNAME=_sswap_ -DNAME=sswap_ -DCNAME=sswap -DCHAR_NAME=\"sswap_\" -DCHAR_CNAME=\"sswap\" -DNO_AFFINITY -I.. -I. -UDOUBLE  -UCOMPLEX -c swap.c -o sswap.o
cc -O2 -DSMALL_MATRIX_OPT -DGEMM_GEMV_FORWARD -DMAX_STACK_ALLOC=2048 -fopenmp -Wall -DF_INTERFACE_GFORT -fPIC -DSMP_SERVER -DUSE_OPENMP -DNO_WARMUP -DMAX_CPU_NUMBER=10 -DMAX_PARALLEL_NUMBER=1 -DBUILD_SINGLE=1 -DBUILD_DOUBLE=1 -DBUILD_COMPLEX=1 -DBUILD_COMPLEX16=1 -DVERSION=\"0.3.29\" -march=armv8.3-a -UASMNAME -UASMFNAME -UNAME -UCNAME -UCHAR_NAME -UCHAR_CNAME -DASMNAME=_scopy -DASMFNAME=_scopy_ -DNAME=scopy_ -DCNAME=scopy -DCHAR_NAME=\"scopy_\" -DCHAR_CNAME=\"scopy\" -DNO_AFFINITY -I.. -I. -UDOUBLE  -UCOMPLEX -c copy.c -o scopy.o
cc -O2 -DSMALL_MATRIX_OPT -DGEMM_GEMV_FORWARD -DMAX_STACK_ALLOC=2048 -fopenmp -Wall -DF_INTERFACE_GFORT -fPIC -DSMP_SERVER -DUSE_OPENMP -DNO_WARMUP -DMAX_CPU_NUMBER=10 -DMAX_PARALLEL_NUMBER=1 -DBUILD_SINGLE=1 -DBUILD_DOUBLE=1 -DBUILD_COMPLEX=1 -DBUILD_COMPLEX16=1 -DVERSION=\"0.3.29\" -march=armv8.3-a -UASMNAME -UASMFNAME -UNAME -UCNAME -UCHAR_NAME -UCHAR_CNAME -DASMNAME=_sscal -DASMFNAME=_sscal_ -DNAME=sscal_ -DCNAME=sscal -DCHAR_NAME=\"sscal_\" -DCHAR_CNAME=\"sscal\" -DNO_AFFINITY -I.. -I. -UDOUBLE  -UCOMPLEX -c scal.c -o sscal.o
cc -O2 -DSMALL_MATRIX_OPT -DGEMM_GEMV_FORWARD -DMAX_STACK_ALLOC=2048 -fopenmp -Wall -DF_INTERFACE_GFORT -fPIC -DSMP_SERVER -DUSE_OPENMP -DNO_WARMUP -DMAX_CPU_NUMBER=10 -DMAX_PARALLEL_NUMBER=1 -DBUILD_SINGLE=1 -DBUILD_DOUBLE=1 -DBUILD_COMPLEX=1 -DBUILD_COMPLEX16=1 -DVERSION=\"0.3.29\" -march=armv8.3-a -UASMNAME -UASMFNAME -UNAME -UCNAME -UCHAR_NAME -UCHAR_CNAME -DASMNAME=_sdot -DASMFNAME=_sdot_ -DNAME=sdot_ -DCNAME=sdot -DCHAR_NAME=\"sdot_\" -DCHAR_CNAME=\"sdot\" -DNO_AFFINITY -I.. -I. -UDOUBLE  -UCOMPLEX -c dot.c -o sdot.o
cc -O2 -DSMALL_MATRIX_OPT -DGEMM_GEMV_FORWARD -DMAX_STACK_ALLOC=2048 -fopenmp -Wall -DF_INTERFACE_GFORT -fPIC -DSMP_SERVER -DUSE_OPENMP -DNO_WARMUP -DMAX_CPU_NUMBER=10 -DMAX_PARALLEL_NUMBER=1 -DBUILD_SINGLE=1 -DBUILD_DOUBLE=1 -DBUILD_COMPLEX=1 -DBUILD_COMPLEX16=1 -DVERSION=\"0.3.29\" -march=armv8.3-a -UASMNAME -UASMFNAME -UNAME -UCNAME -UCHAR_NAME -UCHAR_CNAME -DASMNAME=_sdsdot -DASMFNAME=_sdsdot_ -DNAME=sdsdot_ -DCNAME=sdsdot -DCHAR_NAME=\"sdsdot_\" -DCHAR_CNAME=\"sdsdot\" -DNO_AFFINITY -I.. -I. -UDOUBLE  -UCOMPLEX -c sdsdot.c -o sdsdot.o
cc -O2 -DSMALL_MATRIX_OPT -DGEMM_GEMV_FORWARD -DMAX_STACK_ALLOC=2048 -fopenmp -Wall -DF_INTERFACE_GFORT -fPIC -DSMP_SERVER -DUSE_OPENMP -DNO_WARMUP -DMAX_CPU_NUMBER=10 -DMAX_PARALLEL_NUMBER=1 -DBUILD_SINGLE=1 -DBUILD_DOUBLE=1 -DBUILD_COMPLEX=1 -DBUILD_COMPLEX16=1 -DVERSION=\"0.3.29\" -march=armv8.3-a -UASMNAME -UASMFNAME -UNAME -UCNAME -UCHAR_NAME -UCHAR_CNAME -DASMNAME=_dsdot -DASMFNAME=_dsdot_ -DNAME=dsdot_ -DCNAME=dsdot -DCHAR_NAME=\"dsdot_\" -DCHAR_CNAME=\"dsdot\" -DNO_AFFINITY -I.. -I. -UDOUBLE  -UCOMPLEX -c dsdot.c -o dsdot.o
cc -O2 -DSMALL_MATRIX_OPT -DGEMM_GEMV_FORWARD -DMAX_STACK_ALLOC=2048 -fopenmp -Wall -DF_INTERFACE_GFORT -fPIC -DSMP_SERVER -DUSE_OPENMP -DNO_WARMUP -DMAX_CPU_NUMBER=10 -DMAX_PARALLEL_NUMBER=1 -DBUILD_SINGLE=1 -DBUILD_DOUBLE=1 -DBUILD_COMPLEX=1 -DBUILD_COMPLEX16=1 -DVERSION=\"0.3.29\" -march=armv8.3-a -UASMNAME -UASMFNAME -UNAME -UCNAME -UCHAR_NAME -UCHAR_CNAME -DASMNAME=_sasum -DASMFNAME=_sasum_ -DNAME=sasum_ -DCNAME=sasum -DCHAR_NAME=\"sasum_\" -DCHAR_CNAME=\"sasum\" -DNO_AFFINITY -I.. -I. -UDOUBLE  -UCOMPLEX -c asum.c -o sasum.o
cc -O2 -DSMALL_MATRIX_OPT -DGEMM_GEMV_FORWARD -DMAX_STACK_ALLOC=2048 -fopenmp -Wall -DF_INTERFACE_GFORT -fPIC -DSMP_SERVER -DUSE_OPENMP -DNO_WARMUP -DMAX_CPU_NUMBER=10 -DMAX_PARALLEL_NUMBER=1 -DBUILD_SINGLE=1 -DBUILD_DOUBLE=1 -DBUILD_COMPLEX=1 -DBUILD_COMPLEX16=1 -DVERSION=\"0.3.29\" -march=armv8.3-a -UASMNAME -UASMFNAME -UNAME -UCNAME -UCHAR_NAME -UCHAR_CNAME -DASMNAME=_ssum -DASMFNAME=_ssum_ -DNAME=ssum_ -DCNAME=ssum -DCHAR_NAME=\"ssum_\" -DCHAR_CNAME=\"ssum\" -DNO_AFFINITY -I.. -I. -UDOUBLE  -UCOMPLEX -c sum.c -o ssum.o
cc -O2 -DSMALL_MATRIX_OPT -DGEMM_GEMV_FORWARD -DMAX_STACK_ALLOC=2048 -fopenmp -Wall -DF_INTERFACE_GFORT -fPIC -DSMP_SERVER -DUSE_OPENMP -DNO_WARMUP -DMAX_CPU_NUMBER=10 -DMAX_PARALLEL_NUMBER=1 -DBUILD_SINGLE=1 -DBUILD_DOUBLE=1 -DBUILD_COMPLEX=1 -DBUILD_COMPLEX16=1 -DVERSION=\"0.3.29\" -march=armv8.3-a -UASMNAME -UASMFNAME -UNAME -UCNAME -UCHAR_NAME -UCHAR_CNAME -DASMNAME=_snrm2 -DASMFNAME=_snrm2_ -DNAME=snrm2_ -DCNAME=snrm2 -DCHAR_NAME=\"snrm2_\" -DCHAR_CNAME=\"snrm2\" -DNO_AFFINITY -I.. -I. -UDOUBLE  -UCOMPLEX -c nrm2.c -o snrm2.o
clang: error: unsupported option '-fopenmp'
clang: error: unsupported option '-fopenmp'
clang: error: unsupported option '-fopenmp'
clang: error: unsupported option '-fopenmp'
make[1]: *** [scopy.o] Error 1
make[1]: *** Waiting for unfinished jobs....
clang: error: unsupported option '-fopenmp'
clang: error: unsupported option '-fopenmp'
make[1]: *** [sswap.o] Error 1
make[1]: *** [sscal.o] Error 1
clang: error: unsupported option '-fopenmp'
clang: error: make[1]: *** [saxpy.o] Error 1
unsupported option '-fopenmp'
make[1]: *** [sdot.o] Error 1
make[1]: *** [sdsdot.o] Error 1
make[1]: *** [ssum.o] Error 1
make[1]: *** [sasum.o] Error 1
clang: error: unsupported option '-fopenmp'
make[1]: *** [snrm2.o] Error 1
clang: error: unsupported option '-fopenmp'
make[1]: *** [dsdot.o] Error 1
make: *** [libs] Error 1

[dimpase]

another option might be to have openblas linked to libomp, not libgomp.
How does one setup cmake bindings for libomp using brew? I can't figure it out.

[dimpase]

another option might be to have openblas linked to libomp, not libgomp. How does one setup cmake bindings for libomp using brew? I can't figure it out.

This actually appears doable, except that it's linking to both of them, libomp (for C) and libgomp (for Fortran). Modulo a couple cmake quirks, it appears to work, see
OpenMathLib/OpenBLAS#5156 (comment)

Would you be interested in a PR switching this build to such a setup?

[carlocab]

I don't think we want to link to both libomp and libgomp, I'm afraid. Mixing OpenMP runtimes is a huge no-no.

[dimpase]

Python people deal with multiple libomp's in one application on a regular basis. Perhaps @rgommers can comment on this.

I also don't know whether there is any C OpenMP in openblas.

[rgommers]

Mixing OpenMP runtimes is a huge no-no.

This is correct, it's a recipe for major trouble.

Python people deal with multiple libomp's in one application on a regular basis.

The only reason for this is that wheels uploaded to PyPI cannot depend on a common shared library, and aren't built as a coherent stack. To make it work at all, extensive symbol mangling machinery is used. The end result is lower performance without any real upside, because you have two OpenMP runtimes that don't know about each other and hence won't manage resources as well as a single runtime could in multithreading or multiprocessing use cases.

[dimpase]

Does this mean that merely correcting flags acceptable by clang (by adding -Xclang in front of -fopenmp) will allow the use of GNU openmp (and openblas with GNU openmp) in clang-compiled C/C++ code?

[ywwry66]

Does this mean that merely correcting flags acceptable by clang (by adding -Xclang in front of -fopenmp) will allow the use of GNU openmp (and openblas with GNU openmp) in clang-compiled C/C++ code?

For openmp to work using Apple clang, you need to build openblas as in https://gist.github.com/ywwry66/19d0f280632fb5b8c296e8a6ca06b6d7. However, you might still need to patch the .pc files to include -Xclang or -Xpreprocessor.

In my opinion, there is no ideal solution for now. Two ways that can possibly improve the current situation is: 1. Homebrew using Apple clang to build openblas when Apple adds flang frontend. 2. Homebrew using Apple's Accelerate framework instead of openblas to build python, R, etc (see #181251), unless your workflow requires direct usage of openblas.