Duplicate symbol updating EMS

[exonent]

Hi all!

I was updating my EMS and using my sentence to compile my code and now I have a new errors that I haven't with the EMS older version:

wasm-ld: error: duplicate symbol: ini1_
>>> defined in /var/folders/pj/s807zy0s5f79bbr_yvpkmgw00000gq/T/emscripten_temp_YHWD8i/File1.o
>>> defined in /var/folders/pj/s807zy0s5f79bbr_yvpkmgw00000gq/T/emscripten_temp_YHWD8i/File2.o

(Have this error in multiples files, not only in File1 and File2)
This is a new error...
I tried to make
emar rc File1.a File1.o
then
emcc File1.a File2.o -c -o Test.html
But doesn't work!

Some idea?

[kripken]

I'm not sure why this worked before, but the error details say that the symbol is defined in both of those files - why is it defined in both? That should give an error, unless it is defined as a weak symbol or some other method. If you can share the source files, that would help figure this out.

[exonent]

Hi, Thx for the reply. The code is duplicated because I’m trying to convert my Fortran77 code to EMS, so I have some duplicated variables for that. I used GCC to deploy this code in Objective-C and XCode and works perfectly, but when I replace in the makefile the gcc syntaxis doesn’t work. I can’t upload the code to public people because is a Company Code and I can’t do that... ;( Have a good day mate!

[kripken]

Another idea is try to port it first to clang. If it works with clang, there's a much better chance it will work with emscripten.

Aside from that, I'd investigate why your duplicated code works in gcc and others. It likely works either because one of the two files is actually included but not both (and maybe your emcc makefile has that wrong), or it uses some weak symbol linking or such (which should be obvious in the source code).

Finally, if nothing else works, I'd suggest creating a new testcase from scratch, a super-simplified version of the problem part of that codebase, which you can make public. It should show the same commands working in gcc but failing in emcc.

[exonent]

Ok so that's what I'm using in Fortran 77 to XCode (Objective-C) and WORKS FINE in iOS:

F77 to Objective-C:

- rm makefile
- cp makefile.u makefile
- bash for2c.sh
- bash build_libc_mac.sh

for2c.sh content:

F2C_PATH=../ThirdParty/f2c
for fn in *.FOR; do
	echo "Formatting: ${fn}"
	cp ${fn} ${fn%.FOR}.F
	${F2C_PATH}/f2c -Nn802 ${fn%.FOR}.F
	rm ${fn%.FOR}.F
done

build_libc_mac.sh content:

ARCH_MAC=x86_64
ARCH_ARMV8=ARMv8

SET_ARCH() { export ARCH=$1 }
SET_SDKROOT() { export SDKROOT=`xcrun --show-sdk-path --sdk $1` }

# Prepare makefile
sed 's/CC = cc/CC = cc -arch ${ARCH}/' makefile.u > makefile.u2
sed 's/CFLAGS = -O/CFLAGS = -isysroot ${SDKROOT} -arch ${ARCH}/' makefile.u2 > makefile
rm makefile.u2

# Compile SIM
SET_ARCH $ARCH_ARM64
SET_SDKROOT iphoneos
make clean
make
mv libc.a $ARCH-libc.a

make clean
lipo -create -arch $ARCH_ARM64 $ARCH_ARM64-libc.a -output libc.a

Makefile content

#I have a libf2c.a
PATH_F2C=../ThirdParty/f2c
SRCF77 = File1.c File2.c File3.c File4.c File5.c File6.c #Watch NO File7.c here!!
OBJS = File7.o File1.o File2.o File3.o File4.o File5.o File6.o
CC = gcc

CFLAGS = -isysroot ${SDKROOT} -arch ${ARCH}
AR = ar

libcaps.a : $(OBJS)
	$(AR) rcs libcaps.a $(OBJS)
	
%.o : %.c
	$(CC) -c $*.c $(CFLAGS) -I$(PATH_F2C)

File7.o : File7.h

# test
test : test.o libc.a
	$(CC) -Wall -o test test.o -L. -lcaps -L$(PATH_F2C) -lf2c -lsqlite3

test.o : test.c
	$(CC) -Wall -c test.c $(CFLAGS) -I$(PATH_F2C)

.PHONY : clean
clean :
	rm -f $(wildcard *.o) libc.a

---------------------EMS Part
And now what I am trying to convert: Fortran 77 into iONIC (EMScripten).

F77 to EMScripten:

- rm makefile
- cp makefile.u makefile
- bash for2c.sh #<-- Same as above
- bash build_libEMS_mac.sh

for2c.sh content:

Same as above. No changes

build_libEMS_mac.sh content:

ARCH_MAC=x86_64
ARCH_ARMV8=ARMv8

SET_ARCH() { export ARCH=$1 }
SET_SDKROOT() { export SDKROOT=`xcrun --show-sdk-path --sdk $1` }

# Prepare makefile
sed 's/CC = cc/CC = cc -arch ${ARCH}/' makefile.u > makefile.u2
sed 's/CFLAGS = -O/CFLAGS = -isysroot ${SDKROOT} -arch ${ARCH}/' makefile.u2 > makefile
rm makefile.u2

# Compile SIM
SET_ARCH $ARCH_ARM64
SET_SDKROOT iphoneos
make clean
make
# That's create libEMS.a                     <--
# NO "lipo" command required. No?  <--

Makefile content (link libf2c.zip official page)

# I have created a new libf2c.a because the XCode version crash. 
# To create the new libf2c.a I take all the ".c" files included in the link above.
# Sentences used: emcc -c FileX.c -o FileX.o |--Then--> emar rcs libf2c.a  AllFilesX.o
PATH_F2C=../ThirdParty/f2c
SRCF77 = File1.c File2.c File3.c File4.c File5.c File6.c #Watch NO File7.c here!!
OBJS = File7.o File1.o File2.o File3.o File4.o File5.o File6.o
CC = emcc
EMS = -If2c \
-Lf2c \
-O0 \
-lf2c \
-s MODULARIZE=1 \
-s WASM=1 \
-s EMBIND_STD_STRING_IS_UTF8=1 \
-s EXPORT_ALL=1 \
-s EXPORTED_FUNCTIONS='["_main", _new_inputTf", "_free_inputTf", "_new_outputTf", "_free_outputTf"]' \
-s EXTRA_EXPORTED_RUNTIME_METHODS='["ccall", "cwrap"]' \
-s TOTAL_MEMORY=562949953421312 \
-s ALLOW_MEMORY_GROWTH=1 \
-s ERROR_ON_UNDEFINED_SYMBOLS=0 \

OUTPUT = -o results/libEMS.html
AR = emar

libEMS.a : $(OBJS) 
	$(AR) rcs libEMS.a $(OBJS)
	
%.o : %.c
	$(CC) -c $*.c -I$(PATH_F2C) $(EMS) $(OUTPUT)

File7.o : File7.h

# test
test : test.o libEMS.a
	$(CC) -Wall -o test test.o -L. -lems -L$(PATH_F2C) -lf2c -lsqlite3 

test.o : test.c
	$(CC) -Wall -c test.c $(CFLAGS) -I$(PATH_F2C)

.PHONY : clean
clean :
	rm -f $(wildcard *.o) $(wildcard *.wasm) $(wildcard *.js) libEMS.a

I put some comments to know what I'm doing.
Thank you very much for your help! Have a good Weekend!

[sbc100]

Could you post the full emcc failing command line, along with full output of the command when run with EMCC_DEBUG=1?

My first suggestion would be that you remove EXPORT_ALL and ERROR_ON_UNDEFINED_SYMBOLS flag. Most projects should not be using those.

[exonent]

I removed both sentences:

EMCC_DEBUG=1 emcc \
 File0.c File00.c File1.c File3.c File2.c File4.c File5.c File6.c File7.c File8.c File9.c File10.c File11.c \
 libf2c.a \
 -If2c \
 -Lf2c \
 -O0 \
 -lf2c \
 -s MODULARIZE=1 \
 -s WASM=1 \
 -s EMBIND_STD_STRING_IS_UTF8=1 \
 -s EXPORTED_FUNCTIONS='["_main", _new_inputTf", "_free_inputTf", "_new_outputTf", "_free_outputTf","_new_inputCb", "_free_inputCb", "_new_outputCb", "_free_outputCb", "_new_inputCe", "_free_inputCe", "_new_outputCe", "_free_outputCe", "_new_inputDt", "_free_inputDt", "_new_outputDt", "_free_outputDt", "_new_inputLg", "_free_inputLg", "_new_outputLg", "_free_outputLg"]' \
 -s EXTRA_EXPORTED_RUNTIME_METHODS='["ccall", "cwrap"]' \
 -s TOTAL_MEMORY=562949953421312 \
 -s ALLOW_MEMORY_GROWTH=1 \
 -o results/fibonacci.js

But still saying:

wasm-ld: error: duplicate symbol: afm_
>>> defined in /var/folders/pj/s807zy0s5f79bbr_yvpkmgw00000gq/T/emscripten_temp_8aOQVG/File1_2.o
>>> defined in /var/folders/pj/s807zy0s5f79bbr_yvpkmgw00000gq/T/emscripten_temp_8aOQVG/LFile2_3.o

The EMCC_DEBUG=1 File, attached because is too long.

[sbc100]

I looks like you just have to fix your multiple defined symbols.

For example afm_ seems to be defined by both File1.c and File3.c. One thing to try is compiling each C file on its own using -c which will generate object files. Then you can easily see that they have duplicate symbols by running the llvm-nm tool on them. On way to debug duplicate symbols errors when its no object where the symbols are coming from is to pre-process the sources rather than compile whem, using -E.

I don't think this is an emscripten issue though, more of general C++ development issue. I imagine you would see the same issues with you desktop clang / gcc compiler.

[exonent]

hi @sbc100 !
Thx for your reply. I can compile it fine using GCC as you can se above in the "Makefile content" - "F77 to Objective-C". No problem and can use it very well into my iOS project. But I can't do the same in EMS.

Is my "Makefile content (link libf2c.zip official page)" correct? Here I'm using -c then convert to .a file. But, Is it possible to make a .wasm and .js using an .a file? I mean, Can I do the following steps and get .wasm and .js?:

PATH_F2C=../ThirdParty/f2c
SRCF77 = File1.c File2.c File3.c File4.c File5.c File6.c #Watch NO File7.c here!!
OBJS = File7.o File1.o File2.o File3.o File4.o File5.o File6.o
CC = emcc
EMS = -If2c \
-Lf2c \
-O0 \
-lf2c \
-s MODULARIZE=1 \
-s WASM=1 \
-s EMBIND_STD_STRING_IS_UTF8=1 \
-s EXPORT_ALL=1 \
-s EXPORTED_FUNCTIONS='["_main", _new_inputTf", "_free_inputTf", "_new_outputTf", "_free_outputTf"]' \
-s EXTRA_EXPORTED_RUNTIME_METHODS='["ccall", "cwrap"]' \
-s TOTAL_MEMORY=562949953421312 \
-s ALLOW_MEMORY_GROWTH=1 \
-s ERROR_ON_UNDEFINED_SYMBOLS=0 \

OUTPUT = -o results/libEMS.html
AR = emar

libEMS.a : $(OBJS) 
	$(AR) rcs libEMS.a $(OBJS)
	
%.o : %.c
	$(CC) -c $*.c -I$(PATH_F2C) $(EMS) $(OUTPUT)

File7.o : File7.h

# test
test : test.o libEMS.a
	$(CC) -Wall -o test test.o -L. -lems -L$(PATH_F2C) -lf2c -lsqlite3 

test.o : test.c
	$(CC) -Wall -c test.c $(CFLAGS) -I$(PATH_F2C)

.PHONY : clean
clean :
	rm -f $(wildcard *.o) $(wildcard *.wasm) $(wildcard *.js) libEMS.a

@kripken said something about do different thing:

Another idea is try to port it first to clang. If it works with clang, there's a much better chance it will work with emscripten.

How can I try that? What I have to do? The steps, I mean.

[sbc100]

In general emscripten, clang and gcc should all behave exactly the same with respect to multiply defined symbols.

Is there any thing special about the symbols that are reported as multiply defined? What type of symbols are they? How do they get defined in the source?

[exonent]

They are defined using F2C converter. I don't modify the the files.
I just use the f2c.sh to convert Fortran 77 to C and then I deploy it in iOS using GCC with the commands listed above.

[kripken]

Ok, I found some time to look at the files you sent. I couldn't find instructions, but after some poking around I found an emcc command at the bottom of a .sh file. I had to remove -lf2c from the command, since it errors on not finding it. Without that, I get duplicate symbol errors like you saw.

Running gcc or clang (I had to remove EMSCRIPTEN_KEEPALIVE and a few other things) I get multiple definition errors. I would suggest trying to fix those, that is, get the native build working first, and only then try to cross-compile.

To fix them, I tried adding static to the dagen3_ struct, which is defined in many files (identically, btw). Doing that removes the error on that struct. So I think you may fix them that way, but I only tried one.

That is, these duplicates are from something like

struct {
  int x;
} foo;

that appear in many files, all the same. Changing them all to

static struct {
  int x;
} foo;

fixes it.

[exonent]

@kripken @sbc100

I did something different. I Renamed the duplicate symbols with other names and used the following sentences:

emcc --clear-cache --clear-ports

For each .C file I use the follow command:

emcc -If2c -Wall -Wformat -O0 -s WASM=1 -s ALLOW_MEMORY_GROWTH=1 -s DISABLE_EXCEPTION_CATCHING=1 -s NO_EXIT_RUNTIME=0 -s ASSERTIONS=1 -s NO_FILESYSTEM=1  -c -o SLEC.o SLEC.c -s EXTRA_EXPORTED_RUNTIME_METHODS='["ccall", "cwrap"]'

Then, I take all the .o files and run:

emcc -o results/libEMS.html \
libf2c.a File1.o File2.o File3.o File4.o File5.o File6.o File7.o File8.o File9.o File10.o File11.o File12.o File13.o\
-s WASM=1 \
-s ALLOW_MEMORY_GROWTH=1 \
-s DISABLE_EXCEPTION_CATCHING=1 \
-s NO_EXIT_RUNTIME=0 \
-s ASSERTIONS=1 \
-s NO_FILESYSTEM=1 \
-s EXPORTED_FUNCTIONS='["_new_inputTf", "_free_inputTf", "_new_outputTf", "_free_outputTf","_new_inputCb", "_free_inputCb", "_new_outputCb", "_free_outputCb", "_new_inputCe", "_free_inputCe", "_new_outputCe", "_free_outputCe", "_new_inputDt", "_free_inputDt", "_new_outputDt", "_free_outputDt", "_new_inputLg", "_free_inputLg", "_new_outputLg", "_free_outputLg", "_main"]' \
-s EXTRA_EXPORTED_RUNTIME_METHODS='["ccall", "cwrap"]' \
-s TOTAL_MEMORY=562949953421312

And I have many warnings, but I don't know if they are important:

....
wasm-ld: warning: function signature mismatch: afm_
>>> defined as (i32, i32, i32, i32, i32, i32, i32) -> i32 in File1.o
>>> defined as (i32, i32, i32, i32, i32, i32, i32, i32, i32) -> i32 in File3.o
...

When I run the last command using EMSCRIPTEN_KEEPALIVE and including emscripten.h in each .c file, I have an error at the end of the compilation:

cache:INFO: generating system asset: generated_struct_info.json... (this will be cached in "/Users/MyUser/.emscripten_cache/wasm-obj/generated_struct_info.json" for subsequent builds)
cache:INFO:  - ok
[parse exception: attempted pop from empty stack / beyond block start boundary at 62501 (at 0:62501)]
Fatal: error in parsing input
shared:ERROR: '/Users/MyUser/Desktop/EMScripten/emsdk/upstream/bin/wasm-emscripten-finalize --detect-features --global-base=1024 --check-stack-overflow /var/folders/pj/s807zy0s5f79bbr_yvpkmgw00000gq/T/emscripten_temp_ebnF8B/fibonacci.wasm -o /var/folders/pj/s807zy0s5f79bbr_yvpkmgw00000gq/T/emscripten_temp_ebnF8B/fibonacci.wasm.o.wasm' failed (1)

I don't know what to do here. So I tried to remove all the EMSCRIPTEN_KEEPALIVE and emscripten.h includes. Just to know if I can compile it. NO Errors in compilation. Feels good, but when I tried to use the .js and .wasm an error appear on the web console:

failed to asynchronously prepare wasm: LinkError: WebAssembly.instantiate(): Import #0 module="env" function="emscripten_resize_heap" error: function import requires a callable

libEMS.js:27433 LinkError: WebAssembly.instantiate(): Import #0 module="env" function="emscripten_resize_heap" error: function import requires a callable

libEMS.js:27434 LinkError: WebAssembly.instantiate(): Import #0 module="env" function="emscripten_resize_heap" error: function import requires a callable

core.js:1449 ERROR Error: Uncaught (in promise): abort(LinkError: WebAssembly.instantiate(): Import #0 module="env" function="emscripten_resize_heap" error: function import requires a callable) at Error
    at jsStackTrace (libEMS.js:1192)
    at stackTrace (libEMS.js:1209)
    at abort (libEMS.js:27440)
    at libEMS.js:1699
    at t.invoke (polyfills.js:3)
    at Object.onInvoke (core.js:4760)
    at t.invoke (polyfills.js:3)
    at r.run (polyfills.js:3)
    at polyfills.js:3
    at t.invokeTask (polyfills.js:3)
    at c (polyfills.js:3)
    at polyfills.js:3
    at t.invokeTask (polyfills.js:3)
    at Object.onInvokeTask (core.js:4751)
    at t.invokeTask (polyfills.js:3)
    at r.runTask (polyfills.js:3)
    at o (polyfills.js:3)

Some idea? Thank you very much for your help guys!

[exonent]

Ok the error:

When I run the last command using EMSCRIPTEN_KEEPALIVE and including emscripten.h in each .c file, I have an error at the end of the compilation:

cache:INFO: generating system asset: generated_struct_info.json... (this will be cached in "/Users/MyUser/.emscripten_cache/wasm-obj/generated_struct_info.json" for subsequent builds)
cache:INFO:  - ok
[parse exception: attempted pop from empty stack / beyond block start boundary at 62501 (at 0:62501)]
Fatal: error in parsing input
shared:ERROR: '/Users/MyUser/Desktop/EMScripten/emsdk/upstream/bin/wasm-emscripten-finalize --detect-features --global-base=1024 --check-stack-overflow /var/folders/pj/s807zy0s5f79bbr_yvpkmgw00000gq/T/emscripten_temp_ebnF8B/fibonacci.wasm -o /var/folders/pj/s807zy0s5f79bbr_yvpkmgw00000gq/T/emscripten_temp_ebnF8B/fibonacci.wasm.o.wasm' failed (1)

It's due to the libf2c.a. So... I downloaded from the official web f2c then unzip the libf2c.zip and run the followings commands:
cd libf2c | emcc -c *.c | emar rcs libf2c.a *.o

Should I add #include "emscripten.h" in each .c file and EMSCRIPTEN_KEEPALIVE in all functions?

What I have to do to compile libf2c.zip correctly?

[sbc100]

Using EMSCRIPTEN_KEEPALIVE is the equivalent of using EXPORTED_FUNCTIONS, so you don't need to use both. Sounds like the command line is working for you so you should need EMSCRIPTEN_KEEPALIVE at all.

[exonent]

I only use EMSCRIPTEN_KEEPALIVE in .c converted files and EXPORTED_FUNCTIONS on my own .c.

The problem now, is when I use libf2c.a.
I'm using the makefile included in libf2c.zip:

cp makefile.u makefile | make

I did a short test using only one .c file (fortran77 converted)

emcc -c File1.c -o File1.o
emcc -o results/libEMS.js libf2c.a File1.o  -s WASM=1

Get the next error when include the libf2c.a:

wasm-ld: error: i_indx.o: machine type must be wasm32
wasm-ld: error: s_copy.o: machine type must be wasm32
wasm-ld: error: s_cmp.o: machine type must be wasm32
wasm-ld: error: i_nint.o: machine type must be wasm32

Some idea?