Compiling fails on Mac OS 10.15 with "unable to execute command: Illegal instruction" error

[craymond]

I am trying to compile Flexpart for the first time and would greatly appreciate some help. I've gotten nearly to the end of the process (having installed the necessary libraries, etc.) but am now quite stuck. My set-up is as follows:

Mac OS 10.15.7
gcc10.2 (this version is also used to build mpicc)
ulimit -n 1000
ulimit -s 16383
LIBRARY_PATH, LD_LIBRARY_PATH, and various FLAGS are set to (I believe) the appropriate directories.

I then try to compile with

make -f makefile gcc=10.2 FC=gfortran ncf=yes -I/dir/of/eccodes/build/include/grib_api.mod

which results in the error

clang: error: unable to execute command: Illegal instruction: 4
clang: error: clang integrated assembler command failed due to signal (use -v to see invocation)
Apple clang version 12.0.0 (clang-1200.0.32.29)
Target: x86_64-apple-darwin19.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
clang: note: diagnostic msg: PLEASE submit a bug report to http://developer.apple.com/bugreporter/ and include the crash backtrace, preprocessed source, and associated run script.
clang: note: diagnostic msg: Error generating preprocessed source(s) - no preprocessable inputs.
make: *** [com_mod.o] Error 1

Experimenting with various flags and compilers (i.e. gcc, gfortran, mpicc, mpif90) has not helped so far, and the answers found from Google are either too complex for me to understand, or too specific to seem applicable. What I can tell is it has something to do with wrong compiler options and/or Mac compatibility with 32-bit operations, but what these are exactly I can't determine. I'm hoping someone else has had this issue (or a similar one) as well and can advise!

Thank you,
Colin Raymond

[pesei]

I am not familiar with MacOS, but information such as the compiler to use or the include paths should generally go into your makefile, not as arguments to the make command. You need to adjust the makefile to your needs. If you still have problems, please add your makefile as an attachment to this ticket.

[craymond]

makefile

[craymond]

I have modified the makefile to point to what I think are the proper libraries and other settings (following the website instructions). It is attached here. I am having a hard time verifying the details of the options, but clearly I have done some of them incorrectly or Flexpart would be happier.

[pesei]

and are you trying to compile with

make -f makefile ncf=yes

now? Do you still get exactly the same error message?

Have you tried to manually compile one source file (best one that does not depend on external libs and modules)?

[craymond]

Yes, this gives the same "illegal instruction" error. Trying to compile a single source file, e.g.

mpif90 advance.f90
gives the error

Fatal Error: Cannot open module file 'point_mod.mod' for reading at (1): No such file or directory

Compiling other individual source files results in very similar errors.

[pesei]

As you see, this subroutine mpif90 advance.f90 depends on a module file. The following subroutines do NOT depend on modules:

xmass_mod.f90 
windalign.f90 
sort2.f90 
skplin.f90 
random_mod.f90 
par_mod.f90
point_mod.f90 
photo_O1D.f90 
oh_mod.f90 
hanna_mod.f90 
flux_mod.f90 
ew.f90 0

Please try one or more of those.

[craymond]

These .f90 subroutines are for me located at /Users/me/flexpart/src. I have LIBPATH1 set to this path in the makefile, and also set it as the LIBRARY_PATH and LD_LIBRARY_PATH. With these settings,

mpif90 -c hanna_mod.f90; echo $?

successfully results in

0

[pesei]

That's good, so your compiler works. You should first try to compile the serial version, however. When this works, you can try mpi. Please do the following manually:

gfortran -c par_mod.f90
gfortran -c -I. com_mod.f90

then try to compile all the other subroutines with -c -I. as well as the include and library information for grib_api and gcc built-in modules. Don't care if some subroutines don't build immediately, as we are ignoring dependencies. After some iterations, however, everything should compile. If not, please post the error. If it does, you can then manually link. If that works as well, you know that you have a problem with your makefile.

[craymond]

Ok, after some iterations I have gotten

gfortran -c -I/folder/1 -I/folder/2 *.f90

to compile all the subroutines under flexpart/src. I am still a bit lost however about how to link these (via the makefile or manually), or what changes I could possibly put in place in the makefile that would address the

clang: error: unable to execute command: Illegal instruction: 4
clang: error: clang integrated assembler command failed due to signal (use -v to see invocation)

error. Thanks very much for the continuing assistance.

[pesei]

It is not clear to me why you get error messages from "clang" - I think this is a C compiler? Can you explain why you get messages from clang?

To link: gfortran -o FLEXPART.out *.o -Llibpath... -llib...

You will have to remove any double files, i.e. all mpi objects (assuming that you are building a serial version).

If that works, you may want to redo this with all the flags that are in the makefile ...

[craymond]

From what I can tell the clang error messages have something to do with Mac architecture and/or compiler options, but whatever is going on has stymied all my attempts to diagnose or fix it.

In the manual approach,

gfortran -o FLEXPART.out *.o -L/flexpart/src

results in

Undefined symbols for architecture x86_64:
  "_main", referenced from:
     implicit entry/start for main executable
ld: symbol(s) not found for architecture x86_64
collect2: error: ld returned 1 exit status

Unfortunately, I am not too optimistic of making it all the way through even with the manual approach, considering the apparent difficulty of the problem and the level of my competence on these topics. There are probably a few simple fixes out there for all these issues, but it would take me a very long time to figure out what they are.

[pesei]

any news?

[craymond]

With the manual linking I still end up getting the same "illegal instruction" command, and cannot find online any guidance that is specific enough to this problem that it seems applicable (although I tried a wide variety of things). I know that Flexpart is not usually installed on Macs (perhaps there is someone who has done so who could provide assistance?). I have the feeling that I am just one small trick away from success, however I really have no idea right now of what that trick might be.

Thank you again for the help, and I still hope that it leads to a successful outcome.

[mduetsch]

I think the problem is that Mac OS does not support (and need) the -mcmodel compiler flag. Can you try to remove -mcmodel=medium in your makefile and see if it works?

[pesei]

Hello Craymond, does it work now?

[craymond]

Thank you for the follow-up. This remains an active issue for me. I have removed the -mcmodel=medium statements in the makefile, but in rerunning the commands I now get stuck even earlier. Specifically, running 'make' with the same options as my initial posting now leads to

ld: library not found for -lnetcdff
collect2: error: ld returned 1 exit status

This seems like it would be easily resolvable by Googling, however I am unable to find a solution that works for makefiles and on Macs. Currently I have the libnetcdf.a, libnetcdf.la, and libnetcdff.la files located at /Users/me/this/folder, but setting

export LDFLAGS="-L/usr/local/lib -L/Users/me/this/folder

and

export LD_LIBRARY_PATH="/usr/local/lib /Users/me/this/folder"

(as is suggested in many places) does not solve the issue.

I would greatly appreciate any thoughts on what to do about the problem, as simple as it may appear! Thank you

[mduetsch]

It might be because you are missing the libnetcdff.a library (with two f). How did you install netcdf? If you have homebrew, you can try "brew install netcdf". This will include all the necessary libraries.