MPI Compilation

[mschoeppner]

We had problems with the compilation of Flexpart-WRF 3.3 with GNU/OMP. Our system runs with gcc/4.4.7 and openmpi/1.8.1.
In case other users have similar problems, here are the changes in makefile.mom that solved it in our case:
Line 69: FCOMP = mpif90
Line 74: Add “-lgomp” to LDFLAGS

[adingwell]

(Updated the title since the bug here seem to related to MPI, not OMP)

Two comments on this:
1) yes, there is a bug here, the MPI option needs to be updated or compilation will fail.
2) I believe you are confusing MPI with OpenMP and your problem is probably not related to MPI.

I believe so because you state that you are compiling with GNU/OMP and set the FCOMP option, while at the same time referring to OpenMPI. I might as well clarify this a bit here, in case new users stumble upon this in the future.

​OpenMPI is an implementation of MPI (there are many others), which basically is used for parallel processing with distributed memory. On the other hand, ​OpenMP (also called OMP) is used for parallel processing with shared memory. Compiling and running software with OpenMP should not require any MPI-packages.

This becomes even more confusing with the way this is implemented in FLEXPART-WRF. FLEXPART-WRF has two options for parallelization, omp and mpi. The omp option will compile with OpenMP while the mpi option will compile with both OpenMP and MPI (hybrid parallelization). For those used to working with the WRF model: omp in FLEXPART-WRF is equivalent to the "smpar" options in WRF, mpi in FLEXPART-WRF is equivalent to the "dm+sm" options in WRF.

When compiling software using OMP, you usually only need to add a flag to the compiler command (e.g. -fopenmp in the case of gfortran). The GNU compilers also requires that you specify the library for reasons explained ​here. MPI is far more complicated and requires several additional compiler flags, which is why it is ​recommended to use a wrapper for the compiler (e.g. mpif90 which is a wrapper for gfortran).

With that said, the compiler flags for GCC should be:

  FCSER      = gfortran
  FCOMP      = gfortran -fopenmp -lgomp
  FCMPI      = mpif90 -fopenmp -lgomp

I am not sure, but it seems like the "-fopenmp" option is not needed to get openmp support when using mpif90. This could be why your problem is solved when setting:

  FCOMP = mpif90 -lgomp

since it (among other things) gives you this:

  FCOMP = gfortran -fopenmp -lgomp

However, using mpif90 will also link in several of MPI-libraries, which do nothing in the OMP version but add to the dependencies of the executable. This is probably not a big issue unless you try to use the executable on other systems or update your MPI-version.

I will apply these changes as well as corresponding changes for pgi and intel (but I have no possibility to test compilation on intel/pgi)

[mschoeppner]

Sorry for the confusion. We played around with the makefile until it worked and at some point also changed the FCOMP to mpif90. Now I realize that it was enough to move -fopenmp and -lgomp from FCOMP to LDFLAGS to make it work on our system. Otherwise, with the original version, the compilation crashes at calcpv_nests.f90 with an internal compiler error. Thanks for the explanation.

[pesei]

Ticket retargeted after milestone closed

[pesei]

It seems that it was just an issue of different names of flags with different compilers and the problem is solved. Thus I close the ticket.