Opened 3 years ago

Last modified 22 months ago

#159 accepted Defect

MPI Compilation

Reported by: mschoeppner Owned by: adingwell
Priority: major Milestone: FLEXPART_WRF_3.4_FPbase_9
Component: FP coding/compilation Version: FLEXPART-WRF
Keywords: Cc:

Description (last modified by admin)

We had problems with the compilation of Flexpart-WRF 3.3 with GNU/OMP. Our system runs with gcc/4.4.7 and openmpi/1.8.1.
In case other users have similar problems, here are the changes in makefile.mom that solved it in our case:
Line 69: FCOMP = mpif90
Line 74: Add “-lgomp” to LDFLAGS

Change History (5)

comment:1 Changed 3 years ago by admin

  • Description modified (diff)
  • Summary changed from Compilation to OMP Compilation

comment:2 Changed 3 years ago by adingwell

  • Milestone set to FLEXPART_WRF_3.2_FPbase_9
  • Owner set to adingwell
  • Status changed from new to accepted
  • Summary changed from OMP Compilation to MPI Compilation

(Updated the title since the bug here seem to related to MPI, not OMP)

Two comments on this:
1) yes, there is a bug here, the MPI option needs to be updated or compilation will fail.
2) I believe you are confusing MPI with OpenMP and your problem is probably not related to MPI.

I believe so because you state that you are compiling with GNU/OMP and set the FCOMP option, while at the same time referring to OpenMPI. I might as well clarify this a bit here, in case new users stumble upon this in the future.

OpenMPI is an implementation of MPI (there are many others), which basically is used for parallel processing with distributed memory. On the other hand, OpenMP (also called OMP) is used for parallel processing with shared memory. Compiling and running software with OpenMP should not require any MPI-packages.

This becomes even more confusing with the way this is implemented in FLEXPART-WRF. FLEXPART-WRF has two options for parallelization, omp and mpi. The omp option will compile with OpenMP while the mpi option will compile with both OpenMP and MPI (hybrid parallelization). For those used to working with the WRF model: omp in FLEXPART-WRF is equivalent to the "smpar" options in WRF, mpi in FLEXPART-WRF is equivalent to the "dm+sm" options in WRF.

When compiling software using OMP, you usually only need to add a flag to the compiler command (e.g. -fopenmp in the case of gfortran). The GNU compilers also requires that you specify the library for reasons explained here. MPI is far more complicated and requires several additional compiler flags, which is why it is recommended to use a wrapper for the compiler (e.g. mpif90 which is a wrapper for gfortran).

With that said, the compiler flags for GCC should be:

  FCSER      = gfortran
  FCOMP      = gfortran -fopenmp -lgomp
  FCMPI      = mpif90 -fopenmp -lgomp

I am not sure, but it seems like the "-fopenmp" option is not needed to get openmp support when using mpif90. This could be why your problem is solved when setting:

  FCOMP = mpif90 -lgomp

since it (among other things) gives you this:

  FCOMP = gfortran -fopenmp -lgomp

However, using mpif90 will also link in several of MPI-libraries, which do nothing in the OMP version but add to the dependencies of the executable. This is probably not a big issue unless you try to use the executable on other systems or update your MPI-version.

I will apply these changes as well as corresponding changes for pgi and intel (but I have no possibility to test compilation on intel/pgi)

comment:3 Changed 3 years ago by adingwell

  • Milestone changed from FLEXPART_WRF_3.2_FPbase_9 to FLEXPART_WRF_3.3_FPbase_9

comment:4 Changed 3 years ago by mschoeppner

Sorry for the confusion. We played around with the makefile until it worked and at some point also changed the FCOMP to mpif90. Now I realize that it was enough to move -fopenmp and -lgomp from FCOMP to LDFLAGS to make it work on our system. Otherwise, with the original version, the compilation crashes at calcpv_nests.f90 with an internal compiler error. Thanks for the explanation.

Last edited 22 months ago by pesei (previous) (diff)

comment:5 Changed 2 years ago by pesei

  • Milestone changed from FLEXPART_WRF_3.3_FPbase_9 to FLEXPART_WRF_3.4_FPbase_9

Ticket retargeted after milestone closed

Note: See TracTickets for help on using tickets.
hosted by ZAMG