Opened 3 years ago

Last modified 13 months ago

#151 assigned Defect

Improper use of MPI_IN_PLACE

Reported by: adingwell Owned by: espen
Priority: minor Milestone: FLEXPART 10
Component: FP coding/compilation Version: FLEXPART 10.0
Keywords: mpi, buffer, parallel Cc:

Description (last modified by pesei)

In mpi_mod.f90 and FLEXPART_MPI.f90 there are calls to MPI_REDUCE which use MPI_IN_PLACE.
The non-root calls to MPI_RECEIVE currently use the same buffer for send and receive; this is will produce errors when running with Intel-MPI.
The correct way of doing this is to set RECBUF (second argument) to 0, in non-root calls where MPI_IN_PLACE was assigned by the root.

E.g. in FLEXPART_MPI.f90, this block:

  if (lroot) then
    call MPI_Reduce(MPI_IN_PLACE, tot_blc_count, 1, mp_pp, MPI_SUM, id_root, &
         & mp_comm_used, mp_ierr)
    call MPI_Reduce(MPI_IN_PLACE, tot_inc_count, 1, mp_pp, MPI_SUM, id_root, &
         & mp_comm_used, mp_ierr)
  else
    if (mp_partgroup_pid.ge.0) then ! Skip for readwind process 
      call MPI_Reduce(tot_blc_count, tot_blc_count, 1, mp_pp, MPI_SUM, id_root, &
           & mp_comm_used, mp_ierr)
      call MPI_Reduce(tot_inc_count, tot_inc_count, 1, mp_pp, MPI_SUM, id_root, &
           & mp_comm_used, mp_ierr)
    end if
  end if

should be replaced with this:

if (lroot) then
    call MPI_Reduce(MPI_IN_PLACE, tot_blc_count, 1, mp_pp, MPI_SUM, id_root, &
         & mp_comm_used, mp_ierr)
    call MPI_Reduce(MPI_IN_PLACE, tot_inc_count, 1, mp_pp, MPI_SUM, id_root, &
         & mp_comm_used, mp_ierr)
  else
    if (mp_partgroup_pid.ge.0) then ! Skip for readwind process 
      call MPI_Reduce(tot_blc_count, 0, 1, mp_pp, MPI_SUM, id_root, &
           & mp_comm_used, mp_ierr)
      call MPI_Reduce(tot_blc_count, 0, 1, mp_pp, MPI_SUM, id_root, &
           & mp_comm_used, mp_ierr)
    end if
  end if

(Corresponding changes should also be made in mpi_mod.f90)

Change History (2)

comment:1 Changed 3 years ago by espen

gfortran allows this, as the receive buffers for non-root processes are not actually used (only the root process receives data). The old formulation is technically a breach of the Fortran standard I assume. Good to have this tested with other compilers!

Fix will be implemented as suggested for version 10.0

comment:2 Changed 13 months ago by pesei

  • Description modified (diff)
  • Owner set to espen
  • Status changed from new to assigned

Hi Espen, could you please accept this ticket and state what has happend since then?

Note: See TracTickets for help on using tickets.
hosted by ZAMG