Preliminary background on grib2flexpart utility

grib2flexpart is a command line utility that processes ECMWF and GFS input data into an intermediate format (.fp)directly readable by FLEXPART. The main idea is that meteorological fields do not need to be read in and the parameters recalculated multiple times.

Several issues needed to be addressed:

• Decide what variables we think we need to save to .fp files - all the related variables in com_mod.f90
• Decide what source files should go into the library - that would be all the ones in charge of reading the meteorological data, checking it and calculating the parameters needed for FLEXPART to run
• Inititate a conceptual drafting of the procedures/algorithm inlcuding the treatment of the reading of the AVAILABLE files and the gridchecking.

The conceptual approach, which makes use as much as possible of original source code from FLEXPART, is as follows:

---

GRIB2FLEXPART UTILITY

Procedure for preprocessing of GRIB data

1. Preprocessing

There are currently two options for execution based on the same developed software:

a) Command line option: this option is meant to process one file or a list of ordered files via command line execution. The main characteristics are as follow:

• Execution line:

     GRIB2FLEXPART forward/backward <lsubgrid> <output directory> <source file> [<source file>...]
  

• the output directory is not generated automatically, the user needs to ensure that the directory already exists
• no configuration files are parsed
• the lsubgrid flag can be set to 1 or 0. The normal FLEXPART user will set it to 1.
• GRIB2FLEXPART processes files in the order provided on command line in forward mode and in reverse order in backward mode. This leads to the same results as in the usual functioning of flexpart
• the output files are named <source file>_<fwd|bwd>.fp
• This option will initialise the FLEXPART internal vertical coordinate system from the first file. If a list of files is provided they will all have the same vertical grid. If, on the contrary, one file is processed at a time, each time the vertical coordinate system will be reinitialised to match the processed file.
• The nested runs should be done one at a time per timestamp and together mother and nest.

b) Use Available option: this option makes use of the structure of FLEXPART and uses the pathnames, AVAILABLE and COMMAND files to process the meteorological data.

• Execution line:

      GRIB2FLEXPART useAvailable <output directory> 
  

• pathnames, AVAILABLE and COMMAND configuration files are read in
• the output directory is not generated automatically, the user needs to ensure that the directory already exists
• grib2flexpart preprocesses only those GRIB files which would be read by flexpart and in the same order as in flexpart since it uses the same input and configuration files.
• the output files are named <source file>_<fwd|bwd>.fp
• This option will initialise the FLEXPART internal vertical coordinate system from the first file as it is usually done by FLEXPART

The two options, and the reusing of FLEXPART source code for each of them, is exemplified in the following diagram:

2. Execution of FLEXPART with preprocessed data

• The COMMAND file is extended to support a new PREPROCESSED_DATA parameter. If set to 1, flexpart uses preprocessed data instead of GRIB files.
• The AVAILABLE file has no changed format. It will have to be copied to the location where the fp files are stored and adapt the pathnames file accordingly.

3. Software package

• GRIB2FLEXPART is a part of flexpart source code package and by default it is built together with flexpart executable (make -f makefile.gfortran)
• It is possible to build grib2flexpart and/or flexpart separately by calling make -f makefile.gfortran flexpart or make -f makefile.gfortran grib2flexpart

---

EXAMPLES OF USAGE

Pre-processing using useAvailable

• Prepare files and directories for test. Test directory should contain:
  • pathnames
  • options directory (including COMMAND file)
  • met_data directory (including AVAILABLE file, copied from original met_data, without any changes - these should look exactly like the AVAILABLE file you would use with GRIB data)

         DATE     TIME         FILENAME     SPECIFICATIONS
         YYYYMMDD HHMISS
         ________ ______      __________      __________
         20140919 000000      EL14091900      ON DISC
         20140919 030000      EL14091903      ON DISC
         20140919 060000      EL14091906      ON DISC
    

  • preprocessed_data directory (empty)
• In options/COMMAND, add new parameter PREPROCESSED_DATA at the end of the parameter list. For preprocessing, value is not important but parameter must be defined so that readcommand routine succeeds.
• While in test case directory, run command "GRIB2FLEXPART useAvailable preprocessed_data"
• FP data are created in preprocessed_data directory.

Differences compared to original Flexpart test case:

• PREPROCESSED_DATA parameter in COMMAND file must exist

Running FLEXPART using preprocessed data

• Prepare files for test. Test directory should contain:
  • pathnames
  • options directory
  • preprocessed_data directory
  • preprocessed_data/AVAILABLE (copied from original met_data, without any changes)
  • output directory (empty)
• In pathnames, change met_data to preprocessed_data and met_data/AVAILABLE to preprocessed_data/AVAILABLE
• In options/COMMAND, set PREPROCESSED_DATA parameter to 1
• Run FLEXPART

Differences compared to original Flexpart test case:

• PREPROCESSED_DATA parameter in COMMAND file must exist and the value must be 1
• paths in pathnames must point to preprocessed data instead of original GRIB files

Pre-processing using command line, for ecmwf_tiny/so_simple_ec_33h_fwd test case

• Prepare files and directories for test. Test directory should contain:
  • met_data directory
  • preprocessed_data directory (empty)
• While in test case directory, run command "GRIB2FLEXPART forward 1 preprocessed_data met_data/EL14091815 met_data/EL14091818 met_data/EL14091821 met_data/EL14091900 met_data/EL14091903 met_data/EL14091906 met_data/EL14091909 met_data/EL14091912 met_data/EL14091915 met_data/EL14091918 met_data/EL14091921 met_data/EL14092000"
• FP data are created in preprocessed_data directory.

Differences compared to original Flexpart test case: The standard Flexpart setup of config files is not needed at all in this case. Everything is provided on the command line.

Running FLEXPART using GRIB data

• Prepare files for test. Test directory should contain:
  • pathnames
  • options directory
  • preprocessed_data directory
  • preprocessed_data/AVAILABLE
  • output directory
• In options/COMMAND, set PREPROCESSED_DATA parameter to 0
• Run FLEXPART

Differences compared to original Flexpart test case:

• PREPROCESSED_DATA parameter in COMMAND file must exist and the value must be 0

---

DISCUSSION

Vertical grid coordinates

Issue

In FLEXPART the internal vertical coordinate system is initialised from the first meteorological file read in (that is, from its surface pressure and the orography field at the lower left corner of the met domain). If this is not treated in a consistent way, and the first file read in is different every time, then the vertical coordinate system and, when passed through the testing environment, errors appear.

Approach

There are three different approaches possible and only two have been prepared and made ready available to CTBTO. The two provided options are as follows:

1. Option A: produce the meteo file as they would be used in FLEXPART, in the exact same order. That means that, instead of processing one set of new files per day, every day a whole set of 15 days of meteorological data should be redone,but just twice: one forward and one backward. And they could use then these files for any simulation within that time frame. The run time would still be significantly reduced since the transformations on the fly would be skipped. In this option, the utility grib2flexpart would process all the 15 days meteo files at once to ensure the vertical grid is initialised with the first meteo data and then the data is useful for any runs following the same temporal configuration. This option is the one traditionally used in other modelling systems (CMAQ, WRF-CHEM, WRF, HYSPLIT …) whereby the user processes the meteorological driving data to match the runs to be done. Option A makes use of many of the structures in FLEXPART, going through reading of AVAILABLE, gridcheck until the processing of the meteorological data in getfields.

2. Option B: this option is a trade-off and imply some potential inconsistency in the treatment of the vertical coordinates. In this option we guarantee that only one single day of data needs to be processed (for backward and forward) nevertheless, although the number of internal vertical levels is kept the same, each time a file is generated a new system is initialised. For this case, each meteorological file (or in case of nests, mother and nest at the same time) may be processed via command line.

The third approach (Option C) is a combination of both and require the generation of one additional control file (with the risk it implies of being lost, moved…). This option would need a routine to initialise the vertical grid into a file (for GFS, ECMWF and for nests) with the height(kz) variable. This file should be read in at the time of processing the binary files.

Impact

There are two main impacts related to this issue:

1. Two options are provided instead of one and duplication is not necessarily good.

2. Potential inconsistencies (with undetermined effect in the final concentrations, we do not expect it to be very large due to the interpolations performed during runtime, but some sensitivity tests are needed) in the vertical treatment.

Nested configuration

Issue

FLEXPART running with ECMWF data has the potentiality of using nested input with increasing horizontal resolution. The vertical resolution of the input data is not allowed to change from mother to nest. This means that, in order to read nested domains and process them properly, a mother domain must be read in first (gridcheck.f90 and verttransform.f90 are the routines in charge of initialising the vertical grid).

Approach

Two actions has been taken to handle input nests properly:

1. the fp binary files for the nests are not produced as standalone files. Both a mother and a nest are processed for the same timestamp and one single fp binary files is produced.

2. the just mentioned mother and nest into a single fp file approach implies that FLEXPART, in the case of working with fp files, will not go through gridcheck and gridcheck_nest routines but only one time through fpgridcheck routine, that has been programmed specifically to check the input fp files in terms of dimensions.

Impact

There are two impacts resulting from this issue:

1. fp binary files must be processed for mother and nest together into one single fp file

2. the usual workflow of FLEXPART has been modified to account for the single fp file including the meteorological data from the mother and nested inputs.

Specific options and compatibility

Issue

This WO has included modifications of existing code and addition of functionalities. Nevertheless, traditional FLEXPART use had to be ensured.

Approach

The additional routines needed for handling the fp fp formats are provided with the source code and can be compiled with them. A new set of additional makefiles are provided to the user to compile single or multiple components of the new system. In order to activate the different routines and options, ifdef directive have been added throughout the code allowing for conditional use of routines or section of routines and thus limit to a minimum the modifications in the options files.

This has been supported by documentation.

Impact

New set of makefiles and additional compilation steps if the grib2flexpart and related libraries are needed.

Deep convection

Issue

FLEXPART has the option to run with deep convection. Due to its computational demands, the common user, included CTBTO so far, opts out this feature. This option requires additional handling of the meteorological input data and therefore the independent treatment of GFS, ECMWF and nested meteorological fields.

Approach

It is clear that to include deep convection, the variables needed for it must be stored in the fp binary files. They are indeed included. Nevertheless, the new routines to read in those and used them are not yet developed.

Investigations in this direction have already been done. The routines involved vary in algorithms depending on whether it is ECMWF data or NCEP data. For the former, native ECMWF levels are used whereas for the GFS version, the FLEXPART native model vertical levels are the ones used. Given that the fp files are already in native coordinates, the routines that will be kept are the ones from GFS. Initial tests show that the code automatically handles this but it would require further testing.

Impact

Deep convection has not yet been fully implemented for runs initialiased by fp data