[0b00607] | 1 | ************ |
---|
[f20342a] | 2 | Program flow |
---|
[0b00607] | 3 | ************ |
---|
[274f9ef] | 4 | |
---|
[0b00607] | 5 | |
---|
| 6 | |
---|
| 7 | |
---|
| 8 | General program flow |
---|
| 9 | ==================== |
---|
| 10 | |
---|
| 11 | |
---|
| 12 | The following flow diagram shows the general steps performed by ``flex_extract``. |
---|
| 13 | |
---|
| 14 | .. _ref-fig-submit: |
---|
| 15 | |
---|
| 16 | .. figure:: ../../_files/submit.png |
---|
| 17 | |
---|
[f20342a] | 18 | Overview of the call of the``submit.py`` Python script and raw sequence of work steps in ``flex_extract``. |
---|
[0b00607] | 19 | |
---|
| 20 | |
---|
[f20342a] | 21 | The ``submit.py`` Python script is called by the shell script ``run.sh`` or ``run_local.sh`` and accomplishes the following steps: |
---|
[0b00607] | 22 | |
---|
[f20342a] | 23 | 1. Setup of control data: |
---|
| 24 | Command-line and ``CONTROL``-file parameters are read, as well as (optionally) the ECMWF user credentials. Depending the :doc:`app_modes`, a job script might be prepared which is then sent to the ECMWF queue. |
---|
| 25 | 2. Retrieval of data from MARS: |
---|
| 26 | MARS requests are created either on the local machine or on the ECMWF server and then submitted which retrieve the data and store them in GRIB files. If the parameter ``REQUEST`` was set ``1``, the data are not retrieved and instead a file ``mars_requests.csv`` is created, which contains a list of the MARS requests and their settings. If ``REQEST`` is set to ``2``, the csv file is created in addition to retrieving the data. The requests are created in an optimised way by splitting with respect to time, jobs and parameters. |
---|
| 27 | 3. Post-processing of data to create final ``FLEXPART`` input files: |
---|
| 28 | After all data have been retrieved, flux fields are disaggregated (`see here <../disagg.html>`_ ) and the vertical velocity is calculated (`see here <../vertco.html>`_) by the Fortran program ``calc_etadot``. Finally, the GRIB fields are merged into a single grib file per time step containing all the fields for ``FLEXPART``. Since model-level fields are typically in *GRIB2* format, whereas surface-level fields are still in *GRIB1* format, they will be converted into GRIB2 if parameter ``FORMAT`` is set to *GRIB2*. Please note, however, that older versions of FLEXPART may have difficulties to read these *GRIB2* files since some parameter IDs have been change in *GRIB2*. If the retrieval is executed remotely at ECMWF, the resulting files will be sent to the local gateway server via the ``ECtrans`` utility if the parameter ``ECTRANS`` is set to ``1`` and the parameters ``GATEWAY``, ``DESTINATION`` have been set properly during installation. The status of the transfer can be checked with the command ``ecaccess-ectrans-list`` (on the local gateway server). If the script is executed locally, the progress of the script can be followed with the usual Linux tools. |
---|
[0b00607] | 29 | |
---|
| 30 | |
---|
| 31 | |
---|
| 32 | Workflows of different application modes |
---|
| 33 | ======================================== |
---|
| 34 | |
---|
[f20342a] | 35 | The following diagrams show how different the program flow is for the different :doc:`app_modes`: |
---|
[0b00607] | 36 | |
---|
| 37 | +-------------------------------------------------+------------------------------------------------+ |
---|
| 38 | | .. figure:: ../../_files/mode_remote.png | .. figure:: ../../_files/mode_gateway.png | |
---|
| 39 | +-------------------------------------------------+------------------------------------------------+ |
---|
| 40 | |
---|
| 41 | +-------------------------------------------------+------------------------------------------------+ |
---|
| 42 | | .. figure:: ../../_files/mode_local_member.png | .. figure:: ../../_files/mode_local_public.png | |
---|
| 43 | +-------------------------------------------------+------------------------------------------------+ |
---|
| 44 | |
---|
| 45 | |
---|
| 46 | Example application setting for a local member user |
---|
| 47 | =================================================== |
---|
| 48 | |
---|
| 49 | .. figure:: ../../_files/ex_runlocal_en.png |
---|
| 50 | |
---|
| 51 | |
---|
| 52 | |
---|
| 53 | |
---|
| 54 | .. toctree:: |
---|
| 55 | :hidden: |
---|
| 56 | :maxdepth: 2 |
---|