| 1 | ********************************** |
|---|
| 2 | The executable script - ``run.sh`` |
|---|
| 3 | ********************************** |
|---|
| 4 | |
|---|
| 5 | The execution of ``flex_extract`` is done by the ``run.sh`` shell script, which is a wrapper script for the top-level Python script ``submit.py``. |
|---|
| 6 | The Python script constitutes the entry point to ECMWF data retrievals with ``flex_extract`` and controls the program flow. |
|---|
| 7 | |
|---|
| 8 | ``submit.py`` has two (or three) sources for input parameters with information about program flow and ECMWF data selection, the so-called ``CONTROL`` file, |
|---|
| 9 | the command line parameters, and the so-called ``ECMWF_ENV`` file. Command line parameters will override parameters specified in the ``CONTROL`` file. |
|---|
| 10 | |
|---|
| 11 | Based on this input information, ``flex_extract`` applies one of the application modes to either retrieve the ECMWF data via a web API on a local maschine, or submit a job script to an ECMWF server and retrieve the data there, and at the end sends the files to the local system. |
|---|
| 12 | |
|---|
| 13 | |
|---|
| 14 | |
|---|
| 15 | |
|---|
| 16 | Submission parameters |
|---|
| 17 | --------------------- |
|---|
| 18 | |
|---|
| 19 | |
|---|
| 20 | .. exceltable:: Parameters for submission |
|---|
| 21 | :file: ../../_files/SubmitParameters.xls |
|---|
| 22 | :header: 1 |
|---|
| 23 | :sheet: 0 |
|---|
| 24 | |
|---|
| 25 | |
|---|
| 26 | |
|---|
| 27 | |
|---|
| 28 | Content of ``run.sh`` |
|---|
| 29 | --------------------- |
|---|
| 30 | |
|---|
| 31 | .. literalinclude:: ../../../../../Run/run.sh |
|---|
| 32 | :language: bash |
|---|
| 33 | :caption: run.sh |
|---|
| 34 | |
|---|
| 35 | |
|---|
| 36 | |
|---|
| 37 | Usage of ``submit.py`` (optional) |
|---|
| 38 | --------------------------------- |
|---|
| 39 | |
|---|
| 40 | It is also possible to start ``flex_extract`` directly from command line by using the ``submit.py`` script instead of the wrapper shell script ``run.sh``. This top-level script is located in |
|---|
| 41 | ``flex_extract_vX.X/Source/Python`` and is executable. With the ``--help`` parameter |
|---|
| 42 | we see again all possible command line parameters. |
|---|
| 43 | |
|---|
| 44 | .. code-block:: bash |
|---|
| 45 | |
|---|
| 46 | submit.py --help |
|---|
| 47 | |
|---|
| 48 | usage: submit.py [-h] [--start_date START_DATE] [--end_date END_DATE] |
|---|
| 49 | [--date_chunk DATE_CHUNK] [--job_chunk JOB_CHUNK] |
|---|
| 50 | [--controlfile CONTROLFILE] [--basetime BASETIME] |
|---|
| 51 | [--step STEP] [--levelist LEVELIST] [--area AREA] |
|---|
| 52 | [--debug DEBUG] [--oper OPER] [--request REQUEST] |
|---|
| 53 | [--public PUBLIC] [--rrint RRINT] [--inputdir INPUTDIR] |
|---|
| 54 | [--outputdir OUTPUTDIR] [--ppid PPID] |
|---|
| 55 | [--job_template JOB_TEMPLATE] [--queue QUEUE] |
|---|
| 56 | |
|---|
| 57 | Retrieve FLEXPART input from ECMWF MARS archive |
|---|
| 58 | |
|---|
| 59 | optional arguments: |
|---|
| 60 | -h, --help show this help message and exit |
|---|
| 61 | --start_date START_DATE |
|---|
| 62 | start date YYYYMMDD (default: None) |
|---|
| 63 | --end_date END_DATE end_date YYYYMMDD (default: None) |
|---|
| 64 | --date_chunk DATE_CHUNK |
|---|
| 65 | # of days to be retrieved at once (default: None) |
|---|
| 66 | --job_chunk JOB_CHUNK |
|---|
| 67 | # of days to be retrieved within a single job |
|---|
| 68 | (default: None) |
|---|
| 69 | --controlfile CONTROLFILE |
|---|
| 70 | The file with all CONTROL parameters. (default: |
|---|
| 71 | CONTROL_EA5) |
|---|
| 72 | --basetime BASETIME base such as 0 or 12 (for half day retrievals) |
|---|
| 73 | (default: None) |
|---|
| 74 | --step STEP Forecast steps such as 00/to/48 (default: None) |
|---|
| 75 | --levelist LEVELIST Vertical levels to be retrieved, e.g. 30/to/60 |
|---|
| 76 | (default: None) |
|---|
| 77 | --area AREA area defined as north/west/south/east (default: None) |
|---|
| 78 | --debug DEBUG debug mode - leave temporary files intact (default: |
|---|
| 79 | None) |
|---|
| 80 | --oper OPER operational mode - prepares dates with environment |
|---|
| 81 | variables (default: None) |
|---|
| 82 | --request REQUEST list all mars requests in file mars_requests.dat |
|---|
| 83 | (default: None) |
|---|
| 84 | --public PUBLIC public mode - retrieves the public datasets (default: |
|---|
| 85 | None) |
|---|
| 86 | --rrint RRINT Selection of old or new precipitation interpolation: 0 |
|---|
| 87 | - old method 1 - new method (additional subgrid |
|---|
| 88 | points) (default: None) |
|---|
| 89 | --inputdir INPUTDIR Path to the temporary directory for the retrieval grib |
|---|
| 90 | files and other processing files. (default: None) |
|---|
| 91 | --outputdir OUTPUTDIR |
|---|
| 92 | Path to the final directory where the final FLEXPART |
|---|
| 93 | ready input files are stored. (default: None) |
|---|
| 94 | --ppid PPID This is the specify parent process id of a single |
|---|
| 95 | flex_extract run to identify the files. It is the |
|---|
| 96 | second number in the GRIB files. (default: None) |
|---|
| 97 | --job_template JOB_TEMPLATE |
|---|
| 98 | The job template file which are adapted to be |
|---|
| 99 | submitted to the batch system on ECMWF server. |
|---|
| 100 | (default: job.temp) |
|---|
| 101 | --queue QUEUE The ECMWF server name for submission of the job script |
|---|
| 102 | to the batch system (e.g. ecgate | cca | ccb) |
|---|
| 103 | (default: None) |
|---|
| 104 | |
|---|
| 105 | |
|---|
| 106 | |
|---|
| 107 | |
|---|
| 108 | |
|---|
| 109 | |
|---|
| 110 | |
|---|
| 111 | .. toctree:: |
|---|
| 112 | :hidden: |
|---|
| 113 | :maxdepth: 2 |
|---|
