Changeset f61e1df in flex_extract.git for Documentation/html/_sources/quick_start.rst.txt
- Timestamp:
- Jul 9, 2020, 8:13:25 AM (4 years ago)
- Branches:
- master, ctbto, dev
- Children:
- 82564d8
- Parents:
- 3e13e02 (diff), 6931f61 (diff)
Note: this is a merge changeset, the changes displayed below correspond to the merge itself.
Use the (diff) links above to see all the changes relative to each parent. - File:
-
- 1 edited
-
Documentation/html/_sources/quick_start.rst.txt (modified) (25 diffs)
Legend:
- Unmodified
- Added
- Removed
-
Documentation/html/_sources/quick_start.rst.txt
rb1674ed r6931f61 1 ***** ******2 Quick Start 3 ***** ******4 5 `` Flex_extract`` by itself is a command-line tool. With version 7.1 an upstream shell script was implemented which calls``flex_extract`` with the command-line parameters.6 7 To submit a job with ``flex_extract`` change the working directory to the ``Run`` directory which is placed directly under the ``flex_extract_vX.X`` root directory (``X.X`` is the version number):1 ***** 2 Usage 3 ***** 4 5 ``flex_extract`` is a command-line tool. In the first versions, it was started via a korn shell script and since version 6, the entry point was a python script. From version 7.1, a bash shell script was implemented to call ``flex_extract`` with the command-line parameters. 6 7 To submit an extraction job, change the working directory to the subdirectory ``Run`` (directly under the ``flex_extract_vX.X`` root directory, where ``X.X`` is the version number): 8 8 9 9 .. code-block:: bash … … 11 11 cd <path-to-flex_extract_vX.X>/Run 12 12 13 Within this directory you can find everything you need to modify and run ``flex_extract``. The following tree shows a shortened list of directories and important files. The ``*`` serves as a wildcard. The brackets ``[]`` means that the file appearance depends on the mode of application and night not be present.13 Within this directory you can find everything you need to modify and run ``flex_extract``. The following tree shows a shortened list of directories and important files. The ``*`` serves as a wildcard. The brackets ``[]`` indicate that the file is present only in certain modes of application. 14 14 15 15 .. code-block:: bash … … 29 29 └── run.sh 30 30 31 The ``Jobscripts`` directory is used to store the Korn shell job scripts generated by a ``flex_extract`` run in the **Remote** or **Gateway** mode. They are used to submit the setup information to the ECMWF server and start the jobs in ECMWF's batch mode. The typical user must not touch these files. They will be generated from template files which are stored in the ``Templates`` directory under ``flex_extract_vX.X``. Usually there will be a ``compilejob.ksh`` and a ``job.ksh`` script which are explained in the section :doc:`Documentation/input`. In the rare case of operational data extraction there will be a ``joboper.ksh`` which is designed to get the time parametersfrom environment variables at the ECMWF servers.32 33 The ``Controls`` directory contains a number of example ``CONTROL`` files. These ``CONTROL`` files represent the current range of possible dataset retrievals with ``flex_extract``. Some parameters in the ``CONTROL`` files can be adapted and some others should not be changed. In this :doc:`quick_start` guide we explain how an extraction with ``flex_extract`` can be started in the different :doc:`Documentation/Overview/app_modes` and point out some specifics of each dataset and ``CONTROL`` file.31 The ``Jobscripts`` directory is used to store the Korn shell job scripts generated by a ``flex_extract`` run in the **Remote** or **Gateway** mode. They are used to submit the setup information to the ECMWF server and start the jobs in ECMWF's batch mode. The typical user must not touch these files. They are generated from template files which are stored in the ``Templates`` directory under ``flex_extract_vX.X``. Usually there will be a ``compilejob.ksh`` and a ``job.ksh`` script which are explained in the section :doc:`Documentation/input`. In the rare case of operational data extraction there will be a ``joboper.ksh`` which reads time information from environment variables at the ECMWF servers. 32 33 The ``Controls`` directory contains a number of sample ``CONTROL`` files. They cover the current range of possible kinds of extractions. Some parameters in the ``CONTROL`` files can be adapted and some others should not be changed. In this :doc:`quick_start` guide we explain how an extraction with ``flex_extract`` can be started in the different :doc:`Documentation/Overview/app_modes` and point out some specifics of each dataset and ``CONTROL`` file. 34 34 35 35 Directly under ``Run`` you find the files ``run.sh`` and ``run_local.sh`` and according to your selected :doc:`Documentation/Overview/app_modes` there might also be a file named ``ECMWF_ENV`` for the user credentials to quickly and automatically access ECMWF servers. 36 36 37 From version 7.1 on, the ``run.sh`` (or ``run_local.sh``) script is the main accesspoint to ``flex_extract``.37 From version 7.1 on, the ``run.sh`` (or ``run_local.sh``) script is the main entry point to ``flex_extract``. 38 38 39 39 .. note:: 40 40 41 Note that , for experienced users (or users of older versions), it is still possible to access ``flex_extract`` directly via the ``submit.py`` script which can be found in thedirectory ``flex_extract_vX.X/Source/Python``.41 Note that for experienced users (or users of older versions), it is still possible to start ``flex_extract`` directly via the ``submit.py`` script in directory ``flex_extract_vX.X/Source/Python``. 42 42 43 43 … … 46 46 =============== 47 47 48 To actually start a job with ``flex_extract`` it is only necessary to submit the ``run.sh`` or ``run_local.sh`` script on the command-line. For the specification of dataset and the selection of the access mode it is necessary to select and modify a ``CONTROL`` file and change the parameters in the user section of the ``run`` scripts. The following sections describes the differences in the application modes and where the results will be stored.48 To actually start a job with ``flex_extract`` it is sufficient to start either ``run.sh`` or ``run_local.sh``. Data sets and access modes are selected in ``CONTROL`` files and within the user section of the ``run`` scripts. One should select one of the sample ``CONTROL`` files. The following sections describes the differences in the application modes and where the results will be stored. 49 49 50 50 … … 52 52 ------------------------ 53 53 54 For member state users it is recommended to use the *remote* or *gateway* mode, especially for more demanding tasks, to retrieve and convert data on ECMWF machines and to transfer only the final output filesto the local host.54 For member-state users it is recommended to use the *remote* or *gateway* mode, especially for more demanding tasks, which retrieve and convert the data on ECMWF machines; only the final output files are transferrred to the local host. 55 55 56 56 Remote mode … … 65 65 66 66 # On ECMWF server 67 [<ecuid>@ecgb11 ~]$ module load python368 67 [<ecuid>@ecgb11 ~]$ cd flex_extract_vX.X/Run 69 68 … … 104 103 OUTPUTDIR=None 105 104 PP_ID=None 106 JOB_TEMPLATE='job .temp'105 JOB_TEMPLATE='jobscript.template' 107 106 CONTROLFILE='CONTROL_CERA' 108 107 DEBUG=0 … … 138 137 ls -rthl 139 138 140 The last command lists the most recent logs and temporary retrieval directories (usually `` pythonXXXXX``, where XXXXX is the process id). Under ``pythonXXXXX`` a copy of the ``CONTROL`` file is stored under the name ``CONTROL``, the protocol is stored in the file ``prot`` and the temporary files as well as the resulting files are stored in a directory ``work``. The original name of the ``CONTROL`` file is stored in this new file under parameter ``controlfile``.139 The last command lists the most recent logs and temporary retrieval directories (usually ``extractXXXXX``, where XXXXX is the process id). Under ``extractXXXXX`` a copy of the ``CONTROL`` file is stored under the name ``CONTROL``, the protocol is stored in the file ``prot`` and the temporary files as well as the resulting files are stored in a directory ``work``. The original name of the ``CONTROL`` file is stored in this new file under parameter ``controlfile``. 141 140 142 141 .. code-block:: bash 143 142 :caption: "Example structure of ``flex_extract`` output directory on ECMWF servers." 144 143 145 pythonXXXXX144 extractXXXXX 146 145 ├── CONTROL 147 146 ├── prot … … 263 262 CONTROL_OD.OPER.FC.gauss.highres 264 263 CONTROL_OD.OPER.FC.operational 265 CONTROL_OD.OPER.FC.twice aday.1hourly266 CONTROL_OD.OPER.FC.twice aday.3hourly264 CONTROL_OD.OPER.FC.twicedaily.1hourly 265 CONTROL_OD.OPER.FC.twicedaily.3hourly 267 266 268 267 … … 277 276 278 277 279 A common problem for beginners in retrieving ECMWF datasets is the mismatch in the definition of these parameters. For example, if you would like to retrieve operational data before ``June 25th 2013`` and set the maximum level to ``137`` you will get an error because this number of levels was first introduced at this effective day. So, be cautious in the combination of space and time resolution as well as the field types which are not available all the time.278 A common problem for beginners in retrieving ECMWF datasets is a mismatch in the choice of values for these parameters. For example, if you try to retrieve operational data for 24 June 2013 or earlier and set the maximum level to 137, you will get an error because this number of levels was introduced only on 25 June 2013. Thus, be careful in the combination of space and time resolution as well as the field types. 280 279 281 280 282 281 .. note:: 283 282 284 Sometimes it might not be clear how specific parameters in the control file must be set in terms of format. Please see the description of the parameters in section `CONTROL parameters <Documentation/Input/control_params.html>`_ or have a look at the ECMWF user documentation for `MARS keywords <https://confluence.ecmwf.int/display/UDOC/MARS+keywords>`_ 285 286 287 In the following we shortly discuss the main retrieval opportunities of the different datasets and categoize the ``CONTROL`` files. 283 Sometimes it might not be clear how specific parameters in the control file must be set in terms of format. Please consult the description of the parameters in section `CONTROL parameters <Documentation/Input/control_params.html>`_ or have a look at the ECMWF user documentation for `MARS keywords <https://confluence.ecmwf.int/display/UDOC/MARS+keywords>`_ 284 285 In the following, we shortly discuss the typical retrievals for the different datasets and point to the respective ``CONTROL`` files. 288 286 289 287 … … 291 289 --------------- 292 290 293 The main difference in the definition of a ``CONRTOL`` file for a public dataset is the setting of the parameter ``DATASET``. This specification enables the selection of a public dataset in MARS. Otherwisethe request would not find the dataset.291 The main characteristic in the definition of a ``CONTROL`` file for a public dataset is the parameter ``DATASET``. Its specification enables the selection of a public dataset in MARS. Without this parameter, the request would not find the dataset. 294 292 For the two public datasets *CERA-20C* and *ERA-Interim* an example file with the ending ``.public`` is provided and can be used straightaway. 295 293 … … 299 297 CONTROL_EI.public 300 298 301 For *CERA-20C* it seems that there are no differences in the dataset against the full dataset, while the *public ERA-Interim* has only analysis fields every 6 hour without filling forecasts in between for model levels. Thereforeit is only possible to retrieve 6-hourly data for *public ERA-Interim*.299 For *CERA-20C* it seems that there are no differences compared the full dataset, whereas the *public ERA-Interim* has only 6-hourly analysis fields, without forecasts to fill in between, for model levels. Therefore, it is only possible to retrieve 6-hourly data for *public ERA-Interim*. 302 300 303 301 .. note:: 304 302 305 In general, *ERA5* is a public dataset. However, since the model levels are not yet publicly available, it is not possible to retrieve *ERA5* data to drive the ``FLEXPART`` model. As soon as this is possible it will be announced at the community website and per newsletter.303 In principle, *ERA5* is a public dataset. However, since the model levels are not yet publicly available, it is not possible to retrieve *ERA5* data to drive the ``FLEXPART`` model. As soon as this is possible it will be announced at the community website and on the FLEXPART user email list. 306 304 307 305 … … 309 307 ---- 310 308 311 For this dataset it is important to keep in mind that the dataset is available for the period 09/1901 until 12/2010 and the temporal resolution is limited to 3-hourly fields. 312 It is also a pure ensemble data assimilation dataset and is stored under the ``enda`` stream. It has ``10`` ensemble members. The example ``CONTROL`` files will only select the first member (``number=0``). You may change this to another number or a list of numbers (e.g. ``NUMBER 0/to/10``). 313 Another important difference to all other datasets is the forecast starting time which is 18 UTC. Which means that the forecast in *CERA-20C* for flux fields is 12 hours long. Since the forecast extends over a single day we need to extract one day in advance and one day subsequently. This is automatically done in ``flex_extract``. 309 For this dataset, it is important to keep in mind that it is available for the period 09/1901 until 12/2010, and that the temporal resolution is limited to 3 h. 310 It is also a pure ensemble data assimilation dataset and is stored under the ``enda`` stream. 311 There are 10 ensemble members. The example ``CONTROL`` files retrieves the first member only (``number=0``). You may change this to another number or a list of numbers (e.g. ``NUMBER 0/to/10``). 312 Another important difference to all other datasets is that there is one forecast per day, starting at 18 UTC. The forecast lead time is 24 hours and extends beyond the calendar day. Therefore, ``flex_extract`` needs to extract also the day before the first day for which data are desired, which is handled automatically. 314 313 315 314 … … 317 316 ----- 318 317 319 This is the newest re-analysis dataset and has a temporal resolution of 1-hourly analysis fields. Up to dateit is available until April 2019 with regular release of new months.320 The original horizontal resolution is ``0.28125°`` which needs some caution in the definition of the domain, since the length of the domain in longitude or latitude direction must be an exact multiple of the resolution. It might be easier for users to use ``0.25`` for the resolution which MARS will automatically interpolate.321 The forecast starting time is ``06/18 UTC`` which is important for the flux data. This should be set in the ``CONTROL`` file via the ``ACCTIME 06/18`` parameter in correspondence with ``ACCMAXSTEP 12``and ``ACCTYPE FC``.318 This is the latest re-analysis dataset, and has a temporal resolution of 1-h (analysis fields). At the time of writing, it is available until April 2019 with regular release of new months. 319 The original horizontal resolution is 0.28125° which needs some caution in the definition of the domain, since the length of the domain in longitude or latitude direction must be an integer multiple of the resolution. It is also possible to use ``0.25`` for the resolution; MARS will then automatically interpolate to this resolution which is still close enough to be acceptable. 320 The forecast starting time is ``06/18 UTC`` which is important for the flux data. Correspondingly, one should set in the ``CONTROL`` file ``ACCTIME 06/18``, ``ACCMAXSTEP 12``, and ``ACCTYPE FC``. 322 321 323 322 .. note:: 324 323 325 We know that *ERA5* also has an ensemble data assimilation system but this is not yet retrievable with ``flex_extract`` since the deaccumulation of the flux fields works differently in this stream. Ensemble retrieval for *ERA5* is a future ToDo.324 *ERA5* also includes an ensemble data assimilation system but related fields are not yet retrievable with ``flex_extract`` since the deaccumulation of the flux fields works differently in this stream. Ensemble field retrieval for *ERA5* is a *to-do* for the future. 326 325 327 326 … … 330 329 ----------- 331 330 332 This re-analysis dataset will exceed its end of production at 31st August 2019! 333 It is then available from 1st January 1979 to 31st August 2019. The ``etadot`` is not available in this dataset. Therefore ``flex_extract`` must select the ``GAUSS`` parameter to retrieve the divergence field in addition. The vertical velocity is the calculated with the continuity equation in the Fortran program ``calc_etadot``. Since the analysis fields are only available for every 6th hour, the dataset can be made 3 hourly by adding forecast fields in between. No ensemble members are available. 334 331 The production of this re-analysis dataset has stopped on 31 August 2019! 332 It is available for the period from 1 January 1979 to 31 August 2019. The ``etadot`` parameter is not available in this dataset. Therefore, one must use the ``GAUSS`` parameter, which retrieves the divergence field in addition and calculates the vertical velocity from the continuity equation in the Fortran program ``calc_etadot``. While the analysis fields are only available for every 6th hour, the dataset can be made 3-hourly by adding forecast fields in between. No ensemble members are available. 335 333 336 334 … … 338 336 ---------------- 339 337 340 This is the real time atmospheric model in high resolution with a 10-day forecast. This means it underwent regular adaptations and improvements over the years. Hence, retrieving data from this dataset needs extra attention in selecting correct settings of parameter. See :ref:`ref-tab-dataset-cmp` for the most important parameters.341 Nowadays, it is available 1 hourly by filling the gaps of the 6 hourly analysis fields with 1 hourly forecast fields. Since 4th June 2008 the eta coordinate is directly available so that ``ETA`` should be set to ``1`` to save computation time. The horizontal resolution can be up to ``0.1°`` and in combination with ``137`` vertical levels can lead to troubles in retrieving this high resolution dataset in terms of job duration and quota exceedence. 342 It is recommended to submit such high resolution cases forsingle day retrievals (see ``JOB_CHUNK`` parameter in ``run.sh`` script) to avoid job failures due to exceeding limits.343 344 ``CONTROL`` files for normal dailyretrievals with a mix of analysis and forecast fields are listed below:338 This data set provides the output of the real-time atmospheric model runs in high resolution, including 10-day forecasts. The model undergoes frequent adaptations and improvements. Thus, retrieving data from this dataset requires extra attention in selecting correct settings of the parameters. See :ref:`[Table of datasets]<ref-tab-dataset-cmp>` for the most important parameters. 339 Currently, fields can be retrieved at 1 h temporal resolution by filling the gaps between analysis fields with 1-hourly forecast fields. Since 4 June 2008, the eta coordinate vertical velocity is directly available from MARS, therefore ``ETA`` should be set to ``1`` to save computation time. The horizontal resolution can be up to ``0.1°`` and in combination with ``137`` vertical levels can lead to problems in terms of job duration and disk space quota. 340 It is recommended to submit such high resolution cases as single day retrievals (see ``JOB_CHUNK`` parameter in ``run.sh`` script) to avoid job failures due to exceeding limits. 341 342 ``CONTROL`` files for standard retrievals with a mix of analysis and forecast fields are listed below: 345 343 346 344 .. code-block:: bash … … 351 349 CONTROL_OD.OPER.FC.gauss.highres 352 350 353 These files defines the minimum number of parameters necessary to retrieve a daily subset. The setup of field types is optimal and should only be changed if the user understands what he does. The grid, domain and temporal resolution can be changed according to availability. 354 355 351 These files defines the minimum number of parameters necessary to retrieve a daily subset. The given settings for the TYPE parameter are already optimised, and should only be changed if you know what you are doing. Grid, domain, and temporal resolution may be changed according to availability. 352 356 353 357 354 .. note:: 358 355 359 Please see `Information about MARS retrievement <https://confluence.ecmwf.int/display/UDOC/Retrieve#Retrieve-Retrievalefficiency>`_ to get to know hints about retrieval efficiency and troubleshooting. 360 361 362 363 Pure forecast 364 It is possible to retrieve pure forecasts exceeding a day. The forecast period available depends on the date and forecast field type. Please use MARS catalogue to check the availability. Below are some examples for 36 hour forecast of *Forecast (FC)*, *Control forecast (CF)* and *Calibration/Validation forecast (CV)*. 365 The *CV* field type was only available 3-hourly from 2006 up to 2016. It is recommended to use the *CF* type since this is available from 1992 (3-hourly) on up to today in 1-hourly temporal resolution. *CV* and *CF* field types belong to the *Ensemble prediction system (ENFO)* which contain 50 ensemble members. 366 Please be aware that in this case it is necessary to set the specific type for flux fields explicitly, otherwise it could select a default value which might be different from what you expect! 356 Please see `Information about MARS retrievement <https://confluence.ecmwf.int/display/UDOC/Retrieve#Retrieve-Retrievalefficiency>`_ for hints about retrieval efficiency and troubleshooting. 357 358 359 Long forecast 360 It is possible to retrieve long forecasts exceeding one day. The forecast period available depends on the date and forecast field type. Please use MARS catalogue to check the availability. Below are some examples for 36 hour forecasts of *Forecast (FC)*, *Control forecast (CF)* and *Calibration/Validation forecast (CV)*. 361 The *CV* field type was only available 3-hourly from 2006 up to 2016. It is recommended to use the *CF* type since this is available from 1992 (3-hourly) on up to today (1-hourly). *CV* and *CF* field types belong to the *Ensemble prediction system (ENFO)* which currently works with 50 ensemble members. 362 Please be aware that in this case it is necessary to set the type for flux fields explicitly, otherwise a default value might be selected, different from what you expect! 367 363 368 364 .. code-block:: bash … … 373 369 374 370 375 376 371 Half-day retrievals 377 If a forecast for just half a day is wanted it can be done by substituting the analysis fields also by forecast fields as shown in files with ``twiceaday`` in it. They produce a full day retrieval with pure 12 hour forecasts twice a day. It is also possible to use the operational version which would get the time information from ECMWF's environmental variables and therefore get the newest forecast per day. This version uses a ``BASETIME`` parameter which tells MARS to extract the exact 12 hours upfront to the selected date. If the ``CONTROL`` file with ``basetime`` in the filename is used this can be done for any other datetoo.372 If a forecast is wanted for half a day only, this can be done by substituting the analysis fields by forecast fields as shown in files with ``twicedaily`` in their name. They produce a full-day retrieval with pure 12 hour forecasts, twice a day. It is also possible to use the operational version which would obtain the time information from ECMWF's environment variables and therefore use the newest forecast for each day. This version uses a ``BASETIME`` parameter which tells MARS to extract the exact 12 hours up to the selected date. If the ``CONTROL`` file with ``basetime`` in the filename is used, this can be done for any other date, too. 378 373 379 374 .. code-block:: bash … … 381 376 CONTROL_OD.OPER.FC.eta.basetime 382 377 CONTROL_OD.OPER.FC.operational 383 CONTROL_OD.OPER.FC.twiceaday.1hourly 384 CONTROL_OD.OPER.FC.twiceaday.3hourly 385 386 378 CONTROL_OD.OPER.FC.twicedaily.1hourly 379 CONTROL_OD.OPER.FC.twicedaily.3hourly 387 380 388 381 389 382 Ensemble members 390 The retrieval of ensemble members were already mentioned in the pure forecast section and for *CERA-20C* data. 391 In this ``flex_extract`` version there is an additional possibility to retrieve the *Ensemble Long window Data Assimilation (ELDA)* stream from the real-time dataset. This model version has (up to May 2019) 25 ensemble members and a control run (``number 0``). Starting from June 2019 it has 50 ensemble members. Therefore we created the possibility to double up the 25 ensemble members (before June 2019) to 50 members by taking the original 25 members from MARS and subtracting 2 times the difference between the member value and the control value. This is done by selecting the parameter ``DOUBLEELDA`` and set it to ``1``. 392 383 The retrieval of ensemble members was already mentioned in the pure forecast section and for *CERA-20C* data. 384 This ``flex_extract`` version allows to retrieve the *Ensemble Long window Data Assimilation (ELDA)* stream from the operational dataset. Until May 2019, there were 25 ensemble members and a control run (``number 0``). Starting with June 2019, the number of ensemble members has been increased to 50. Therefore, we created the option to create 25 additional "pseudo-ensemble members" for periods before June 2019. The original 25 members from MARS are taken, and the difference between the member value and the control value is subtracted twice. This is done if the parameter ``DOUBLEELDA`` is included and set it to ``1``. 393 385 394 386 .. code-block:: bash … … 396 388 CONTROL_OD.ELDA.FC.eta.ens.double 397 389 CONTROL_OD.ENFO.PF.ens 398 399 400 390 401 391 … … 404 394 405 395 rrint 406 Decides if the precipitation flux data uses the old (``0``) or new (``1``) disaggregation scheme. See :doc:`Documentation/disagg` for explanaition.396 Selects the disaggregation scheme for precipitation flux: old (``0``) or new (``1``). See :doc:`Documentation/disagg` for explanation. 407 397 cwc 408 Decides if the total cloud water content will be retrieved (set to ``1``)in addition. This is the sum of cloud liquid and cloud ice water content.398 If present and set to ``1``, the total cloud water content will be retrieved in addition. This is the sum of cloud liquid and cloud ice water content. 409 399 addpar 410 With this parameter an additional list of 2-dimensional, non-flux parameters can be retrieved. Use format ``param1/param2/.../paramx`` to list the parameters. Please be consistent in using either the parameter IDs or the short names.400 With this parameter. an additional list of 2-dimensional, non-flux parameters can be retrieved. Use the format ``param1/param2/.../paramx`` to list the parameters. Please be consistent in using either the parameter IDs or the short names as defined by MARS. 411 401 doubleelda 412 Use this to double the ensemble member number by adding further disturbance to each member .402 Use this to double the ensemble member number by adding further disturbance to each member (to be used with 25 members). 413 403 debug 414 If set to ``1`` all temporary files were kept at the end. Otherwiseeverything except the final output files will be deleted.404 If set to ``1``, all temporary files are preserved. Otherwise, everything except the final output files will be deleted. 415 405 request 416 This produces an extra *csv* file ``mars_requests.csv`` where the content of each mars request of the job is stored. Useful for debugging and documentation.406 This produces an extra *csv* file ``mars_requests.csv`` where the content of each MARS request submitted within the job is stored, which is useful for debugging and documentation. Possible values are 0 for normal data retrieval, 1 for not retrieving data and just writing out the MARS requests, and 2 to retrieve data and write out requests. 417 407 mailfail 418 A t default the mail is send to the mail connected with the user account. Add additional email addresses if you want. But as soon as you enter a new mail, the default will be overwritten. If you would like to keep the mail from your user account, please add ``${USER}`` to the list ( comma seperated ) or mail addresses.419 420 421 422 Hints for definition of some parameter combinations 423 --------------------------------------------------- 424 425 Field types and times 426 Th is combination is very important. It defines the temporal resolution and which field type is extracted per time step.427 The time declaration for analysis (AN) fields uses the times of the specific analysis and (forecast time) steps have to be ``0``. The forecast field types (e.g. FC, CF, CV, PF) need to declare a combination of (forescast start) times and the (forecast) steps. Both of them together defines the actual time step. It is important to know the forecast starting times for the dataset to be retrieved, since they are different. In general it is enoughto give information for the exact time steps, but it is also possible to have more time step combinations of ``TYPE``, ``TIME`` and ``STEP`` because the temporal (hourly) resolution with the ``DTIME`` parameter will select the correct combinations.428 429 .. code-block:: bash 430 :caption: Example of a setting for the field types and temporal resolution. 408 As a default, e-mails are sent to the mail address defined for the ECMWF user account. It is possible to overwrite this by specifying one or more e-mail addresses (comma-separated list). In order to include the e-mail associated with the user account, add ``${USER}`` to the list. 409 410 411 Hints for proper definition of certain parameter combinations 412 ------------------------------------------------------------- 413 414 Field type and time 415 This combination is very important. It defines the temporal resolution and which field type is extracted on each time step. 416 The time declaration for analysis (AN) fields uses the times of the specific analysis while the (forecast time) step has to be ``0``. 417 The forecast field types (e.g. FC, CF, CV, PF) need to declare a combination of (forescast start) time and the (forecast) step. Together they define the actual time. It is important to know the forecast starting times for the dataset to be retrieved, since they are different. In general, it is sufficient to give information for the exact time steps, but it is also possible to have more time step combinations of ``TYPE``, ``TIME`` and ``STEP`` because the temporal (hourly) resolution with the ``DTIME`` parameter will select the correct combinations. 418 419 .. code-block:: bash 420 :caption: Example of a setting for the field types and temporal resolution. It will retrieve 3-hourly fields, with analyses at 00 and 12 UTC and the corresponding forecasts inbetween. 431 421 432 422 DTIME 3 … … 437 427 438 428 Vertical velocity 439 The vertical velocity for ``FLEXPART`` is not directly available from MARS. Therefore it has to be calculated. There are a couple of different options. The following parameters are responsible for the selection. See :doc:`Documentation/vertco` for a detailed explanation. The ``ETADIFF``, ``OMEGA`` and ``OMEGADIFF`` versions are only recommended for debugging and testing reasons. Usually it is a decision between ``GAUSS`` and ``ETA``, where for ``GAUSS`` spectral fields of the horizontal wind fields and the divergence are to be retrieved and used with the continuity equation to calculate the vertical velocity. For ``ETA`` the latitude/longitude fields of horizontal wind fields and eta-coordinate are to be retrieved. It is recommended to use ``ETA`` where possible due to a reduced computation time. 440 441 .. code-block:: bash 442 :caption: Example setting for the vertical coordinate retrieval. 429 The vertical velocity for ``FLEXPART`` is not directly available from MARS and has to be calculated. 430 There are several options for this, and the following parameters are responsible for the selection. See :doc:`Documentation/vertco` for a detailed explanation. Using ``ETADIFF 1``, ``OMEGA 1`` and ``OMEGADIFF 1`` is recommended for debugging and testing only. 431 Usually, one has to decide between ``GAUSS 1`` and ``ETA 1``. ``GAUSS 1`` means that spectral fields of the horizontal wind fields and the divergence are retrieved and that the vertical velocity is calculate using the continuity equation. ``ETA 1`` means that horizontal wind fields etadot are retrieved on a regular lat-lon grid. It is recommended to use ``ETA 1`` where possible, as there is a substantial computational overhead for solving the continuity equation. 432 433 .. code-block:: bash 434 :caption: Example setting for the vertical coordinate retrieval (recommended if etadot fields are available). 443 435 444 436 GAUSS 0 … … 451 443 452 444 Grid resolution and domain 453 The grid and domain selection depends on each other. The grid can be defined in the format of normal degrees (e.g. ``1.``) or as in older versions by 1/1000. degrees (e.g. ``1000`` for ``1°``). 454 After selecting the grid, the domain has to be defined in a way that the length of the domain in longitude or latitude direction must be an exact multiple of the grid. 455 The horizontal resolution for spectral fields will be set by the parameter ``RESOL``. For information about how to select an appropriate value you can read the explanation of the MARS keyword `here <https://confluence.ecmwf.int/display/UDOC/Post-processing+keywords#Post-processingkeywords-resol>`_ and in `this table <https://confluence.ecmwf.int/display/UDOC/Retrieve#Retrieve-Truncationbeforeinterpolation>`_. 456 457 .. code-block:: bash 458 :caption: Example setting for a northern hemisphere domain with a grid of ``0.25°``. 445 The grid and domain parameters depends on each other. ``grid`` refers to the grid resolution. It can be given as decimal values (e.g., ``1.`` meaning 1.0°), or as in previous versions of flex_extract, as integer values refering to 1/1000 degrees (e.g., ``1000`` means also 1°). The code applies common sense to determine what format is to be assumed. 446 After selecting grid, the ``domain`` has to be defined. The extension in longitude or latitude direction must be an integer multiple of ``grid``. 447 448 449 The horizontal resolution for spectral fields is set by the parameter ``RESOL``. For information about how to select an appropriate value please read the explanation of the MARS keyword RESOL as found `in this entry of the ECMWF on-line documentation <https://confluence.ecmwf.int/display/UDOC/Post-processing+keywords#Post-processingkeywords-resol>`_ and `this table (also ECMWF documentation) <https://confluence.ecmwf.int/display/UDOC/Retrieve#Retrieve-Truncationbeforeinterpolation>`_. 450 451 .. code-block:: bash 452 :caption: Example setting for a domain covering the northern hemisphere domain with a grid resolution of ``0.25°``. 459 453 460 454 GRID 0.25 … … 468 462 469 463 Flux data 470 The flux fields are accumulated forecast fields all the time. Since some re-analysis dataset nowadays have complete set of analysis fields in their temporal resolution it was important to define a new parameter set to define the flux fields since the information could not be taken from ``TYPE``, ``TIME`` and ``STEP`` any longer. Select a forecast field type ``ACCTYPE``, the forecast starting time ``ACCTIME`` and the maximum forecast step ``ACCMAXSTEP``. The ``DTIME`` parameter defines the temporal resolution for the whole period.471 464 Flux fields are always forecast fields and contain values of the fluxes accumulated since the start of the respective forecast. As certain re-analysis dataset cover all time steps with analysis fields, it was necessary to define a new parameter set for the definition of the flux fields. The following parameters are used specifically for flux fields. ``ACCTYPE`` is the field type (must be a type of forecast), ``ACCTIME`` the forecast starting time, and ``ACCMAXSTEP`` the maximum forecast step;``DTIME`` the temporal resolution. ACCTYPE is assumed to be the same during the whole period given by ACCTIME and ACCMAXSTEP. These values will be set automatically if not provided in a ``CONTROL`` file. 465 472 466 .. code-block:: bash 473 467 :caption: Example setting for the definition of flux fields. 468 474 469 475 470 DTIME 3 … … 478 473 ACCMAXSTEP 36 479 474 480 481 475 482 476 .. toctree::
Note: See TracChangeset
for help on using the changeset viewer.
