Changeset f20342a in flex_extract.git for For_developers/Sphinx/source/Documentation/output.rst
- Timestamp:
- May 27, 2020, 8:01:54 PM (4 years ago)
- Branches:
- master, ctbto, dev
- Children:
- 550435b
- Parents:
- a14839a
- File:
-
- 1 edited
Legend:
- Unmodified
- Added
- Removed
-
For_developers/Sphinx/source/Documentation/output.rst
rb1674ed rf20342a 1 1 *********** 2 Output Data2 Output data 3 3 *********** 4 4 5 The output data of ``flex_extract`` are separated mainly into temporary files and the final ``FLEXPART`` inputfiles:5 The output data of ``flex_extract`` can be divided into the final ``FLEXPART`` input files and temporary files: 6 6 7 7 +-----------------------------------------------+----------------------------------------------+ 8 8 | ``FLEXPART`` input files | Temporary files (saved in debug mode) | 9 9 +-----------------------------------------------+----------------------------------------------+ 10 | - Standard output file names | - MARS request file (opt)|10 | - Standard output file names | - MARS request file (optional) | 11 11 | - Output for pure forecast | - flux files | 12 12 | - Output for ensemble members | - VERTICAL.EC | … … 21 21 ======================== 22 22 23 The final output files of ``flex_extract`` are also the meteorological ``FLEXPART`` input files.24 The naming of these files dependon the kind of data extracted by ``flex_extract``.23 The final output files of ``flex_extract`` are the meteorological input files for ``FLEXPART``. 24 The naming convention for these files depends on the kind of data extracted by ``flex_extract``. 25 25 26 26 Standard output files 27 27 --------------------- 28 28 29 In general, there is a file for each time step with the filename format:29 In general, there is one file for each time named: 30 30 31 31 .. code-block:: bash … … 33 33 <prefix>YYMMDDHH 34 34 35 The ``prefix`` is by default defined as ``EN``and can be re-defined in the ``CONTROL`` file.36 Each file contains all meteorological fields needed by ``FLEXPART`` for all selected model levels for a specific time step.37 38 Here is an example output which lists the meteorological fields in a single file called ``CE00010800`` where we extracted only the lowest model level for demonstration reasons:35 where YY are the last two digits of the year, MM is the month, DD the day, and HH the hour (UTC). <prefix> is by default defined as EN, and can be re-defined in the ``CONTROL`` file. 36 Each file contains all meteorological fields at all levels as needed by ``FLEXPART``, valid for the time indicated in the file name. 37 38 Here is an example output which lists the meteorological fields in a single file called ``CE00010800`` (where we extracted only the lowest model level for demonstration purposes): 39 39 40 40 .. code-block:: bash … … 84 84 ------------------------------ 85 85 86 ``Flex_extract`` can retrieve forecasts which can be longer than 23 hours. To avoid collisions of time steps for forecasts of more than one daya new scheme for filenames in pure forecast mode is introduced:86 ``Flex_extract`` is able to retrieve forecasts with a lead time of more than 23 hours. In order to avoid collisions of time steps names, a new scheme for filenames in pure forecast mode is introduced: 87 87 88 88 .. code-block:: bash … … 90 90 <prefix>YYMMDD.HH.<FORECAST_STEP> 91 91 92 The ``<prefix>`` is, as in the standard output, by default ``EN`` and can be re-defined in the ``CONTROL`` file. ``YYMMDD`` is the date format and ``HH`` the forecast time which is the starting time for the forecasts. The ``FORECAST_STEP`` is a 3 92 The ``<prefix>`` is, as in the standard output, by default ``EN`` and can be re-defined in the ``CONTROL`` file. ``YYMMDD`` is the date format and ``HH`` the forecast time which is the starting time for the forecasts. The ``FORECAST_STEP`` is a 3-digit number which represents the forecast step in hours. 93 93 94 94 … … 96 96 ------------------------------------- 97 97 98 Ensembles can be retrieved and are addressed by the grib message parameter ``number``. The ensembles are saved per file and standard filenames are supplemented by the letter ``N`` and the ensemble member number in a 3digit format.98 ``Flex_extract`` is able to retrieve ensembles data; they are labelled by the grib message parameter ``number``. Each ensemble member is saved in a separate file, and standard filenames are supplemented by the letter ``N`` and the ensemble member number in a 3-digit format. 99 99 100 100 .. code-block:: bash … … 106 106 ------------------------------------------------------- 107 107 108 The new disaggregation method for precipitation fields produces two additional precipitation fields for each time step and precipitation type. They serve as sub-grid points in the original time interval. For details of the method see :doc:`disagg`. 109 The two additional fields are marked with the ``step`` parameter in the Grib messages and are set to "1" and "2" for sub-grid point 1 and 2 respectively. 110 The output filenames do not change in this case. 111 Below is an example list of precipitation fields in an output file generated with the new disaggregation method: 108 The new disaggregation method for precipitation fields produces two additional precipitation fields for each time step and precipitation type (large-scale and convective). They serve as sub-grid points in the original time interval. For details of the method see :doc:`disagg`. 109 The two additional fields are addressed using the ``step`` parameter in the GRIB messages, which 110 is set to "1" or "2", for sub-grid points 1 and 2, respectively. 111 The output file names are not altered. 112 An example of the list of precipitation fields in an output file generated with the new disaggregation method is found below: 112 113 113 114 .. code-block:: bash … … 129 130 =============== 130 131 131 ``Flex_extract`` works with a number of temporary data files which are usually deleted after a successful data extraction. They are only storedif the ``DEBUG`` mode is switched on (see :doc:`Input/control_params`).132 ``Flex_extract`` creates a number of temporary data files which are usually deleted at the end of a successful run. They are preserved only if the ``DEBUG`` mode is switched on (see :doc:`Input/control_params`). 132 133 133 134 MARS grib files … … 135 136 136 137 ``Flex_extract`` retrieves all meteorological fields from MARS and stores them in files ending with ``.grb``. 137 Since the request times and data transfer of MARS access are limited and ECMWF asks for efficiency in requesting data from MARS, ``flex_extract`` splits the overall data request in several smaller requests. Each request is stored in an extra ``.grb`` file and the file names are put together by several pieces of information: 138 Since there are limits implemented by ECMWF for the time per request and data transfer from MARS, 139 and as ECMWF asks for efficient MARS retrievals, ``flex_extract`` splits the overall data request 140 into several smaller requests. Each request is stored in its own ``.grb`` file, and the file 141 names are composed of several pieces of information: 138 142 139 143 .. code-block:: bash … … 144 148 145 149 Field type: 146 ``AN`` - Analysis, ``FC`` - Forecast, ``4V`` - 4 dvariational analysis, ``CV`` - Validation forecast, ``CF`` - Control forecast, ``PF`` - Perturbed forecast150 ``AN`` - Analysis, ``FC`` - Forecast, ``4V`` - 4D variational analysis, ``CV`` - Validation forecast, ``CF`` - Control forecast, ``PF`` - Perturbed forecast 147 151 Grid type: 148 ``SH`` - Spherical Harmonics, ``GG`` - Gaussian Grid, ``OG`` - Output Grid (typically lat /lon), ``_OROLSM`` - Orography parameter152 ``SH`` - Spherical Harmonics, ``GG`` - Gaussian Grid, ``OG`` - Output Grid (typically lat / lon), ``_OROLSM`` - Orography parameter 149 153 Temporal property: 150 154 ``__`` - instantaneous fields, ``_acc`` - accumulated fields 151 155 Level type: 152 ``ML`` - Model Level, ``SL`` - Surface Level156 ``ML`` - model level, ``SL`` - surface level 153 157 ppid: 154 The process number of the parent process of submitted script.158 The process number of the parent process of the script submitted. 155 159 pid: 156 The process number of the submitted script. 157 158 The process ids should avoid mixing of fields if several ``flex_extract`` jobs are performed in parallel (which is, however, not recommended). The date format is YYYYMMDDHH. 159 160 Example ``.grb`` files for a day of CERA-20C data: 160 The process number of the script submitted. 161 162 163 Example ``.grb`` files for one day of CERA-20C data: 161 164 162 165 .. code-block:: bash … … 172 175 ----------------- 173 176 174 This file is a ``csv`` file called ``mars_requests.csv`` with a list of the actual settings of MARS request parameters (one request per line) in a flex_extract job. It is used for documenting the data which were retrieved and for testing reasons. 175 176 Each request consist of the following parameters, whose meaning mainly can be taken from :doc:`Input/control_params` or :doc:`Input/run`: 177 This file is a ``csv`` file called ``mars_requests.csv`` listing the actual settings of the MARS 178 request (one request per line) in a flex_extract job. 179 It is used for documenting which data were retrieved, and for testing. 180 181 Each request consists of the following parameters, whose meaning mostly can be taken from :doc:`Input/control_params` or :doc:`Input/run`: 177 182 request_number, accuracy, area, dataset, date, expver, gaussian, grid, levelist, levtype, marsclass, number, param, repres, resol, step, stream, target, time, type 178 183 179 Example output of a one day retrieval of CERA-20cdata:184 Example output of a one-day retrieval of CERA-20C data: 180 185 181 186 .. code-block:: bash … … 192 197 ----------- 193 198 194 The vertical discretization of model levels. This file contains the ``A`` and ``B`` parameters to calculate the model level height in meters. 199 This file contains information describing the vertical discretisation (model levels) 200 in form of the ``A`` and ``B`` parameters which allow to calculate the actual pressure of a model level from the surface pressure. 195 201 196 202 … … 198 204 ---------- 199 205 200 This file is usually called ``date_time_stepRange.idx``. It contains indices pointing to specific grib messages from one or more grib files. The messages are selected with a composition of grib message keywords. 201 202 203 flux files 206 This file is called ``date_time_stepRange.idx``. It contains indices pointing to specific grib messages from one or more grib files. The messages are selected with a composition of grib message keywords. 207 #PS NEEDS MORE DESCRIPTION 208 209 210 Flux files 204 211 ---------- 205 212 206 The flux files contain the de-accumulated and dis-aggregated flux fields of large scale and convective precipitation, eastward turbulent surface stress, northward turbulent surface stress, surface sensible heat fluxand the surface net solar radiation.213 The flux files contain the de-accumulated and dis-aggregated flux fields of large-scale and convective precipitation, east- and northward turbulent surface stresses, the surface sensible heat flux, and the surface net solar radiation. 207 214 208 215 .. code-block:: bash … … 210 217 flux<date>[.N<xxx>][.<xxx>] 211 218 212 The date format is YYYYMMDDHH . The optional block ``[.N<xxx>]`` marks the ensemble forecast number, where ``<xxx>`` is the ensemble member number. The optional block ``[.<xxx>]`` marks a pure forecast with ``<xxx>`` being the forecast step.219 The date format is YYYYMMDDHH as explained before. The optional block ``[.N<xxx>]`` is used for the ensemble forecast date, where ``<xxx>`` is the ensemble member number. The optional block ``[.<xxx>]`` marks a pure forecast with ``<xxx>`` being the forecast step. 213 220 214 221 .. note:: 215 222 216 In the case of the new dis-aggregation method for precipitation, two new sub-intervals are added in between each time interval. They are identified by the forecast step parameter which is ``0`` for the original time interval and ``1`` or ``2`` for the two new intervals respectively.223 In the case of the new dis-aggregation method for precipitation, two new sub-intervals are added in between each time interval. They are identified by the forecast step parameter which is ``0`` for the original time interval, and ``1`` or ``2``, respectively, for the two new intervals. 217 224 218 225 … … 226 233 fort.xx 227 234 228 where ``xx`` is thenumber which defines the meteorological fields stored in these files.229 They are generated by the Python part of ``flex_extract`` by just splitting the meteorological fields for a unique time stamp from the ``*.grb`` files into the ``fort`` files.230 The following table defines the numbers with their corresponding content.235 where ``xx`` is a number which defines the meteorological fields stored in these files. 236 They are generated by the Python code in ``flex_extract`` by splitting the meteorological fields for a unique time stamp from the ``*.grb`` files, storing them under the names ``fort.<XX>`` where <XX> represents some number. 237 The following table defines the numbers and the corresponding content: 231 238 232 239 .. csv-table:: Content of fort - files … … 240 247 "16", "surface fields" 241 248 "17", "specific humidity" 242 "18", "surface specific humidity (reduced gaussian)"243 "19", " vertical velocity (pressure) (optional)"249 "18", "surface specific humidity (reduced Gaussian grid)" 250 "19", "omega (vertical velocity in pressure coordinates) (optional)" 244 251 "21", "eta-coordinate vertical velocity (optional)" 245 "22", "total cloud 246 247 Some of the fields are solely retrieved with specific settings, e. g. the eta-coordinate vertical velocity is not available in ERA-Interim datasets and the total cloud water content is an optional fieldfor ``FLEXPART v10`` and newer.252 "22", "total cloud-water content (optional)" 253 254 Some of the fields are solely retrieved with specific settings, e. g., the eta-coordinate vertical velocity is not available in ERA-Interim datasets, and the total cloud-water content is an optional field which is useful for ``FLEXPART v10`` and newer. 248 255 249 256 The ``calc_etadot`` program saves its results in file ``fort.15`` which typically contains: … … 259 266 .. note:: 260 267 261 The ``fort.4`` file is the namelist file to drivethe Fortran program ``calc_etadot``. It is therefore also an input file.268 The ``fort.4`` file is the namelist file to control the Fortran program ``calc_etadot``. It is therefore also an input file. 262 269 263 270 Example of a namelist:
Note: See TracChangeset
for help on using the changeset viewer.
