1 | *********** |
---|
2 | Output data |
---|
3 | *********** |
---|
4 | |
---|
5 | The output data of ``flex_extract`` can be divided into the final ``FLEXPART`` input files and temporary files: |
---|
6 | |
---|
7 | +-----------------------------------------------+----------------------------------------------+ |
---|
8 | | ``FLEXPART`` input files | Temporary files (saved in debug mode) | |
---|
9 | +-----------------------------------------------+----------------------------------------------+ |
---|
10 | | - Standard output file names | - MARS request file (optional) | |
---|
11 | | - Output for pure forecast | - flux files | |
---|
12 | | - Output for ensemble members | - VERTICAL.EC | |
---|
13 | | - Output for new precip. disaggregation | - index file | |
---|
14 | | | - fort files | |
---|
15 | | | - MARS grib files | |
---|
16 | +-----------------------------------------------+----------------------------------------------+ |
---|
17 | |
---|
18 | |
---|
19 | |
---|
20 | ``FLEXPART`` input files |
---|
21 | ======================== |
---|
22 | |
---|
23 | The final output files of ``flex_extract`` are the meteorological input files for ``FLEXPART``. |
---|
24 | The naming convention for these files depends on the kind of data extracted by ``flex_extract``. |
---|
25 | |
---|
26 | Standard output files |
---|
27 | --------------------- |
---|
28 | |
---|
29 | In general, there is one file for each time named: |
---|
30 | |
---|
31 | .. code-block:: bash |
---|
32 | |
---|
33 | <prefix>YYMMDDHH |
---|
34 | |
---|
35 | where YY are the last two digits of the year, MM is the month, DD the day, and HH the hour (UTC). <prefix> is by default defined as EN, and can be re-defined in the ``CONTROL`` file. |
---|
36 | Each file contains all meteorological fields at all levels as needed by ``FLEXPART``, valid for the time indicated in the file name. |
---|
37 | |
---|
38 | Here is an example output which lists the meteorological fields in a single file called ``CE00010800`` (where we extracted only the lowest model level for demonstration purposes): |
---|
39 | |
---|
40 | .. code-block:: bash |
---|
41 | |
---|
42 | $ grib_ls CE00010800 |
---|
43 | |
---|
44 | edition centre date dataType gridType stepRange typeOfLevel level shortName packingType |
---|
45 | 2 ecmf 20000108 an regular_ll 0 hybrid 91 u grid_simple |
---|
46 | 2 ecmf 20000108 an regular_ll 0 hybrid 91 v grid_simple |
---|
47 | 2 ecmf 20000108 an regular_ll 0 hybrid 91 etadot grid_simple |
---|
48 | 2 ecmf 20000108 an regular_ll 0 hybrid 91 t grid_simple |
---|
49 | 2 ecmf 20000108 an regular_ll 0 surface 1 sp grid_simple |
---|
50 | 2 ecmf 20000108 an regular_ll 0 hybrid 91 q grid_simple |
---|
51 | 2 ecmf 20000108 an regular_ll 0 hybrid 91 qc grid_simple |
---|
52 | 1 ecmf 20000108 fc regular_ll 0 surface 0 sshf grid_simple |
---|
53 | 1 ecmf 20000108 fc regular_ll 0 surface 0 ewss grid_simple |
---|
54 | 1 ecmf 20000108 fc regular_ll 0 surface 0 nsss grid_simple |
---|
55 | 1 ecmf 20000108 fc regular_ll 0 surface 0 ssr grid_simple |
---|
56 | 1 ecmf 20000108 fc regular_ll 0 surface 0 lsp grid_simple |
---|
57 | 1 ecmf 20000108 fc regular_ll 0 surface 0 cp grid_simple |
---|
58 | 1 ecmf 20000108 an regular_ll 0 surface 0 sd grid_simple |
---|
59 | 1 ecmf 20000108 an regular_ll 0 surface 0 msl grid_simple |
---|
60 | 1 ecmf 20000108 an regular_ll 0 surface 0 tcc grid_simple |
---|
61 | 1 ecmf 20000108 an regular_ll 0 surface 0 10u grid_simple |
---|
62 | 1 ecmf 20000108 an regular_ll 0 surface 0 10v grid_simple |
---|
63 | 1 ecmf 20000108 an regular_ll 0 surface 0 2t grid_simple |
---|
64 | 1 ecmf 20000108 an regular_ll 0 surface 0 2d grid_simple |
---|
65 | 1 ecmf 20000108 an regular_ll 0 surface 0 z grid_simple |
---|
66 | 1 ecmf 20000108 an regular_ll 0 surface 0 lsm grid_simple |
---|
67 | 1 ecmf 20000108 an regular_ll 0 surface 0 cvl grid_simple |
---|
68 | 1 ecmf 20000108 an regular_ll 0 surface 0 cvh grid_simple |
---|
69 | 1 ecmf 20000108 an regular_ll 0 surface 0 lcc grid_simple |
---|
70 | 1 ecmf 20000108 an regular_ll 0 surface 0 mcc grid_simple |
---|
71 | 1 ecmf 20000108 an regular_ll 0 surface 0 hcc grid_simple |
---|
72 | 1 ecmf 20000108 an regular_ll 0 surface 0 skt grid_simple |
---|
73 | 1 ecmf 20000108 an regular_ll 0 depthBelowLandLayer 0 stl1 grid_simple |
---|
74 | 1 ecmf 20000108 an regular_ll 0 depthBelowLandLayer 0 swvl1 grid_simple |
---|
75 | 1 ecmf 20000108 an regular_ll 0 surface 0 sr grid_simple |
---|
76 | 1 ecmf 20000108 an regular_ll 0 surface 0 sdor grid_simple |
---|
77 | 1 ecmf 20000108 an regular_ll 0 surface 0 cvl grid_simple |
---|
78 | 1 ecmf 20000108 an regular_ll 0 surface 0 cvh grid_simple |
---|
79 | 1 ecmf 20000108 an regular_ll 0 surface 0 fsr grid_simple |
---|
80 | 35 of 35 messages in CE00010800 |
---|
81 | |
---|
82 | |
---|
83 | Output files for long forecast |
---|
84 | ------------------------------ |
---|
85 | |
---|
86 | ``Flex_extract`` is able to retrieve forecasts with a lead time of more than 23 hours. In order to avoid collisions of time steps names, a new scheme for filenames in long forecast mode is introduced: |
---|
87 | |
---|
88 | .. code-block:: bash |
---|
89 | |
---|
90 | <prefix>YYMMDD.HH.<FORECAST_STEP> |
---|
91 | |
---|
92 | The ``<prefix>`` is, as in the standard output, by default ``EN`` and can be re-defined in the ``CONTROL`` file. ``YYMMDD`` is the date format and ``HH`` the forecast time which is the starting time for the forecasts. The ``FORECAST_STEP`` is a 3-digit number which represents the forecast step in hours. |
---|
93 | |
---|
94 | |
---|
95 | Output files for ensemble predictions |
---|
96 | ------------------------------------- |
---|
97 | |
---|
98 | ``Flex_extract`` is able to retrieve ensembles data; they are labelled by the grib message parameter ``number``. Each ensemble member is saved in a separate file, and standard filenames are supplemented by the letter ``N`` and the ensemble member number in a 3-digit format. |
---|
99 | |
---|
100 | .. code-block:: bash |
---|
101 | |
---|
102 | <prefix>YYMMDDHH.N<ENSEMBLE_MEMBER> |
---|
103 | |
---|
104 | |
---|
105 | Additional fields with new precipitation disaggregation |
---|
106 | ------------------------------------------------------- |
---|
107 | |
---|
108 | The new disaggregation method for precipitation fields produces two additional precipitation fields for each time step and precipitation type (large-scale and convective). They serve as sub-grid points in the original time interval. For details of the method see :doc:`disagg`. |
---|
109 | The two additional fields are addressed using the ``step`` parameter in the GRIB messages, which |
---|
110 | is set to "1" or "2", for sub-grid points 1 and 2, respectively. |
---|
111 | The output file names are not altered. |
---|
112 | An example of the list of precipitation fields in an output file generated with the new disaggregation method is found below: |
---|
113 | |
---|
114 | .. code-block:: bash |
---|
115 | |
---|
116 | $ grib_ls |
---|
117 | |
---|
118 | edition centre date dataType gridType stepRange typeOfLevel level shortName packingType |
---|
119 | 1 ecmf 20000108 fc regular_ll 0 surface 0 lsp grid_simple |
---|
120 | 1 ecmf 20000108 fc regular_ll 1 surface 0 lsp grid_simple |
---|
121 | 1 ecmf 20000108 fc regular_ll 2 surface 0 lsp grid_simple |
---|
122 | 1 ecmf 20000108 fc regular_ll 0 surface 0 cp grid_simple |
---|
123 | 1 ecmf 20000108 fc regular_ll 1 surface 0 cp grid_simple |
---|
124 | 1 ecmf 20000108 fc regular_ll 2 surface 0 cp grid_simple |
---|
125 | |
---|
126 | |
---|
127 | |
---|
128 | |
---|
129 | Temporary files |
---|
130 | =============== |
---|
131 | |
---|
132 | ``Flex_extract`` creates a number of temporary data files which are usually deleted at the end of a successful run. They are preserved only if the ``DEBUG`` mode is switched on (see :doc:`Input/control_params`). |
---|
133 | |
---|
134 | MARS grib files |
---|
135 | --------------- |
---|
136 | |
---|
137 | ``Flex_extract`` retrieves all meteorological fields from MARS and stores them in files ending with ``.grb``. |
---|
138 | Since there are limits implemented by ECMWF for the time per request and data transfer from MARS, |
---|
139 | and as ECMWF asks for efficient MARS retrievals, ``flex_extract`` splits the overall data request |
---|
140 | into several smaller requests. Each request is stored in its own ``.grb`` file, and the file |
---|
141 | names are composed of several pieces of information: |
---|
142 | |
---|
143 | .. code-block:: bash |
---|
144 | |
---|
145 | <field_type><grid_type><temporal_property><level_type>.<date>.<ppid>.<pid>.grb |
---|
146 | |
---|
147 | Description: |
---|
148 | |
---|
149 | Field type: |
---|
150 | ``AN`` - Analysis, ``FC`` - Forecast, ``4V`` - 4D variational analysis, ``CV`` - Validation forecast, ``CF`` - Control forecast, ``PF`` - Perturbed forecast |
---|
151 | Grid type: |
---|
152 | ``SH`` - Spherical Harmonics, ``GG`` - Gaussian Grid, ``OG`` - Output Grid (typically lat / lon), ``_OROLSM`` - Orography parameter |
---|
153 | Temporal property: |
---|
154 | ``__`` - instantaneous fields, ``_acc`` - accumulated fields |
---|
155 | Level type: |
---|
156 | ``ML`` - model level, ``SL`` - surface level |
---|
157 | ppid: |
---|
158 | The process number of the parent process of the script submitted. |
---|
159 | pid: |
---|
160 | The process number of the script submitted. |
---|
161 | |
---|
162 | |
---|
163 | Example ``.grb`` files for one day of CERA-20C data: |
---|
164 | |
---|
165 | .. code-block:: bash |
---|
166 | |
---|
167 | ANOG__ML.20000908.71851.71852.grb |
---|
168 | FCOG_acc_SL.20000907.71851.71852.grb |
---|
169 | ANOG__SL.20000908.71851.71852.grb |
---|
170 | OG_OROLSM__SL.20000908.71851.71852.grb |
---|
171 | ANSH__SL.20000908.71851.71852.grb |
---|
172 | |
---|
173 | |
---|
174 | MARS request file |
---|
175 | ----------------- |
---|
176 | |
---|
177 | This file is a ``csv`` file called ``mars_requests.csv`` listing the actual settings of the MARS |
---|
178 | request (one request per line) in a flex_extract job. |
---|
179 | It is used for documenting which data were retrieved, and for testing. |
---|
180 | |
---|
181 | Each request consists of the following parameters, whose meaning mostly can be taken from :doc:`Input/control_params` or :doc:`Input/run`: |
---|
182 | request_number, accuracy, area, dataset, date, expver, gaussian, grid, levelist, levtype, marsclass, number, param, repres, resol, step, stream, target, time, type |
---|
183 | |
---|
184 | Example output of a one-day retrieval of CERA-20C data: |
---|
185 | |
---|
186 | .. code-block:: bash |
---|
187 | |
---|
188 | request_number, accuracy, area, dataset, date, expver, gaussian, grid, levelist, levtype, marsclass, number, param, repres, resol, step, stream, target, time, type |
---|
189 | 1, 24, 40.0/-5.0/30.0/5.0, None, 20000107/to/20000109, 1, , 1.0/1.0, 1, SFC, EP, 000, 142.128/143.128/146.128/180.128/181.128/176.128, , 159, 3/to/24/by/3, ENDA, /mnt/nas/Anne/Interpolation/flexextract/flex_extract_v7.1/run/./workspace/CERA_testgrid_local_cds/FCOG_acc_SL.20000107.23903.23904.grb, 18, FC |
---|
190 | 1, 24, 40.0/-5.0/30.0/5.0, None, 20000108/to/20000108, 1, , 1.0/1.0, 85/to/91, ML, EP, 000, 130.128/133.128/131.128/132.128/077.128/246.128/247.128, , 159, 00, ENDA, /mnt/nas/Anne/Interpolation/flexextract/flex_extract_v7.1/run/./workspace/CERA_testgrid_local_cds/ANOG__ML.20000108.23903.23904.grb, 00/03/06/09/12/15/18/21, AN |
---|
191 | 2, 24, 40.0/-5.0/30.0/5.0, None, 20000108/to/20000108, 1, , OFF, 1, ML, EP, 000, 152.128, , 159, 00, ENDA, /mnt/nas/Anne/Interpolation/flexextract/flex_extract_v7.1/run/./workspace/CERA_testgrid_local_cds/ANSH__SL.20000108.23903.23904.grb, 00/03/06/09/12/15/18/21, AN |
---|
192 | 3, 24, 40.0/-5.0/30.0/5.0, None, 20000108, 1, , 1.0/1.0, 1, SFC, EP, 000, 160.128/027.128/028.128/244.128, , 159, 000, ENDA, /mnt/nas/Anne/Interpolation/flexextract/flex_extract_v7.1/run/./workspace/CERA_testgrid_local_cds/OG_OROLSM__SL.20000108.23903.23904.grb, 00, AN |
---|
193 | 4, 24, 40.0/-5.0/30.0/5.0, None, 20000108/to/20000108, 1, , 1.0/1.0, 1, SFC, EP, 000, 141.128/151.128/164.128/165.128/166.128/167.128/168.128/129.128/172.128/027.128/028.128/186.128/187.128/188.128/235.128/139.128/039.128/173.128, , 159, 00, ENDA, /mnt/nas/Anne/Interpolation/flexextract/flex_extract_v7.1/run/./workspace/CERA_testgrid_local_cds/ANOG__SL.20000108.23903.23904.grb, 00/03/06/09/12/15/18/21, AN |
---|
194 | |
---|
195 | |
---|
196 | |
---|
197 | Index file |
---|
198 | ---------- |
---|
199 | |
---|
200 | This file is called ``date_time_stepRange.idx``. It contains indices pointing to specific grib messages from one or more grib files. The messages are selected with a composition of grib message keywords. |
---|
201 | |
---|
202 | |
---|
203 | Flux files |
---|
204 | ---------- |
---|
205 | |
---|
206 | The flux files contain the de-accumulated and dis-aggregated flux fields of large-scale and convective precipitation, east- and northward turbulent surface stresses, the surface sensible heat flux, and the surface net solar radiation. |
---|
207 | |
---|
208 | .. code-block:: bash |
---|
209 | |
---|
210 | flux<date>[.N<xxx>][.<xxx>] |
---|
211 | |
---|
212 | The date format is YYYYMMDDHH as explained before. The optional block ``[.N<xxx>]`` is used for the ensemble forecast date, where ``<xxx>`` is the ensemble member number. The optional block ``[.<xxx>]`` marks a pure forecast with ``<xxx>`` being the forecast step. |
---|
213 | |
---|
214 | .. note:: |
---|
215 | |
---|
216 | In the case of the new dis-aggregation method for precipitation, two new sub-intervals are added in between each time interval. They are identified by the forecast step parameter which is ``0`` for the original time interval, and ``1`` or ``2``, respectively, for the two new intervals. |
---|
217 | |
---|
218 | |
---|
219 | fort files |
---|
220 | ---------- |
---|
221 | |
---|
222 | There are a number of input files for the ``calc_etadot`` Fortran program named |
---|
223 | |
---|
224 | .. code-block:: bash |
---|
225 | |
---|
226 | fort.xx |
---|
227 | |
---|
228 | where ``xx`` is a number which defines the meteorological fields stored in these files. |
---|
229 | They are generated by the Python code in ``flex_extract`` by splitting the meteorological fields for a unique time stamp from the ``*.grb`` files, storing them under the names ``fort.<XX>`` where <XX> represents some number. |
---|
230 | The following table defines the numbers and the corresponding content: |
---|
231 | |
---|
232 | .. csv-table:: Content of fort - files |
---|
233 | :header: "Number", "Content" |
---|
234 | :widths: 5, 20 |
---|
235 | |
---|
236 | "10", "U and V wind components" |
---|
237 | "11", "temperature" |
---|
238 | "12", "logarithm of surface pressure" |
---|
239 | "13", "divergence (optional)" |
---|
240 | "16", "surface fields" |
---|
241 | "17", "specific humidity" |
---|
242 | "18", "surface specific humidity (reduced Gaussian grid)" |
---|
243 | "19", "omega (vertical velocity in pressure coordinates) (optional)" |
---|
244 | "21", "eta-coordinate vertical velocity (optional)" |
---|
245 | "22", "total cloud-water content (optional)" |
---|
246 | |
---|
247 | Some of the fields are solely retrieved with specific settings, e. g., the eta-coordinate vertical velocity is not available in ERA-Interim datasets, and the total cloud-water content is an optional field which is useful for ``FLEXPART v10`` and newer. |
---|
248 | |
---|
249 | The ``calc_etadot`` program saves its results in file ``fort.15`` which typically contains: |
---|
250 | |
---|
251 | .. csv-table:: Output file of the Fortran program ``calc_etadot`` |
---|
252 | :header: "Number", "Content" |
---|
253 | :widths: 5, 20 |
---|
254 | |
---|
255 | "15", "U and V wind components, eta-coordinate vertical velocity, temperature, surface pressure, specific humidity " |
---|
256 | |
---|
257 | More details about the content of ``calc_etadot`` can be found in :doc:`vertco`. |
---|
258 | |
---|
259 | .. note:: |
---|
260 | |
---|
261 | The ``fort.4`` file is the namelist file to control the Fortran program ``calc_etadot``. It is therefore also an input file. |
---|
262 | |
---|
263 | Example of a namelist: |
---|
264 | |
---|
265 | .. code-block:: bash |
---|
266 | |
---|
267 | &NAMGEN |
---|
268 | maxl = 11, |
---|
269 | maxb = 11, |
---|
270 | mlevel = 91, |
---|
271 | mlevelist = "85/to/91", |
---|
272 | mnauf = 159, |
---|
273 | metapar = 77, |
---|
274 | rlo0 = -5.0, |
---|
275 | rlo1 = 5.0, |
---|
276 | rla0 = 30.0, |
---|
277 | rla1 = 40.0, |
---|
278 | momega = 0, |
---|
279 | momegadiff = 0, |
---|
280 | mgauss = 0, |
---|
281 | msmooth = 0, |
---|
282 | meta = 1, |
---|
283 | metadiff = 0, |
---|
284 | mdpdeta = 1 |
---|
285 | / |
---|
286 | |
---|
287 | |
---|
288 | .. toctree:: |
---|
289 | :hidden: |
---|
290 | :maxdepth: 2 |
---|
291 | |
---|