Changeset 274f9ef in flex_extract.git for source/python/classes/MarsRetrieval.py
- Timestamp:
- Oct 22, 2018, 11:44:47 AM (5 years ago)
- Branches:
- master, ctbto, dev
- Children:
- db27c09
- Parents:
- 708c667
- File:
-
- 1 edited
Legend:
- Unmodified
- Added
- Removed
-
source/python/classes/MarsRetrieval.py
r70fee58 r274f9ef 73 73 # ------------------------------------------------------------------------------ 74 74 class MarsRetrieval(object): 75 ''' 76 Class for submitting MARS retrievals. 75 '''Class for submitting MARS retrievals. 77 76 78 77 A description of MARS keywords/arguments and examples of their … … 80 79 https://software.ecmwf.int/wiki/display/UDOC/\ 81 80 Identification+keywords#Identificationkeywords-class 82 83 81 ''' 84 82 … … 88 86 number="", accuracy="", grid="", gaussian="", target="", 89 87 param=""): 90 ''' 91 @Description: 92 Initialises the instance of the MarsRetrieval class and 93 defines and assigns a set of the necessary retrieval parameters 94 for the FLEXPART input data. 95 A description of MARS keywords/arguments, their dependencies 96 on each other and examples of their values can be found here: 97 98 https://software.ecmwf.int/wiki/display/UDOC/MARS+keywords 99 100 @Input: 101 self: instance of MarsRetrieval 102 For description see class documentation. 103 104 server: instance of ECMWFService (from ECMWF Web-API) 105 This is the connection to the ECMWF data servers. 106 It is needed for the pythonic access of ECMWF data. 107 108 public: integer 109 Decides which Web API version is used: 110 0: member-state users and full archive access 111 1: public access and limited access to the public server and 112 datasets. Needs the parameter dataset. 113 Default is "0" and for member-state users. 114 115 marsclass: string, optional 116 Characterisation of dataset. E.g. EI (ERA-Interim), 117 E4 (ERA40), OD (Operational archive), ea (ERA5). 118 Default is the ERA-Interim dataset "ei". 119 120 dataset: string, optional 121 For public datasets there is the specific naming and parameter 122 dataset which has to be used to characterize the type of 123 data. Usually there is less data available, either in times, 124 domain or parameter. 125 Default is an empty string. 126 127 type: string, optional 128 Determines the type of fields to be retrieved. 129 Selects between observations, images or fields. 130 Examples for fields: Analysis (an), Forecast (fc), 131 Perturbed Forecast (pf), Control Forecast (cf) and so on. 132 Default is an empty string. 133 134 levtype: string, optional 135 Denotes type of level. Has a direct implication on valid 136 levelist values! 137 E.g. model level (ml), pressure level (pl), surface (sfc), 138 potential vorticity (pv), potential temperature (pt) 139 and depth (dp). 140 Default is an empty string. 141 142 levelist: string, optional 143 Specifies the required levels. It has to have a valid 144 correspondence to the selected levtype. 145 Examples: model level: 1/to/137, pressure levels: 500/to/1000 146 Default is an empty string. 147 148 repres: string, optional 149 Selects the representation of the archived data. 150 E.g. sh - spherical harmonics, gg - Gaussian grid, 151 ll - latitude/longitude, ... 152 Default is an empty string. 153 154 date: string, optional 155 Specifies the Analysis date, the Forecast base date or 156 Observations date. Valid formats are: 157 Absolute as YYYY-MM-DD or YYYYMMDD. 158 Default is an empty string. 159 160 resol: string, optional 161 Specifies the desired triangular truncation of retrieved data, 162 before carrying out any other selected post-processing. 163 The default is automatic truncation (auto), by which the lowest 164 resolution compatible with the value specified in grid is 165 automatically selected for the retrieval. 166 Users wanting to perform post-processing from full spectral 167 resolution should specify Archived Value (av). 168 The following are examples of existing resolutions found in 169 the archive: 63, 106, 159, 213, 255, 319, 399, 511, 799 or 1279. 170 This keyword has no meaning/effect if the archived data is 171 not in spherical harmonics representation. 172 The best selection can be found here: 173 https://software.ecmwf.int/wiki/display/UDOC/\ 174 Retrieve#Retrieve-Truncationbeforeinterpolation 175 Default is an empty string. 176 177 stream: string, optional 178 Identifies the forecasting system used to generate the data. 179 E.g. oper (Atmospheric model), enfo (Ensemble forecats), ... 180 Default is an empty string. 181 182 area: string, optional 183 Specifies the desired sub-area of data to be extracted. 184 Areas can be defined to wrap around the globe. 185 186 Latitude values must be given as signed numbers, with: 187 north latitudes (i.e. north of the equator) 188 being positive (e.g: 40.5) 189 south latitutes (i.e. south of the equator) 190 being negative (e.g: -50.5) 191 Longtitude values must be given as signed numbers, with: 192 east longitudes (i.e. east of the 0 degree meridian) 193 being positive (e.g: 35.0) 194 west longitudes (i.e. west of the 0 degree meridian) 195 being negative (e.g: -20.5) 196 197 E.g.: North/West/South/East 198 Default is an empty string. 199 200 time: string, optional 201 Specifies the time of the data in hours and minutes. 202 Valid values depend on the type of data: Analysis time, 203 Forecast base time or First guess verification time 204 (all usually at synoptic hours: 00, 06, 12 and 18 ). 205 Observation time (any combination in hours and minutes is valid, 206 subject to data availability in the archive). 207 The syntax is HHMM or HH:MM. If MM is omitted it defaults to 00. 208 Default is an empty string. 209 210 step: string, optional 211 Specifies the forecast time step from forecast base time. 212 Valid values are hours (HH) from forecast base time. It also 213 specifies the length of the forecast which verifies at 214 First Guess time. 215 E.g. 1/3/6-hourly 216 Default is an empty string. 217 218 expver: string, optional 219 The version of the dataset. Each experiment is assigned a 220 unique code (version). Production data is assigned 1 or 2, 221 and experimental data in Operations 11, 12 ,... 222 Research or Member State's experiments have a four letter 223 experiment identifier. 224 Default is "1". 225 226 number: string, optional 227 Selects the member in ensemble forecast run. (Only then it 228 is necessary.) It has a different meaning depending on 229 the type of data. 230 E.g. Perturbed Forecasts: specifies the Ensemble forecast member 231 Default is an empty string. 232 233 accuracy: string, optional 234 Specifies the number of bits per value to be used in the 235 generated GRIB coded fields. 236 A positive integer may be given to specify the preferred number 237 of bits per packed value. This must not be greater than the 238 number of bits normally used for a Fortran integer on the 239 processor handling the request (typically 32 or 64 bit). 240 Within a compute request the accuracy of the original fields 241 can be passed to the result field by specifying accuracy=av. 242 Default is an empty string. 243 244 grid: string, optional 245 Specifies the output grid which can be either a Gaussian grid 246 or a Latitude/Longitude grid. MARS requests specifying 247 grid=av will return the archived model grid. 248 249 Lat/Lon grid: The grid spacing needs to be an integer 250 fraction of 90 degrees e.g. grid = 0.5/0.5 251 252 Gaussian grid: specified by a letter denoting the type of 253 Gaussian grid followed by an integer (the grid number) 254 representing the number of lines between the Pole and Equator, 255 e.g. 256 grid = F160 - full (or regular) Gaussian grid with 257 160 latitude lines between the pole and equator 258 grid = N320 - ECMWF original reduced Gaussian grid with 259 320 latitude lines between the pole and equator, 260 see Reduced Gaussian Grids for grid numbers used at ECMWF 261 grid = O640 - ECMWF octahedral (reduced) Gaussian grid with 262 640 latitude lines between the pole and equator 263 Default is an empty string. 264 265 gaussian: string, optional 266 This parameter is deprecated and should no longer be used. 267 Specifies the desired type of Gaussian grid for the output. 268 Valid Gaussian grids are quasi-regular (reduced) or regular. 269 Keyword gaussian can only be specified together with 270 keyword grid. Gaussian without grid has no effect. 271 Default is an empty string. 272 273 target: string, optional 274 Specifies a file into which data is to be written after 275 retrieval or manipulation. Path names should always be 276 enclosed in double quotes. The MARS client supports automatic 277 generation of multiple target files using MARS keywords 278 enclosed in square brackets [ ]. If the environment variable 279 MARS_MULTITARGET_STRICT_FORMAT is set to 1 before calling mars, 280 the keyword values will be used in the filename as shown by 281 the ecCodes GRIB tool grib_ls -m, e.g. with 282 MARS_MULTITARGET_STRICT_FORMAT set to 1 the keywords time, 283 expver and param will be formatted as 0600, 0001 and 129.128 284 rather than 600, 1 and 129. 285 Default is an empty string. 286 287 param: string, optional 288 Specifies the meteorological parameter. 289 The list of meteorological parameters in MARS is extensive. 290 Their availability is directly related to their meteorological 291 meaning and, therefore, the rest of directives specified 292 in the MARS request. 293 Meteorological parameters can be specified by their 294 GRIB code (param=130), their mnemonic (param=t) or 295 full name (param=temperature). 296 The list of parameter should be seperated by a "/"-sign. 297 E.g. 130/131/133 298 Default is an empty string. 299 300 @Return: 301 <nothing> 88 '''Initialises the instance of the MarsRetrieval class and 89 defines and assigns a set of the necessary retrieval parameters 90 for the FLEXPART input data. 91 A description of MARS keywords/arguments, their dependencies 92 on each other and examples of their values can be found here: 93 94 https://software.ecmwf.int/wiki/display/UDOC/MARS+keywords 95 96 Parameters 97 ---------- 98 server : :obj:`ECMWFService` 99 This is the connection to the ECMWF data servers. 100 It is needed for the pythonic access of ECMWF data. 101 102 public : :obj:`integer` 103 Decides which Web API version is used: 104 0: member-state users and full archive access 105 1: public access and limited access to the public server and 106 datasets. Needs the parameter dataset. 107 Default is "0" and for member-state users. 108 109 marsclass : :obj:`string`, optional 110 Characterisation of dataset. E.g. EI (ERA-Interim), 111 E4 (ERA40), OD (Operational archive), ea (ERA5). 112 Default is the ERA-Interim dataset "ei". 113 114 dataset : :obj:`string`, optional 115 For public datasets there is the specific naming and parameter 116 dataset which has to be used to characterize the type of 117 data. Usually there is less data available, either in times, 118 domain or parameter. 119 Default is an empty string. 120 121 type : :obj:`string`, optional 122 Determines the type of fields to be retrieved. 123 Selects between observations, images or fields. 124 Examples for fields: Analysis (an), Forecast (fc), 125 Perturbed Forecast (pf), Control Forecast (cf) and so on. 126 Default is an empty string. 127 128 levtype : :obj:`string`, optional 129 Denotes type of level. Has a direct implication on valid 130 levelist values! 131 E.g. model level (ml), pressure level (pl), surface (sfc), 132 potential vorticity (pv), potential temperature (pt) 133 and depth (dp). 134 Default is an empty string. 135 136 levelist : :obj:`string`, optional 137 Specifies the required levels. It has to have a valid 138 correspondence to the selected levtype. 139 Examples: model level: 1/to/137, pressure levels: 500/to/1000 140 Default is an empty string. 141 142 repres : :obj:`string`, optional 143 Selects the representation of the archived data. 144 E.g. sh - spherical harmonics, gg - Gaussian grid, 145 ll - latitude/longitude, ... 146 Default is an empty string. 147 148 date : :obj:`string`, optional 149 Specifies the Analysis date, the Forecast base date or 150 Observations date. Valid formats are: 151 Absolute as YYYY-MM-DD or YYYYMMDD. 152 Default is an empty string. 153 154 resol : :obj:`string`, optional 155 Specifies the desired triangular truncation of retrieved data, 156 before carrying out any other selected post-processing. 157 The default is automatic truncation (auto), by which the lowest 158 resolution compatible with the value specified in grid is 159 automatically selected for the retrieval. 160 Users wanting to perform post-processing from full spectral 161 resolution should specify Archived Value (av). 162 The following are examples of existing resolutions found in 163 the archive: 63, 106, 159, 213, 255, 319, 399, 511, 799 or 1279. 164 This keyword has no meaning/effect if the archived data is 165 not in spherical harmonics representation. 166 The best selection can be found here: 167 https://software.ecmwf.int/wiki/display/UDOC/\ 168 Retrieve#Retrieve-Truncationbeforeinterpolation 169 Default is an empty string. 170 171 stream : :obj:`string`, optional 172 Identifies the forecasting system used to generate the data. 173 E.g. oper (Atmospheric model), enfo (Ensemble forecats), ... 174 Default is an empty string. 175 176 area : :obj:`string`, optional 177 Specifies the desired sub-area of data to be extracted. 178 Areas can be defined to wrap around the globe. 179 180 Latitude values must be given as signed numbers, with: 181 north latitudes (i.e. north of the equator) 182 being positive (e.g: 40.5) 183 south latitutes (i.e. south of the equator) 184 being negative (e.g: -50.5) 185 Longtitude values must be given as signed numbers, with: 186 east longitudes (i.e. east of the 0 degree meridian) 187 being positive (e.g: 35.0) 188 west longitudes (i.e. west of the 0 degree meridian) 189 being negative (e.g: -20.5) 190 191 E.g.: North/West/South/East 192 Default is an empty string. 193 194 time : :obj:`string`, optional 195 Specifies the time of the data in hours and minutes. 196 Valid values depend on the type of data: Analysis time, 197 Forecast base time or First guess verification time 198 (all usually at synoptic hours: 00, 06, 12 and 18 ). 199 Observation time (any combination in hours and minutes is valid, 200 subject to data availability in the archive). 201 The syntax is HHMM or HH:MM. If MM is omitted it defaults to 00. 202 Default is an empty string. 203 204 step : :obj:`string`, optional 205 Specifies the forecast time step from forecast base time. 206 Valid values are hours (HH) from forecast base time. It also 207 specifies the length of the forecast which verifies at 208 First Guess time. 209 E.g. 1/3/6-hourly 210 Default is an empty string. 211 212 expver : :obj:`string`, optional 213 The version of the dataset. Each experiment is assigned a 214 unique code (version). Production data is assigned 1 or 2, 215 and experimental data in Operations 11, 12 ,... 216 Research or Member State's experiments have a four letter 217 experiment identifier. 218 Default is "1". 219 220 number : :obj:`string`, optional 221 Selects the member in ensemble forecast run. (Only then it 222 is necessary.) It has a different meaning depending on 223 the type of data. 224 E.g. Perturbed Forecasts: specifies the Ensemble forecast member 225 Default is an empty string. 226 227 accuracy : :obj:`string`, optional 228 Specifies the number of bits per value to be used in the 229 generated GRIB coded fields. 230 A positive integer may be given to specify the preferred number 231 of bits per packed value. This must not be greater than the 232 number of bits normally used for a Fortran integer on the 233 processor handling the request (typically 32 or 64 bit). 234 Within a compute request the accuracy of the original fields 235 can be passed to the result field by specifying accuracy=av. 236 Default is an empty string. 237 238 grid : :obj:`string`, optional 239 Specifies the output grid which can be either a Gaussian grid 240 or a Latitude/Longitude grid. MARS requests specifying 241 grid=av will return the archived model grid. 242 243 Lat/Lon grid: The grid spacing needs to be an integer 244 fraction of 90 degrees e.g. grid = 0.5/0.5 245 246 Gaussian grid: specified by a letter denoting the type of 247 Gaussian grid followed by an integer (the grid number) 248 representing the number of lines between the Pole and Equator, 249 e.g. 250 grid = F160 - full (or regular) Gaussian grid with 251 160 latitude lines between the pole and equator 252 grid = N320 - ECMWF original reduced Gaussian grid with 253 320 latitude lines between the pole and equator, 254 see Reduced Gaussian Grids for grid numbers used at ECMWF 255 grid = O640 - ECMWF octahedral (reduced) Gaussian grid with 256 640 latitude lines between the pole and equator 257 Default is an empty string. 258 259 gaussian : :obj:`string`, optional 260 This parameter is deprecated and should no longer be used. 261 Specifies the desired type of Gaussian grid for the output. 262 Valid Gaussian grids are quasi-regular (reduced) or regular. 263 Keyword gaussian can only be specified together with 264 keyword grid. Gaussian without grid has no effect. 265 Default is an empty string. 266 267 target : :obj:`string`, optional 268 Specifies a file into which data is to be written after 269 retrieval or manipulation. Path names should always be 270 enclosed in double quotes. The MARS client supports automatic 271 generation of multiple target files using MARS keywords 272 enclosed in square brackets [ ]. If the environment variable 273 MARS_MULTITARGET_STRICT_FORMAT is set to 1 before calling mars, 274 the keyword values will be used in the filename as shown by 275 the ecCodes GRIB tool grib_ls -m, e.g. with 276 MARS_MULTITARGET_STRICT_FORMAT set to 1 the keywords time, 277 expver and param will be formatted as 0600, 0001 and 129.128 278 rather than 600, 1 and 129. 279 Default is an empty string. 280 281 param : :obj:`string`, optional 282 Specifies the meteorological parameter. 283 The list of meteorological parameters in MARS is extensive. 284 Their availability is directly related to their meteorological 285 meaning and, therefore, the rest of directives specified 286 in the MARS request. 287 Meteorological parameters can be specified by their 288 GRIB code (param=130), their mnemonic (param=t) or 289 full name (param=temperature). 290 The list of parameter should be seperated by a "/"-sign. 291 E.g. 130/131/133 292 Default is an empty string. 293 294 Return 295 ------ 296 302 297 ''' 303 298 … … 328 323 329 324 def display_info(self): 330 ''' 331 @Description: 332 Prints all class attributes and their values to the 333 standard output. 334 335 @Input: 336 self: instance of MarsRetrieval 337 For description see class documentation. 338 339 @Return: 340 <nothing> 325 '''Prints all class attributes and their values to the 326 standard output. 327 328 Parameters 329 ---------- 330 331 Return 332 ------ 333 341 334 ''' 342 335 # Get all class attributes and their values as a dictionary … … 355 348 356 349 def print_info(self, inputdir, request_number): 357 ''' 358 @Description: 359 Prints all mars requests to an extra file for debugging and 360 information. 361 362 @Input: 363 self: instance of MarsRetrieval 364 For description see class documentation. 365 366 inputdir: string 367 The path where all data from the retrievals are stored. 368 369 request_number: integer 370 Number of mars requests for flux and non-flux data. 371 372 @Return: 373 <nothing> 350 '''Prints all mars requests to an extra file for debugging and 351 information. 352 353 Parameters 354 ---------- 355 inputdir : :obj:`string` 356 The path where all data from the retrievals are stored. 357 358 request_number : :obj:`integer` 359 Number of mars requests for flux and non-flux data. 360 361 Return 362 ------ 363 374 364 ''' 375 365 # Get all class attributes and their values as a dictionary … … 393 383 394 384 def print_infodata_csv(self, inputdir, request_number): 395 ''' 396 @Description: 397 Write all request parameter in alpabetical order into a "csv" file. 398 399 @Input: 400 self: instance of MarsRetrieval 401 For description see class documentation. 402 403 inputdir: string 404 The path where all data from the retrievals are stored. 405 406 request_number: integer 407 Number of mars requests for flux and non-flux data. 408 409 @Return: 410 <nothing> 385 '''Write all request parameter in alpabetical order into a "csv" file. 386 387 Parameters 388 ---------- 389 inputdir : :obj:`string` 390 The path where all data from the retrievals are stored. 391 392 request_number : :obj:`integer` 393 Number of mars requests for flux and non-flux data. 394 395 Return 396 ------ 397 411 398 ''' 412 399 … … 427 414 428 415 def data_retrieve(self): 429 ''' 430 @Description: 431 Submits a MARS retrieval. Depending on the existence of 432 ECMWF Web-API it is submitted via Python or a 433 subprocess in the Shell. The parameter for the mars retrieval 434 are taken from the defined class attributes. 435 436 @Input: 437 self: instance of MarsRetrieval 438 For description see class documentation. 439 440 @Return: 441 <nothing> 416 '''Submits a MARS retrieval. Depending on the existence of 417 ECMWF Web-API it is submitted via Python or a 418 subprocess in the Shell. The parameter for the mars retrieval 419 are taken from the defined class attributes. 420 421 Parameters 422 ---------- 423 424 Return 425 ------ 426 442 427 ''' 443 428 # Get all class attributes and their values as a dictionary
Note: See TracChangeset
for help on using the changeset viewer.