Changeset 274f9ef in flex_extract.git for source/python/classes
- Timestamp:
- Oct 22, 2018, 11:44:47 AM (6 years ago)
- Branches:
- master, ctbto, dev
- Children:
- db27c09
- Parents:
- 708c667
- Location:
- source/python/classes
- Files:
-
- 5 edited
Legend:
- Unmodified
- Added
- Removed
-
source/python/classes/ControlFile.py
r708c667 r274f9ef 71 71 72 72 def __init__(self, filename): 73 ''' 74 @Description: 75 Initialises the instance of ControlFile class and defines 76 all class attributes with default values. Afterwards calls 77 function __read_controlfile__ to read parameter from 78 Control file. 79 80 @Input: 81 self: instance of ControlFile class 82 Description see class documentation. 83 84 filename: string 85 Name of CONTROL file. 86 87 @Return: 88 <nothing> 73 '''Initialises the instance of ControlFile class and defines 74 all class attributes with default values. Afterwards calls 75 function __read_controlfile__ to read parameter from Control file. 76 77 Parameters 78 ---------- 79 filename : :obj:`string` 80 Name of CONTROL file. 81 82 Return 83 ------ 84 89 85 ''' 90 86 … … 162 158 163 159 def __read_controlfile__(self): 164 ''' 165 @Description: 166 Read CONTROL file and assign all CONTROL file variables. 167 168 @Input: 169 self: instance of ControlFile class 170 Description see class documentation. 171 172 @Return: 173 <nothing> 160 '''Read CONTROL file and assign all CONTROL file variables. 161 162 Parameters 163 ---------- 164 165 Return 166 ------ 167 174 168 ''' 175 169 … … 235 229 236 230 def __str__(self): 237 ''' 238 @Description:239 Prepares a string which have all the ControlFile240 class attributes with its associated values. 241 Each attribute is printed in one line and in242 alphabetical order.243 244 Example:245 'age': 10246 'color': 'Spotted'247 'kids': 0248 'legs': 2249 'name': 'Dog' 250 'smell': 'Alot'251 252 @Input: 253 self: instance of ControlFile class254 Description see class documentation.255 256 @Return:257 string of ControlFile class attributeswith their values231 '''Prepares a string which have all the ControlFile class attributes 232 with its associated values. Each attribute is printed in one line and 233 in alphabetical order. 234 235 Example 236 ------- 237 'age': 10 238 'color': 'Spotted' 239 'kids': 0 240 'legs': 2 241 'name': 'Dog' 242 'smell': 'Alot' 243 244 Parameters 245 ---------- 246 247 Return 248 ------ 249 string 250 Single string of concatenated ControlFile class attributes 251 with their values 258 252 ''' 259 253 import collections … … 265 259 266 260 def assign_args_to_control(self, args): 267 ''' 268 @Description: 269 Overwrites the existing ControlFile instance attributes with 270 the command line arguments. 271 272 @Input: 273 self: instance of ControlFile class 274 Description see class documentation. 275 276 args: instance of ArgumentParser 277 Contains the commandline arguments from script/program call. 278 279 @Return: 280 <nothing> 261 '''Overwrites the existing ControlFile instance attributes with 262 the command line arguments. 263 264 Parameters 265 ---------- 266 args : :obj:`Namespace` 267 Contains the commandline arguments from script/program call. 268 269 Return 270 ------ 271 281 272 ''' 282 273 … … 294 285 295 286 def assign_envs_to_control(self, envs): 296 ''' 297 @Description: 298 Assigns the ECMWF environment parameter.299 300 @Input:301 envs: dict of strings302 Contains the ECMWF environment parameternames "ECUID", "ECGID",303 "DESTINATION" and "GATEWAY" with its corresponding values.304 They were read from the file "ECMWF_ENV". 305 306 @Return:307 <nothing> 287 '''Assigns the ECMWF environment parameter. 288 289 Parameters 290 ---------- 291 envs : :obj:`dictionary` of :obj:`strings` 292 Contains the ECMWF environment parameternames "ECUID", "ECGID", 293 "DESTINATION" and "GATEWAY" with its corresponding values. 294 They were read from the file "ECMWF_ENV". 295 296 Return 297 ------ 298 308 299 ''' 309 300 … … 314 305 315 306 def check_conditions(self, queue): 316 ''' 317 @Description: 318 Checks a couple of necessary attributes and conditions, 319 such as if they exist and contain values. 320 Otherwise set default values. 321 322 @Input: 323 self: instance of ControlFile class 324 Description see class documentation. 325 326 queue: string 327 Name of the queue if submitted to the ECMWF servers. 328 Used to check if ecuid, ecgid, gateway and destination 329 are set correctly and are not empty. 330 331 @Return: 332 <nothing> 307 '''Checks a couple of necessary attributes and conditions, 308 such as if they exist and contain values. 309 Otherwise set default values. 310 311 Parameters 312 ---------- 313 queue : :obj:`string` 314 Name of the queue if submitted to the ECMWF servers. 315 Used to check if ecuid, ecgid, gateway and destination 316 are set correctly and are not empty. 317 318 Return 319 ------ 320 333 321 ''' 334 322 from mods.tools import my_error … … 501 489 502 490 def check_install_conditions(self): 503 ''' 504 @Description: 505 Checks a couple of necessary attributes and conditions 506 for the installation such as if they exist and contain values. 507 Otherwise set default values. 508 509 @Input: 510 self: instance of ControlFile class 511 Description see class documentation. 512 513 @Return: 514 <nothing> 491 '''Checks a couple of necessary attributes and conditions 492 for the installation such as if they exist and contain values. 493 Otherwise set default values. 494 495 Parameters 496 ---------- 497 498 Return 499 ------ 500 515 501 ''' 516 502 … … 549 535 550 536 def to_list(self): 551 ''' 552 @Description: 553 Just generates a list of strings containing the attributes and 554 assigned values except the attributes "_expanded", "exedir", 537 '''Just generates a list of strings containing the attributes and 538 assigned values except the attributes "_expanded", "exedir", 539 "ecmwfdatadir" and "flexpart_root_scripts". 540 541 Parameters 542 ---------- 543 544 Return 545 ------ 546 l : :obj:`list` 547 A sorted list of the all ControlFile class attributes with 548 their values except the attributes "_expanded", "exedir", 555 549 "ecmwfdatadir" and "flexpart_root_scripts". 556 557 @Input:558 self: instance of ControlFile class559 Description see class documentation.560 561 @Return:562 l: list563 A sorted list of the all ControlFile class attributes with564 their values except the attributes "_expanded", "exedir",565 "ecmwfdatadir" and "flexpart_root_scripts".566 550 ''' 567 551 -
source/python/classes/EcFlexpart.py
r0934db1 r274f9ef 111 111 # -------------------------------------------------------------------------- 112 112 def __init__(self, c, fluxes=False): 113 ''' 114 @Description: 115 Creates an object/instance of EcFlexpart with the 116 associated settings of its attributes for the retrieval. 117 118 @Input: 119 self: instance of EcFlexpart 120 The current object of the class. 121 122 c: instance of class ControlFile 123 Contains all the parameters of CONTROL file and 124 command line. 125 For more information about format and content of the parameter 126 see documentation. 127 128 fluxes: boolean, optional 129 Decides if the flux parameter settings are stored or 130 the rest of the parameter list. 131 Default value is False. 132 133 @Return: 134 <nothing> 113 '''Creates an object/instance of EcFlexpart with the associated 114 settings of its attributes for the retrieval. 115 116 Parameters: 117 ----------- 118 c : :obj:`ControlFile` 119 Contains all the parameters of CONTROL file and 120 command line. 121 122 fluxes : :obj:`boolean`, optional 123 Decides if the flux parameter settings are stored or 124 the rest of the parameter list. 125 Default value is False. 126 127 Return 128 ------ 129 135 130 ''' 136 131 # set a counter for the number of mars requests generated … … 306 301 307 302 def _mk_targetname(self, ftype, param, date): 308 ''' 309 @Description:310 Creates the filename for the requested grib data to be stored in. 311 This name is passed as the "target" parameter in the request.312 313 @Input:314 ftype: string315 Shortcut name of the type of the field. E.g. AN, FC, PF, ... 316 317 param: string318 Shortcut of the grid type. E.g. SH__ML, SH__SL, GG__ML,319 GG__SL, OG__ML, OG__SL, OG_OROLSM_SL, OG_acc_SL 320 321 date: string322 The date period of the grib data to be stored in this file. 323 324 @Return:325 targetname: string326 303 '''Creates the filename for the requested grib data to be stored in. 304 This name is passed as the "target" parameter in the request. 305 306 Parameters 307 ---------- 308 ftype : :obj:`string` 309 Shortcut name of the type of the field. E.g. AN, FC, PF, ... 310 311 param : :obj:`string` 312 Shortcut of the grid type. E.g. SH__ML, SH__SL, GG__ML, 313 GG__SL, OG__ML, OG__SL, OG_OROLSM_SL, OG_acc_SL 314 315 date : :obj:`string` 316 The date period of the grib data to be stored in this file. 317 318 Return 319 ------ 320 targetname : :obj:`string` 321 The target filename for the grib data. 327 322 ''' 328 323 targetname = (self.inputdir + '/' + ftype + param + '.' + date + '.' + … … 333 328 334 329 def _start_retrievement(self, request, par_dict): 335 ''' 336 @Description: 337 Creates the Mars Retrieval and prints or submits the request 338 depending on the status of the request variable. 339 340 @Input: 341 self: instance of EcFlexpart 342 The current object of the class. 343 344 request: integer 345 Selects the mode of retrieval. 346 0: Retrieves the data from ECMWF. 347 1: Prints the mars requests to an output file. 348 2: Retrieves the data and prints the mars request. 349 350 par_dict: dictionary 351 Contains all parameter which have to be set for creating the 352 Mars Retrievals. The parameter are: 353 marsclass, dataset, stream, type, levtype, levelist, resol, 354 gaussian, accuracy, grid, target, area, date, time, number, 355 step, expver, param 356 357 @Return: 358 <nothing> 330 '''Creates the Mars Retrieval and prints or submits the request 331 depending on the status of the request variable. 332 333 Parameters 334 ---------- 335 request : :obj:`integer` 336 Selects the mode of retrieval. 337 0: Retrieves the data from ECMWF. 338 1: Prints the mars requests to an output file. 339 2: Retrieves the data and prints the mars request. 340 341 par_dict : :obj:`dictionary` 342 Contains all parameter which have to be set for creating the 343 Mars Retrievals. The parameter are: 344 marsclass, dataset, stream, type, levtype, levelist, resol, 345 gaussian, accuracy, grid, target, area, date, time, number, 346 step, expver, param 347 348 Return 349 ------ 350 359 351 ''' 360 352 # increase number of mars requests … … 398 390 399 391 def _mk_index_values(self, inputdir, inputfiles, keys): 400 ''' 401 @Description:402 Creates an index file for a set of grib parameter keys. 403 The values from the index keys are returned in a list.404 405 @Input:406 keys: dictionary407 List of parameter names which serves as index. 408 409 inputfiles: instance of UioFiles410 Contains a list of files. 411 412 @Return:413 iid: grib_index414 415 416 417 index_vals: list418 419 420 421 422 423 392 '''Creates an index file for a set of grib parameter keys. 393 The values from the index keys are returned in a list. 394 395 Parameters 396 ---------- 397 keys : :obj:`dictionary` 398 List of parameter names which serves as index. 399 400 inputfiles : :obj:`UioFiles` 401 Contains a list of files. 402 403 Return 404 ------ 405 iid : :obj:`grib_index` 406 This is a grib specific index structure to access 407 messages in a file. 408 409 index_vals : :obj:`list` 410 Contains the values from the keys used for a distinct selection 411 of grib messages in processing the grib files. 412 Content looks like e.g.: 413 index_vals[0]: ('20171106', '20171107', '20171108') ; date 414 index_vals[1]: ('0', '1200', '1800', '600') ; time 415 index_vals[2]: ('0', '12', '3', '6', '9') ; stepRange 424 416 ''' 425 417 iid = None … … 455 447 456 448 def retrieve(self, server, dates, public, request, inputdir='.'): 457 ''' 458 @Description: 459 Finalizing the retrieval information by setting final details 460 depending on grid type. 461 Prepares MARS retrievals per grid type and submits them. 462 463 @Input: 464 self: instance of EcFlexpart 465 The current object of the class. 466 467 server: instance of ECMWFService or ECMWFDataServer 468 The connection to the ECMWF server. This is different 469 for member state users which have full access and non 470 member state users which have only access to the public 471 data sets. The decision is made from command line argument 472 "public"; for public access its True (ECMWFDataServer) 473 for member state users its False (ECMWFService) 474 475 dates: string 476 Contains start and end date of the retrieval in the format 477 "YYYYMMDD/to/YYYYMMDD" 478 479 request: integer 480 Selects the mode of retrieval. 481 0: Retrieves the data from ECMWF. 482 1: Prints the mars requests to an output file. 483 2: Retrieves the data and prints the mars request. 484 485 inputdir: string, optional 486 Path to the directory where the retrieved data is about 487 to be stored. The default is the current directory ('.'). 488 489 @Return: 490 <nothing> 449 '''Finalizing the retrieval information by setting final details 450 depending on grid type. 451 Prepares MARS retrievals per grid type and submits them. 452 453 Parameters 454 ---------- 455 server : :obj:`ECMWFService` or :obj:`ECMWFDataServer` 456 The connection to the ECMWF server. This is different 457 for member state users which have full access and non 458 member state users which have only access to the public 459 data sets. The decision is made from command line argument 460 "public"; for public access its True (ECMWFDataServer) 461 for member state users its False (ECMWFService) 462 463 dates : :obj:`string` 464 Contains start and end date of the retrieval in the format 465 "YYYYMMDD/to/YYYYMMDD" 466 467 request : :obj:`integer` 468 Selects the mode of retrieval. 469 0: Retrieves the data from ECMWF. 470 1: Prints the mars requests to an output file. 471 2: Retrieves the data and prints the mars request. 472 473 inputdir : :obj:`string`, optional 474 Path to the directory where the retrieved data is about 475 to be stored. The default is the current directory ('.'). 476 477 Return 478 ------ 479 491 480 ''' 492 481 self.dates = dates … … 665 654 666 655 def write_namelist(self, c): 667 ''' 668 @Description: 669 Creates a namelist file in the temporary directory and writes 670 the following values to it: maxl, maxb, mlevel, 671 mlevelist, mnauf, metapar, rlo0, rlo1, rla0, rla1, 672 momega, momegadiff, mgauss, msmooth, meta, metadiff, mdpdeta 673 674 @Input: 675 self: instance of EcFlexpart 676 The current object of the class. 677 678 c: instance of class ControlFile 679 Contains all the parameters of CONTROL file and 680 command line. 681 For more information about format and content of the parameter 682 see documentation. 683 684 filename: string 656 '''Creates a namelist file in the temporary directory and writes 657 the following values to it: maxl, maxb, mlevel, 658 mlevelist, mnauf, metapar, rlo0, rlo1, rla0, rla1, 659 momega, momegadiff, mgauss, msmooth, meta, metadiff, mdpdeta 660 661 Parameters 662 ---------- 663 c : :obj:`ControlFile` 664 Contains all the parameters of CONTROL file and 665 command line. 666 667 filename : :obj:`string` 685 668 Name of the namelist file. 686 669 687 @Return: 688 <nothing> 670 Return 671 ------ 672 689 673 ''' 690 674 … … 734 718 735 719 def deacc_fluxes(self, inputfiles, c): 736 ''' 737 @Description: 738 Goes through all flux fields in ordered time and de-accumulate 739 the fields. Afterwards the fields are disaggregated in time. 740 Different versions of disaggregation is provided for rainfall 741 data (darain, modified linear) and the surface fluxes and 742 stress data (dapoly, cubic polynomial). 743 744 @Input: 745 self: instance of EcFlexpart 746 The current object of the class. 747 748 inputfiles: instance of UioFiles 749 Contains a list of files. 750 751 c: instance of class ControlFile 752 Contains all the parameters of CONTROL file and 753 command line. 754 For more information about format and content of the parameter 755 see documentation. 756 757 @Return: 758 <nothing> 720 '''Goes through all flux fields in ordered time and de-accumulate 721 the fields. Afterwards the fields are disaggregated in time. 722 Different versions of disaggregation is provided for rainfall 723 data (darain, modified linear) and the surface fluxes and 724 stress data (dapoly, cubic polynomial). 725 726 Parameters 727 ---------- 728 inputfiles : :obj:`UioFiles` 729 Contains a list of files. 730 731 c : :obj:`ControlFile` 732 Contains all the parameters of CONTROL file and 733 command line. 734 735 Return 736 ------ 737 759 738 ''' 760 739 … … 952 931 953 932 def create(self, inputfiles, c): 954 ''' 955 @Description: 956 This method is based on the ECMWF example index.py 957 https://software.ecmwf.int/wiki/display/GRIB/index.py 958 959 An index file will be created which depends on the combination 960 of "date", "time" and "stepRange" values. This is used to iterate 961 over all messages in each grib file which were passed through the 962 parameter "inputfiles" to seperate specific parameters into fort.* 963 files. Afterwards the FORTRAN program is called to convert 964 the data fields all to the same grid and put them in one file 965 per unique time step (combination of "date", "time" and 966 "stepRange"). 967 968 @Input: 969 self: instance of EcFlexpart 970 The current object of the class. 971 972 inputfiles: instance of UioFiles 973 Contains a list of files. 974 975 c: instance of class ControlFile 976 Contains all the parameters of CONTROL file and 977 command line. 978 For more information about format and content of the parameter 979 see documentation. 980 981 @Return: 982 <nothing> 933 '''An index file will be created which depends on the combination 934 of "date", "time" and "stepRange" values. This is used to iterate 935 over all messages in each grib file which were passed through the 936 parameter "inputfiles" to seperate specific parameters into fort.* 937 files. Afterwards the FORTRAN program is called to convert 938 the data fields all to the same grid and put them in one file 939 per unique time step (combination of "date", "time" and 940 "stepRange"). 941 942 Note 943 ---- 944 This method is based on the ECMWF example index.py 945 https://software.ecmwf.int/wiki/display/GRIB/index.py 946 947 Parameters 948 ---------- 949 inputfiles : :obj:`UioFiles` 950 Contains a list of files. 951 952 c : :obj:`ControlFile` 953 Contains all the parameters of CONTROL file and 954 command line. 955 956 Return 957 ------ 958 983 959 ''' 984 960 … … 1183 1159 1184 1160 def process_output(self, c): 1185 ''' 1186 @Description: 1187 The grib files are postprocessed depending on the selection in 1188 CONTROL file. The resulting files are moved to the output 1189 directory if its not equal to the input directory. 1190 The following modifications might be done if 1191 properly switched in CONTROL file: 1192 GRIB2 - Conversion to GRIB2 1193 ECTRANS - Transfer of files to gateway server 1194 ECSTORAGE - Storage at ECMWF server 1195 1196 @Input: 1197 self: instance of EcFlexpart 1198 The current object of the class. 1199 1200 c: instance of class ControlFile 1201 Contains all the parameters of CONTROL file and 1202 command line. 1203 For more information about format and content of the parameter 1204 see documentation. 1205 1206 @Return: 1207 <nothing> 1161 '''The grib files are postprocessed depending on the selection in 1162 CONTROL file. The resulting files are moved to the output 1163 directory if its not equal to the input directory. 1164 The following modifications might be done if 1165 properly switched in CONTROL file: 1166 GRIB2 - Conversion to GRIB2 1167 ECTRANS - Transfer of files to gateway server 1168 ECSTORAGE - Storage at ECMWF server 1169 1170 Parameters 1171 ---------- 1172 c : :obj:`ControlFile` 1173 Contains all the parameters of CONTROL file and 1174 command line. 1175 1176 Return 1177 ------ 1208 1178 1209 1179 ''' … … 1247 1217 1248 1218 def prepare_fp_files(self, c): 1249 ''' 1250 @Description: 1251 Conversion of GRIB files to FLEXPART binary format. 1252 1253 @Input: 1254 c: instance of class ControlFile 1255 Contains all the parameters of CONTROL file and 1256 command line. 1257 For more information about format and content of the parameter 1258 see documentation. 1259 1260 1261 @Return: 1262 <nothing> 1219 '''Conversion of GRIB files to FLEXPART binary format. 1220 1221 Parameters 1222 ---------- 1223 c : :obj:`ControlFile` 1224 Contains all the parameters of CONTROL file and 1225 command line. 1226 1227 Return 1228 ------ 1229 1263 1230 ''' 1264 1231 # generate AVAILABLE file -
source/python/classes/GribTools.py
r0934db1 r274f9ef 66 66 # -------------------------------------------------------------------------- 67 67 def __init__(self, filenames): 68 ''' 69 @Description: 70 Initialise an object of GribTools and assign a list 71 of filenames. 72 73 @Input: 74 filenames: list of strings 75 A list of filenames. 76 77 @Return: 78 <nothing> 68 '''Initialise an object of GribTools and assign a list of filenames. 69 70 Parameters 71 ---------- 72 filenames : :obj:`list` of :obj:`strings` 73 A list of filenames. 74 75 Return 76 ------ 77 79 78 ''' 80 79 … … 85 84 86 85 def get_keys(self, keynames, wherekeynames=[], wherekeyvalues=[]): 87 ''' 88 @Description:89 get keyvalues for a given list of keynames 90 a where statement can be given (list of key and list of values)91 92 @Input:93 keynames: list of strings94 List of keynames. 95 96 wherekeynames: list of strings, optional97 Default value is an empty list. 98 99 wherekeyvalues: list of strings, optional100 Default value is an empty list. 101 102 @Return:103 return_list: list of strings104 86 '''Get keyvalues for a given list of keynames a where statement 87 can be given (list of key and list of values) 88 89 Parameters 90 ---------- 91 keynames : :obj:`list` of :obj:`string` 92 List of keynames. 93 94 wherekeynames : :obj:`list` of :obj:`string`, optional 95 Default value is an empty list. 96 97 wherekeyvalues : :obj:`list` of :obj:`string`, optional 98 Default value is an empty list. 99 100 Return 101 ------ 102 return_list : :obj:`list` of :obj:`string` 103 List of keyvalues for given keynames. 105 104 ''' 106 105 … … 144 143 def set_keys(self, fromfile, keynames, keyvalues, wherekeynames=[], 145 144 wherekeyvalues=[], strict=False, filemode='w'): 146 ''' 147 @Description: 148 Opens the file to read the grib messages and then write 149 them to a new output file. By default all messages are 150 written out. Also, the keyvalues of the passed list of 151 keynames are set or only those meeting the where statement. 152 (list of key and list of values). 153 154 @Input: 155 fromfile: string 156 Filename of the input file to read the grib messages from. 157 158 keynames: list of strings 159 List of keynames. Default is an empty list. 160 161 keyvalues: list of strings 162 List of keynames. Default is an empty list. 163 164 wherekeynames: list of strings, optional 165 Default value is an empty list. 166 167 wherekeyvalues: list of strings, optional 168 Default value is an empty list. 169 170 strict: boolean, optional 171 Decides if everything from keynames and keyvalues 172 is written out the grib file (False) or only those 173 meeting the where statement (True). Default is False. 174 175 filemode: string, optional 176 Sets the mode for the output file. Default is "w". 177 178 @Return: 179 <nothing> 145 '''Opens the file to read the grib messages and then write 146 them to a new output file. By default all messages are 147 written out. Also, the keyvalues of the passed list of 148 keynames are set or only those meeting the where statement. 149 (list of key and list of values). 150 151 Parameters 152 ---------- 153 fromfile : :obj:`string` 154 Filename of the input file to read the grib messages from. 155 156 keynames : :obj:`list` of :obj:`string` 157 List of keynames. Default is an empty list. 158 159 keyvalues : :obj:`list` of :obj:`string` 160 List of keynames. Default is an empty list. 161 162 wherekeynames : :obj:`list` of :obj:`string`, optional 163 Default value is an empty list. 164 165 wherekeyvalues : :obj:`list` of :obj:`string`, optional 166 Default value is an empty list. 167 168 strict : :obj:`boolean`, optional 169 Decides if everything from keynames and keyvalues 170 is written out the grib file (False) or only those 171 meeting the where statement (True). Default is False. 172 173 filemode : :obj:`string`, optional 174 Sets the mode for the output file. Default is "w". 175 176 Return 177 ------ 180 178 181 179 ''' … … 219 217 def copy(self, filename_in, selectWhere=True, 220 218 keynames=[], keyvalues=[], filemode='w'): 221 ''' 222 Add the content of another input grib file to the objects file but 219 '''Add the content of another input grib file to the objects file but 223 220 only messages corresponding to keys/values passed to the function. 224 221 The selectWhere switch decides if to copy the keys equal to (True) or 225 222 different to (False) the keynames/keyvalues list passed to the function. 226 223 227 @Input: 228 filename_in: string 229 Filename of the input file to read the grib messages from. 230 231 selectWhere: boolean, optional 232 Decides if to copy the keynames and values equal to (True) or 233 different to (False) the keynames/keyvalues list passed to the 234 function. Default is True. 235 236 keynames: list of strings, optional 237 List of keynames. Default is an empty list. 238 239 keyvalues: list of strings, optional 240 List of keynames. Default is an empty list. 241 242 filemode: string, optional 243 Sets the mode for the output file. Default is "w". 244 245 @Return: 246 <nothing> 224 Parameters 225 ---------- 226 filename_in : :obj:`string` 227 Filename of the input file to read the grib messages from. 228 229 selectWhere : :obj:`boolean`, optional 230 Decides if to copy the keynames and values equal to (True) or 231 different to (False) the keynames/keyvalues list passed to the 232 function. Default is True. 233 234 keynames : :obj:`list` of :obj:`string`, optional 235 List of keynames. Default is an empty list. 236 237 keyvalues : :obj:`list` of :obj:`string`, optional 238 List of keynames. Default is an empty list. 239 240 filemode : :obj:`string`, optional 241 Sets the mode for the output file. Default is "w". 242 243 Return 244 ------ 245 247 246 ''' 248 247 … … 284 283 285 284 def index(self, index_keys=["mars"], index_file="my.idx"): 286 ''' 287 @Description:288 Create index file from a list of files if it does not exist or 289 read an index file.290 291 @Input:292 index_keys: list of strings, optional293 Contains the list of key parameter names from294 which the index is to be created.295 Default is a list with a single entry string "mars". 296 297 index_file: string, optional298 Filename where the indices are stored.299 Default is "my.idx". 300 301 @Return:302 iid: integer303 285 '''Create index file from a list of files if it does not exist or 286 read an index file. 287 288 Parameters 289 ---------- 290 index_keys: :obj:`list` of :obj:`string`, optional 291 Contains the list of key parameter names from 292 which the index is to be created. 293 Default is a list with a single entry string "mars". 294 295 index_file: :obj:`string`, optional 296 Filename where the indices are stored. 297 Default is "my.idx". 298 299 Return 300 ------ 301 iid: :obj:`integer` 302 Grib index id. 304 303 ''' 305 304 print("... index will be done") -
source/python/classes/MarsRetrieval.py
r70fee58 r274f9ef 73 73 # ------------------------------------------------------------------------------ 74 74 class MarsRetrieval(object): 75 ''' 76 Class for submitting MARS retrievals. 75 '''Class for submitting MARS retrievals. 77 76 78 77 A description of MARS keywords/arguments and examples of their … … 80 79 https://software.ecmwf.int/wiki/display/UDOC/\ 81 80 Identification+keywords#Identificationkeywords-class 82 83 81 ''' 84 82 … … 88 86 number="", accuracy="", grid="", gaussian="", target="", 89 87 param=""): 90 ''' 91 @Description: 92 Initialises the instance of the MarsRetrieval class and 93 defines and assigns a set of the necessary retrieval parameters 94 for the FLEXPART input data. 95 A description of MARS keywords/arguments, their dependencies 96 on each other and examples of their values can be found here: 97 98 https://software.ecmwf.int/wiki/display/UDOC/MARS+keywords 99 100 @Input: 101 self: instance of MarsRetrieval 102 For description see class documentation. 103 104 server: instance of ECMWFService (from ECMWF Web-API) 105 This is the connection to the ECMWF data servers. 106 It is needed for the pythonic access of ECMWF data. 107 108 public: integer 109 Decides which Web API version is used: 110 0: member-state users and full archive access 111 1: public access and limited access to the public server and 112 datasets. Needs the parameter dataset. 113 Default is "0" and for member-state users. 114 115 marsclass: string, optional 116 Characterisation of dataset. E.g. EI (ERA-Interim), 117 E4 (ERA40), OD (Operational archive), ea (ERA5). 118 Default is the ERA-Interim dataset "ei". 119 120 dataset: string, optional 121 For public datasets there is the specific naming and parameter 122 dataset which has to be used to characterize the type of 123 data. Usually there is less data available, either in times, 124 domain or parameter. 125 Default is an empty string. 126 127 type: string, optional 128 Determines the type of fields to be retrieved. 129 Selects between observations, images or fields. 130 Examples for fields: Analysis (an), Forecast (fc), 131 Perturbed Forecast (pf), Control Forecast (cf) and so on. 132 Default is an empty string. 133 134 levtype: string, optional 135 Denotes type of level. Has a direct implication on valid 136 levelist values! 137 E.g. model level (ml), pressure level (pl), surface (sfc), 138 potential vorticity (pv), potential temperature (pt) 139 and depth (dp). 140 Default is an empty string. 141 142 levelist: string, optional 143 Specifies the required levels. It has to have a valid 144 correspondence to the selected levtype. 145 Examples: model level: 1/to/137, pressure levels: 500/to/1000 146 Default is an empty string. 147 148 repres: string, optional 149 Selects the representation of the archived data. 150 E.g. sh - spherical harmonics, gg - Gaussian grid, 151 ll - latitude/longitude, ... 152 Default is an empty string. 153 154 date: string, optional 155 Specifies the Analysis date, the Forecast base date or 156 Observations date. Valid formats are: 157 Absolute as YYYY-MM-DD or YYYYMMDD. 158 Default is an empty string. 159 160 resol: string, optional 161 Specifies the desired triangular truncation of retrieved data, 162 before carrying out any other selected post-processing. 163 The default is automatic truncation (auto), by which the lowest 164 resolution compatible with the value specified in grid is 165 automatically selected for the retrieval. 166 Users wanting to perform post-processing from full spectral 167 resolution should specify Archived Value (av). 168 The following are examples of existing resolutions found in 169 the archive: 63, 106, 159, 213, 255, 319, 399, 511, 799 or 1279. 170 This keyword has no meaning/effect if the archived data is 171 not in spherical harmonics representation. 172 The best selection can be found here: 173 https://software.ecmwf.int/wiki/display/UDOC/\ 174 Retrieve#Retrieve-Truncationbeforeinterpolation 175 Default is an empty string. 176 177 stream: string, optional 178 Identifies the forecasting system used to generate the data. 179 E.g. oper (Atmospheric model), enfo (Ensemble forecats), ... 180 Default is an empty string. 181 182 area: string, optional 183 Specifies the desired sub-area of data to be extracted. 184 Areas can be defined to wrap around the globe. 185 186 Latitude values must be given as signed numbers, with: 187 north latitudes (i.e. north of the equator) 188 being positive (e.g: 40.5) 189 south latitutes (i.e. south of the equator) 190 being negative (e.g: -50.5) 191 Longtitude values must be given as signed numbers, with: 192 east longitudes (i.e. east of the 0 degree meridian) 193 being positive (e.g: 35.0) 194 west longitudes (i.e. west of the 0 degree meridian) 195 being negative (e.g: -20.5) 196 197 E.g.: North/West/South/East 198 Default is an empty string. 199 200 time: string, optional 201 Specifies the time of the data in hours and minutes. 202 Valid values depend on the type of data: Analysis time, 203 Forecast base time or First guess verification time 204 (all usually at synoptic hours: 00, 06, 12 and 18 ). 205 Observation time (any combination in hours and minutes is valid, 206 subject to data availability in the archive). 207 The syntax is HHMM or HH:MM. If MM is omitted it defaults to 00. 208 Default is an empty string. 209 210 step: string, optional 211 Specifies the forecast time step from forecast base time. 212 Valid values are hours (HH) from forecast base time. It also 213 specifies the length of the forecast which verifies at 214 First Guess time. 215 E.g. 1/3/6-hourly 216 Default is an empty string. 217 218 expver: string, optional 219 The version of the dataset. Each experiment is assigned a 220 unique code (version). Production data is assigned 1 or 2, 221 and experimental data in Operations 11, 12 ,... 222 Research or Member State's experiments have a four letter 223 experiment identifier. 224 Default is "1". 225 226 number: string, optional 227 Selects the member in ensemble forecast run. (Only then it 228 is necessary.) It has a different meaning depending on 229 the type of data. 230 E.g. Perturbed Forecasts: specifies the Ensemble forecast member 231 Default is an empty string. 232 233 accuracy: string, optional 234 Specifies the number of bits per value to be used in the 235 generated GRIB coded fields. 236 A positive integer may be given to specify the preferred number 237 of bits per packed value. This must not be greater than the 238 number of bits normally used for a Fortran integer on the 239 processor handling the request (typically 32 or 64 bit). 240 Within a compute request the accuracy of the original fields 241 can be passed to the result field by specifying accuracy=av. 242 Default is an empty string. 243 244 grid: string, optional 245 Specifies the output grid which can be either a Gaussian grid 246 or a Latitude/Longitude grid. MARS requests specifying 247 grid=av will return the archived model grid. 248 249 Lat/Lon grid: The grid spacing needs to be an integer 250 fraction of 90 degrees e.g. grid = 0.5/0.5 251 252 Gaussian grid: specified by a letter denoting the type of 253 Gaussian grid followed by an integer (the grid number) 254 representing the number of lines between the Pole and Equator, 255 e.g. 256 grid = F160 - full (or regular) Gaussian grid with 257 160 latitude lines between the pole and equator 258 grid = N320 - ECMWF original reduced Gaussian grid with 259 320 latitude lines between the pole and equator, 260 see Reduced Gaussian Grids for grid numbers used at ECMWF 261 grid = O640 - ECMWF octahedral (reduced) Gaussian grid with 262 640 latitude lines between the pole and equator 263 Default is an empty string. 264 265 gaussian: string, optional 266 This parameter is deprecated and should no longer be used. 267 Specifies the desired type of Gaussian grid for the output. 268 Valid Gaussian grids are quasi-regular (reduced) or regular. 269 Keyword gaussian can only be specified together with 270 keyword grid. Gaussian without grid has no effect. 271 Default is an empty string. 272 273 target: string, optional 274 Specifies a file into which data is to be written after 275 retrieval or manipulation. Path names should always be 276 enclosed in double quotes. The MARS client supports automatic 277 generation of multiple target files using MARS keywords 278 enclosed in square brackets [ ]. If the environment variable 279 MARS_MULTITARGET_STRICT_FORMAT is set to 1 before calling mars, 280 the keyword values will be used in the filename as shown by 281 the ecCodes GRIB tool grib_ls -m, e.g. with 282 MARS_MULTITARGET_STRICT_FORMAT set to 1 the keywords time, 283 expver and param will be formatted as 0600, 0001 and 129.128 284 rather than 600, 1 and 129. 285 Default is an empty string. 286 287 param: string, optional 288 Specifies the meteorological parameter. 289 The list of meteorological parameters in MARS is extensive. 290 Their availability is directly related to their meteorological 291 meaning and, therefore, the rest of directives specified 292 in the MARS request. 293 Meteorological parameters can be specified by their 294 GRIB code (param=130), their mnemonic (param=t) or 295 full name (param=temperature). 296 The list of parameter should be seperated by a "/"-sign. 297 E.g. 130/131/133 298 Default is an empty string. 299 300 @Return: 301 <nothing> 88 '''Initialises the instance of the MarsRetrieval class and 89 defines and assigns a set of the necessary retrieval parameters 90 for the FLEXPART input data. 91 A description of MARS keywords/arguments, their dependencies 92 on each other and examples of their values can be found here: 93 94 https://software.ecmwf.int/wiki/display/UDOC/MARS+keywords 95 96 Parameters 97 ---------- 98 server : :obj:`ECMWFService` 99 This is the connection to the ECMWF data servers. 100 It is needed for the pythonic access of ECMWF data. 101 102 public : :obj:`integer` 103 Decides which Web API version is used: 104 0: member-state users and full archive access 105 1: public access and limited access to the public server and 106 datasets. Needs the parameter dataset. 107 Default is "0" and for member-state users. 108 109 marsclass : :obj:`string`, optional 110 Characterisation of dataset. E.g. EI (ERA-Interim), 111 E4 (ERA40), OD (Operational archive), ea (ERA5). 112 Default is the ERA-Interim dataset "ei". 113 114 dataset : :obj:`string`, optional 115 For public datasets there is the specific naming and parameter 116 dataset which has to be used to characterize the type of 117 data. Usually there is less data available, either in times, 118 domain or parameter. 119 Default is an empty string. 120 121 type : :obj:`string`, optional 122 Determines the type of fields to be retrieved. 123 Selects between observations, images or fields. 124 Examples for fields: Analysis (an), Forecast (fc), 125 Perturbed Forecast (pf), Control Forecast (cf) and so on. 126 Default is an empty string. 127 128 levtype : :obj:`string`, optional 129 Denotes type of level. Has a direct implication on valid 130 levelist values! 131 E.g. model level (ml), pressure level (pl), surface (sfc), 132 potential vorticity (pv), potential temperature (pt) 133 and depth (dp). 134 Default is an empty string. 135 136 levelist : :obj:`string`, optional 137 Specifies the required levels. It has to have a valid 138 correspondence to the selected levtype. 139 Examples: model level: 1/to/137, pressure levels: 500/to/1000 140 Default is an empty string. 141 142 repres : :obj:`string`, optional 143 Selects the representation of the archived data. 144 E.g. sh - spherical harmonics, gg - Gaussian grid, 145 ll - latitude/longitude, ... 146 Default is an empty string. 147 148 date : :obj:`string`, optional 149 Specifies the Analysis date, the Forecast base date or 150 Observations date. Valid formats are: 151 Absolute as YYYY-MM-DD or YYYYMMDD. 152 Default is an empty string. 153 154 resol : :obj:`string`, optional 155 Specifies the desired triangular truncation of retrieved data, 156 before carrying out any other selected post-processing. 157 The default is automatic truncation (auto), by which the lowest 158 resolution compatible with the value specified in grid is 159 automatically selected for the retrieval. 160 Users wanting to perform post-processing from full spectral 161 resolution should specify Archived Value (av). 162 The following are examples of existing resolutions found in 163 the archive: 63, 106, 159, 213, 255, 319, 399, 511, 799 or 1279. 164 This keyword has no meaning/effect if the archived data is 165 not in spherical harmonics representation. 166 The best selection can be found here: 167 https://software.ecmwf.int/wiki/display/UDOC/\ 168 Retrieve#Retrieve-Truncationbeforeinterpolation 169 Default is an empty string. 170 171 stream : :obj:`string`, optional 172 Identifies the forecasting system used to generate the data. 173 E.g. oper (Atmospheric model), enfo (Ensemble forecats), ... 174 Default is an empty string. 175 176 area : :obj:`string`, optional 177 Specifies the desired sub-area of data to be extracted. 178 Areas can be defined to wrap around the globe. 179 180 Latitude values must be given as signed numbers, with: 181 north latitudes (i.e. north of the equator) 182 being positive (e.g: 40.5) 183 south latitutes (i.e. south of the equator) 184 being negative (e.g: -50.5) 185 Longtitude values must be given as signed numbers, with: 186 east longitudes (i.e. east of the 0 degree meridian) 187 being positive (e.g: 35.0) 188 west longitudes (i.e. west of the 0 degree meridian) 189 being negative (e.g: -20.5) 190 191 E.g.: North/West/South/East 192 Default is an empty string. 193 194 time : :obj:`string`, optional 195 Specifies the time of the data in hours and minutes. 196 Valid values depend on the type of data: Analysis time, 197 Forecast base time or First guess verification time 198 (all usually at synoptic hours: 00, 06, 12 and 18 ). 199 Observation time (any combination in hours and minutes is valid, 200 subject to data availability in the archive). 201 The syntax is HHMM or HH:MM. If MM is omitted it defaults to 00. 202 Default is an empty string. 203 204 step : :obj:`string`, optional 205 Specifies the forecast time step from forecast base time. 206 Valid values are hours (HH) from forecast base time. It also 207 specifies the length of the forecast which verifies at 208 First Guess time. 209 E.g. 1/3/6-hourly 210 Default is an empty string. 211 212 expver : :obj:`string`, optional 213 The version of the dataset. Each experiment is assigned a 214 unique code (version). Production data is assigned 1 or 2, 215 and experimental data in Operations 11, 12 ,... 216 Research or Member State's experiments have a four letter 217 experiment identifier. 218 Default is "1". 219 220 number : :obj:`string`, optional 221 Selects the member in ensemble forecast run. (Only then it 222 is necessary.) It has a different meaning depending on 223 the type of data. 224 E.g. Perturbed Forecasts: specifies the Ensemble forecast member 225 Default is an empty string. 226 227 accuracy : :obj:`string`, optional 228 Specifies the number of bits per value to be used in the 229 generated GRIB coded fields. 230 A positive integer may be given to specify the preferred number 231 of bits per packed value. This must not be greater than the 232 number of bits normally used for a Fortran integer on the 233 processor handling the request (typically 32 or 64 bit). 234 Within a compute request the accuracy of the original fields 235 can be passed to the result field by specifying accuracy=av. 236 Default is an empty string. 237 238 grid : :obj:`string`, optional 239 Specifies the output grid which can be either a Gaussian grid 240 or a Latitude/Longitude grid. MARS requests specifying 241 grid=av will return the archived model grid. 242 243 Lat/Lon grid: The grid spacing needs to be an integer 244 fraction of 90 degrees e.g. grid = 0.5/0.5 245 246 Gaussian grid: specified by a letter denoting the type of 247 Gaussian grid followed by an integer (the grid number) 248 representing the number of lines between the Pole and Equator, 249 e.g. 250 grid = F160 - full (or regular) Gaussian grid with 251 160 latitude lines between the pole and equator 252 grid = N320 - ECMWF original reduced Gaussian grid with 253 320 latitude lines between the pole and equator, 254 see Reduced Gaussian Grids for grid numbers used at ECMWF 255 grid = O640 - ECMWF octahedral (reduced) Gaussian grid with 256 640 latitude lines between the pole and equator 257 Default is an empty string. 258 259 gaussian : :obj:`string`, optional 260 This parameter is deprecated and should no longer be used. 261 Specifies the desired type of Gaussian grid for the output. 262 Valid Gaussian grids are quasi-regular (reduced) or regular. 263 Keyword gaussian can only be specified together with 264 keyword grid. Gaussian without grid has no effect. 265 Default is an empty string. 266 267 target : :obj:`string`, optional 268 Specifies a file into which data is to be written after 269 retrieval or manipulation. Path names should always be 270 enclosed in double quotes. The MARS client supports automatic 271 generation of multiple target files using MARS keywords 272 enclosed in square brackets [ ]. If the environment variable 273 MARS_MULTITARGET_STRICT_FORMAT is set to 1 before calling mars, 274 the keyword values will be used in the filename as shown by 275 the ecCodes GRIB tool grib_ls -m, e.g. with 276 MARS_MULTITARGET_STRICT_FORMAT set to 1 the keywords time, 277 expver and param will be formatted as 0600, 0001 and 129.128 278 rather than 600, 1 and 129. 279 Default is an empty string. 280 281 param : :obj:`string`, optional 282 Specifies the meteorological parameter. 283 The list of meteorological parameters in MARS is extensive. 284 Their availability is directly related to their meteorological 285 meaning and, therefore, the rest of directives specified 286 in the MARS request. 287 Meteorological parameters can be specified by their 288 GRIB code (param=130), their mnemonic (param=t) or 289 full name (param=temperature). 290 The list of parameter should be seperated by a "/"-sign. 291 E.g. 130/131/133 292 Default is an empty string. 293 294 Return 295 ------ 296 302 297 ''' 303 298 … … 328 323 329 324 def display_info(self): 330 ''' 331 @Description: 332 Prints all class attributes and their values to the 333 standard output. 334 335 @Input: 336 self: instance of MarsRetrieval 337 For description see class documentation. 338 339 @Return: 340 <nothing> 325 '''Prints all class attributes and their values to the 326 standard output. 327 328 Parameters 329 ---------- 330 331 Return 332 ------ 333 341 334 ''' 342 335 # Get all class attributes and their values as a dictionary … … 355 348 356 349 def print_info(self, inputdir, request_number): 357 ''' 358 @Description: 359 Prints all mars requests to an extra file for debugging and 360 information. 361 362 @Input: 363 self: instance of MarsRetrieval 364 For description see class documentation. 365 366 inputdir: string 367 The path where all data from the retrievals are stored. 368 369 request_number: integer 370 Number of mars requests for flux and non-flux data. 371 372 @Return: 373 <nothing> 350 '''Prints all mars requests to an extra file for debugging and 351 information. 352 353 Parameters 354 ---------- 355 inputdir : :obj:`string` 356 The path where all data from the retrievals are stored. 357 358 request_number : :obj:`integer` 359 Number of mars requests for flux and non-flux data. 360 361 Return 362 ------ 363 374 364 ''' 375 365 # Get all class attributes and their values as a dictionary … … 393 383 394 384 def print_infodata_csv(self, inputdir, request_number): 395 ''' 396 @Description: 397 Write all request parameter in alpabetical order into a "csv" file. 398 399 @Input: 400 self: instance of MarsRetrieval 401 For description see class documentation. 402 403 inputdir: string 404 The path where all data from the retrievals are stored. 405 406 request_number: integer 407 Number of mars requests for flux and non-flux data. 408 409 @Return: 410 <nothing> 385 '''Write all request parameter in alpabetical order into a "csv" file. 386 387 Parameters 388 ---------- 389 inputdir : :obj:`string` 390 The path where all data from the retrievals are stored. 391 392 request_number : :obj:`integer` 393 Number of mars requests for flux and non-flux data. 394 395 Return 396 ------ 397 411 398 ''' 412 399 … … 427 414 428 415 def data_retrieve(self): 429 ''' 430 @Description: 431 Submits a MARS retrieval. Depending on the existence of 432 ECMWF Web-API it is submitted via Python or a 433 subprocess in the Shell. The parameter for the mars retrieval 434 are taken from the defined class attributes. 435 436 @Input: 437 self: instance of MarsRetrieval 438 For description see class documentation. 439 440 @Return: 441 <nothing> 416 '''Submits a MARS retrieval. Depending on the existence of 417 ECMWF Web-API it is submitted via Python or a 418 subprocess in the Shell. The parameter for the mars retrieval 419 are taken from the defined class attributes. 420 421 Parameters 422 ---------- 423 424 Return 425 ------ 426 442 427 ''' 443 428 # Get all class attributes and their values as a dictionary -
source/python/classes/UioFiles.py
rca867de r274f9ef 61 61 62 62 class UioFiles(object): 63 ''' 64 Class to manipulate files. At initialisation it has the pattern 63 '''Class to manipulate files. At initialisation it has the pattern 65 64 which stores a regular expression pattern for the files, the path 66 65 to the files and the files already. … … 70 69 # -------------------------------------------------------------------------- 71 70 def __init__(self, path, pattern): 72 ''' 73 @Description: 74 Assignes a specific pattern for these files. 71 '''Assignes a specific pattern for these files. 75 72 76 @Input: 77 self: instance of UioFiles 78 Description see class documentation. 73 Parameters 74 ---------- 75 path : :obj:`string` 76 Directory where to list the files. 79 77 80 path: string81 Directory where to list the files.78 pattern : :obj:`string` 79 Regular expression pattern. For example: '\*.grb' 82 80 83 pattern: string84 Regular expression pattern. For example: '*.grb'81 Return 82 ------ 85 83 86 @Return:87 <nothing>88 84 ''' 89 85 … … 98 94 #@profiling.timefn 99 95 def __list_files__(self, path): 100 ''' 101 @Description: 102 Lists all files in the directory with the matching 103 regular expression pattern. 96 '''Lists all files in the directory with the matching 97 regular expression pattern. 104 98 105 @Input: 106 self: instance of UioFiles 107 Description see class documentation. 99 Parameters 100 ---------- 101 path : :obj:`string` 102 Path to the files. 108 103 109 path: string110 Path to the files.104 Return 105 ------ 111 106 112 @Return:113 <nothing>114 107 ''' 115 108 # Get the absolute path … … 124 117 125 118 def __str__(self): 126 ''' 127 @Description: 128 Converts the list of files into a single string. 129 The entries are sepereated by "," sign. 119 '''Converts the list of files into a single string. 120 The entries are sepereated by "," sign. 130 121 131 @Input: 132 self: instance of UioFiles 133 Description see class documentation. 122 Parameters 123 ---------- 134 124 135 @Return: 136 files_string: string 137 The content of the list as a single string. 125 Return 126 ------ 127 files_string : :obj:`string` 128 The content of the list as a single string. 138 129 ''' 139 130 … … 144 135 145 136 def delete_files(self): 146 ''' 147 @Description: 148 Deletes the files. 137 '''Deletes the files. 149 138 150 @Input: 151 self: instance of UioFiles 152 Description see class documentation. 139 Parameters 140 ---------- 153 141 154 @Return: 155 <nothing> 142 Return 143 ------ 144 156 145 ''' 157 146
Note: See TracChangeset
for help on using the changeset viewer.