Changeset 8209738 in flex_extract.git for Source/Python/Mods/tools.py
- Timestamp:
- May 7, 2020, 3:32:03 PM (4 years ago)
- Branches:
- master, ctbto, dev
- Children:
- b936fd3
- Parents:
- 05e9362
- File:
-
- 1 edited
Legend:
- Unmodified
- Added
- Removed
-
Source/Python/Mods/tools.py
rd1bfa24 r8209738 18 18 # - added documentation 19 19 # - moved all non class methods from former file Flexparttools in here 20 # - sep erated args and control interpretation20 # - separated args and control interpretation 21 21 # - added functions get_list_as_string, read_ecenv, send_mail, make_dir, 22 22 # put_file_to_ecserver, submit_job_to_ecserver, get_informations, … … 118 118 119 119 def none_or_str(value): 120 '''Converts the input string into pythons None-type if the string121 contains string "None".120 '''Converts the input string into Pythons None type if it 121 contains the string "None". 122 122 123 123 Parameters … … 130 130 None or value: 131 131 Return depends on the content of the input value. If it was "None", 132 then the python type None is returned. Otherwise the string itself.132 then the Python type None is returned, otherwise the string itself. 133 133 ''' 134 134 if value == 'None': … … 137 137 138 138 def none_or_int(value): 139 '''Converts the input string into pythons None-type if the string140 contains string "None" . Otherwise it is converted to an integer value.139 '''Converts the input string into Pythons None-type if it 140 contains string "None"; otherwise it is converted to an integer value. 141 141 142 142 Parameters … … 158 158 def get_cmdline_args(): 159 159 '''Decomposes the command line arguments and assigns them to variables. 160 Apply default values for non mentioned arguments.160 Apply default values for arguments not present. 161 161 162 162 Parameters … … 166 166 ------ 167 167 args : Namespace 168 Contains the command line arguments from script/program call.168 Contains the command line arguments from the script / program call. 169 169 ''' 170 170 … … 191 191 parser.add_argument("--basetime", dest="basetime", 192 192 type=none_or_int, default=None, 193 help="base such as 0 or 12 (for half day retrievals)")193 help="base time such as 0 or 12 (for half day retrievals)") 194 194 parser.add_argument("--step", dest="step", 195 195 type=none_or_str, default=None, … … 200 200 parser.add_argument("--area", dest="area", 201 201 type=none_or_str, default=None, 202 help="area defined asnorth/west/south/east")202 help="area, defined by north/west/south/east") 203 203 204 204 # some switches 205 205 parser.add_argument("--debug", dest="debug", 206 206 type=none_or_int, default=None, 207 help="debug mode - leave temporary files intact")207 help="debug mode - temporary files will be conserved") 208 208 parser.add_argument("--oper", dest="oper", 209 209 type=none_or_int, default=None, 210 help='operational mode - prepares dates with'210 help='operational mode - prepares dates from ' 211 211 'environment variables') 212 212 parser.add_argument("--request", dest="request", 213 213 type=none_or_int, default=None, 214 help="list all marsrequests in file mars_requests.dat")214 help="list all MARS requests in file mars_requests.dat") 215 215 parser.add_argument("--public", dest="public", 216 216 type=none_or_int, default=None, 217 help="public mode - retrieves thepublic datasets")217 help="public mode - retrieves public datasets") 218 218 parser.add_argument("--rrint", dest="rrint", 219 219 type=none_or_int, default=None, 220 help='Selection of old or new precipitation'221 'interpolation :\n'220 help='Selection of old or new ' 221 'interpolation method for precipitation:\n' 222 222 ' 0 - old method\n' 223 223 ' 1 - new method (additional subgrid points)') … … 226 226 parser.add_argument("--inputdir", dest="inputdir", 227 227 type=none_or_str, default=None, 228 help='Path to t he temporary directory for the'229 'retriev algrib files and other processing files.')228 help='Path to temporary directory for ' 229 'retrieved grib files and other processing files.') 230 230 parser.add_argument("--outputdir", dest="outputdir", 231 231 type=none_or_str, default=None, 232 help='Path to the final directory where the final'233 'FLEXPART ready input files are stored.')232 help='Path to final directory where ' 233 'FLEXPART input files will be stored.') 234 234 235 235 # this is only used by prepare_flexpart.py to rerun a postprocessing step 236 236 parser.add_argument("--ppid", dest="ppid", 237 237 type=none_or_str, default=None, 238 help='This is the specify parent process id of a '238 help='This is the specify the parent process id of a ' 239 239 'single flex_extract run to identify the files. ' 240 240 'It is the second number in the GRIB files.') … … 243 243 parser.add_argument("--job_template", dest='job_template', 244 244 type=none_or_str, default="job.temp", 245 help='The job template file which are adapted to be ' 246 'submitted to the batch system on ECMWF server.') 245 help='Job template file. Will be used for submission ' 246 'to the batch system on the ECMWF server after ' 247 'modification.') 247 248 parser.add_argument("--queue", dest="queue", 248 249 type=none_or_str, default=None, 249 help='The ECMWF server name for submission of the'250 'job script to the batch system '250 help='The name of the ECMWF server name where the' 251 'job script is to be submitted ' 251 252 '(e.g. ecgate | cca | ccb)') 252 253 … … 289 290 290 291 It keeps the final FLEXPART input files if program runs without 291 ECMWF A piand keywords "ectrans" or "ecstorage" are set to "1".292 ECMWF API and keywords "ectrans" or "ecstorage" are set to "1". 292 293 293 294 Parameters … … 410 411 411 412 This method combines the single characters of the passed arguments 412 with each other . Sothat each character of each argument value413 with each other in a way that each character of each argument value 413 414 will be combined with each character of the other arguments as a tuple. 414 415 … … 519 520 pars : str 520 521 Addpar argument from CONTROL file in the format of 521 parameter names instead of ids. The parameter short 522 names are sepearted with "/" and they are passed as 523 one single string. 522 parameter names instead of IDs. The parameter short 523 names are separated by "/" and passed as one single string. 524 524 525 525 table : dict … … 555 555 556 556 def to_param_id_with_tablenumber(pars, table): 557 '''Transform parameter names to parameter ids and add table id.557 '''Transform parameter names to parameter IDs and add table ID. 558 558 559 559 Conversion with ECMWF grib table 128. … … 563 563 pars : str 564 564 Addpar argument from CONTROL file in the format of 565 parameter names instead of ids. The parameter short 566 names are sepearted with "/" and they are passed as 567 one single string. 565 parameter names instead of ID. The parameter short 566 names are separated by "/" and passed as one single string. 568 567 569 568 table : dict … … 576 575 spar : str 577 576 List of addpar parameters from CONTROL file transformed to 578 parameter ids in the format of integer.577 parameter IDs in the format of integer. 579 578 ''' 580 579 if not pars: … … 598 597 599 598 def get_list_as_string(list_obj, concatenate_sign=', '): 600 '''Converts a list of arbitrary content into a single string. 599 '''Converts a list of arbitrary content into a single string using a given 600 concatenation character. 601 601 602 602 Parameters … … 624 624 '''Creates a directory. 625 625 626 I t gives a warning if the directory already exists and skips process.627 The program stops only if there is another problem.626 If the directory already exists, an information is printed and the creation 627 skipped. The program stops only if there is another problem. 628 628 629 629 Parameters … … 641 641 # errno.EEXIST = directory already exists 642 642 if e.errno == errno.EEXIST: 643 print(' WARNING: Directory {0} already exists!'.format(directory))643 print('INFORMATION: Directory {0} already exists!'.format(directory)) 644 644 else: 645 645 raise # re-raise exception if a different error occured … … 720 720 ------ 721 721 job_id : int 722 The id number of the job as a reference at the ecmwfserver.722 The id number of the job as a reference at the ECMWF server. 723 723 ''' 724 724 … … 732 732 733 733 print('\n... Do you have a valid ecaccess certification key?') 734 sys.exit('... ECACCESS-JOB-SUBMITFAILED!')734 sys.exit('... ecaccess-job-submit FAILED!') 735 735 except OSError as e: 736 736 print('... ERROR CODE: ' + str(e.errno)) … … 738 738 739 739 print('\n... Most likely the ECACCESS library is not available!') 740 sys.exit('... ECACCESS-JOB-SUBMITFAILED!')740 sys.exit('... ecaccess-job-submit FAILED!') 741 741 742 742 return job_id.decode() … … 744 744 745 745 def get_informations(filename): 746 ''' Gets basic information from an example grib file.747 748 Th ese information are importantfor later use and the749 initialization of numpy arrays for data storing.746 '''Extracts basic information from a sample grib file. 747 748 This information is needed for later use and the 749 initialization of numpy arrays where data are stored. 750 750 751 751 Parameters … … 768 768 769 769 # --- open file --- 770 print("Opening file for getting information data--- %s" % filename)770 print("Opening grib file for extraction of information --- %s" % filename) 771 771 with open(filename, 'rb') as f: 772 772 # load first message from file … … 785 785 ] 786 786 787 print('\nInformation s are: ')787 print('\nInformation extracted: ') 788 788 for key in keys: 789 789 # Get the value of the key in a grib message. … … 798 798 799 799 def get_dimensions(info, purefc, dtime, index_vals, start_date, end_date): 800 '''This function specifies the correct dimensions for x, y and t.800 '''This function specifies the correct dimensions for x, y, and t. 801 801 802 802 Parameters … … 817 817 index_vals : list of list of str 818 818 Contains the values from the keys used for a distinct selection 819 of grib messages in processingthe grib files.819 of GRIB messages in processing the grib files. 820 820 Content looks like e.g.: 821 821 index_vals[0]: ('20171106', '20171107', '20171108') ; date … … 849 849 850 850 def execute_subprocess(cmd_list, error_msg='SUBPROCESS FAILED!'): 851 '''Executes a command line instructionvia a subprocess.851 '''Executes a command via a subprocess. 852 852 853 853 Error handling is done if an error occures. … … 856 856 ---------- 857 857 cmd_list : list of str 858 A list of the components for the command line execution. Each 859 list entry is a single part of the command which is seperated from 860 the rest by a blank space. 861 E.g. ['mv', file1, file2] 858 A list of the components for the command line execution. 859 They will be concatenated with blank space for the command 860 to be submitted, like ['mv', file1, file2] for mv file1 file2. 862 861 863 862 Return 864 863 ------ 865 864 error_msg : str, optional 866 The possible error message if the subprocess failed.867 By default it will just tell"SUBPROCESS FAILED!".865 Error message if the subprocess fails. 866 By default it will just say "SUBPROCESS FAILED!". 868 867 ''' 869 868
Note: See TracChangeset
for help on using the changeset viewer.