Changeset 6f951ca in flex_extract.git for source/python/mods
- Timestamp:
- Jan 15, 2019, 1:05:10 AM (5 years ago)
- Branches:
- master, ctbto, dev
- Children:
- d4696e0
- Parents:
- 2625ca8
- Location:
- source/python/mods
- Files:
-
- 5 edited
-
checks.py (modified) (32 diffs)
-
disaggregation.py (modified) (8 diffs)
-
get_mars_data.py (modified) (10 diffs)
-
prepare_flexpart.py (modified) (4 diffs)
-
tools.py (modified) (27 diffs)
Legend:
- Unmodified
- Added
- Removed
-
source/python/mods/checks.py
rf2616a3 r6f951ca 1 1 #!/usr/bin/env python 2 2 # -*- coding: utf-8 -*- 3 # #*******************************************************************************3 #******************************************************************************* 4 4 # @Author: Anne Philipp (University of Vienna) 5 5 # … … 9 9 # 10 10 # @License: 11 # (C) Copyright 2014-2018. 11 # (C) Copyright 2014-2019. 12 # Anne Philipp, Leopold Haimberger 12 13 # 13 # This software is licensed under the terms of the Apache Licence Version 2.0 14 # which can be obtained at http://www.apache.org/licenses/LICENSE-2.0. 15 # 16 # @Modul Description: 17 # 18 # 19 # @Module Content: 20 21 # 14 # This work is licensed under the Creative Commons Attribution 4.0 15 # International License. To view a copy of this license, visit 16 # http://creativecommons.org/licenses/by/4.0/ or send a letter to 17 # Creative Commons, PO Box 1866, Mountain View, CA 94042, USA. 22 18 #******************************************************************************* 19 '''This module contains check methods for the CONTROL paramaters. 20 ''' 23 21 24 22 # ------------------------------------------------------------------------------ … … 27 25 28 26 import os 27 import sys 29 28 import _config 30 29 import exceptions … … 40 39 Parameters 41 40 ---------- 42 c : :obj:`ControlFile`41 c : ControlFile 43 42 Contains all the parameters of CONTROL file and 44 43 command line. 45 44 46 logicals : :obj:`list` of (:obj:`string` or :obj:`integer`)45 logicals : list of (str or int) 47 46 Names of the switches that are used to control the flow of the 48 47 program. … … 67 66 Parameters 68 67 ---------- 69 grid : :obj:`string`68 grid : str 70 69 Contains grid information 71 70 72 71 Return 73 72 ------ 74 grid : :obj:``string`73 grid : str 75 74 Contains grid in format Lat/lon. E.g. 0.1/0.1 76 75 ''' … … 108 107 Parameters 109 108 ---------- 110 grid : :obj:`string`109 grid : str 111 110 Contains grid information. 112 111 113 area : :obj:`string`112 area : str 114 113 Contains area informtion. 115 114 116 upper : :obj:`string`115 upper : str 117 116 The northern most latitude. 118 117 119 lower : :obj:`string`118 lower : str 120 119 The souther most latitude. 121 120 122 left : :obj:`string`121 left : str 123 122 The western most longitude. 124 123 125 right : :obj:`string`124 right : str 126 125 The eastern most longiude. 127 126 128 127 Return 129 128 ------ 130 grid : :obj:``string`129 grid : str 131 130 Contains grid in format Lat/lon. E.g. 0.1/0.1 132 131 ''' … … 173 172 Parameters 174 173 ---------- 175 levelist : :obj:`string`174 levelist : str 176 175 Specifies the level list. 177 176 Examples: model level: 1/to/137, pressure levels: 500/to/1000 178 177 179 level : :obj:`string`178 level : str 180 179 Specifies the maximum level. 181 180 182 181 Return 183 182 ------ 184 levelist : :obj:`string`183 levelist : str 185 184 Specifies the required levels. It has to have a valid 186 185 correspondence to the selected levtype. 187 186 Examples: model level: 1/to/137, pressure levels: 500/to/1000 188 187 189 level : :obj:`string`188 level : str 190 189 Specifies the maximum level. It has to be one of the 191 190 available maximum level number as contained in the variable … … 221 220 Parameters 222 221 ---------- 223 c : :obj:`ControlFile`222 c : ControlFile 224 223 Contains all the parameters of CONTROL file and 225 224 command line. 226 225 227 ppid : :obj:`int` or :obj:`None`226 ppid : int or None 228 227 Contains the ppid number provided by the command line parameter 229 228 of is None otherwise. … … 247 246 Parameters 248 247 ---------- 249 ftype : :obj:`list` of :obj:`string`248 ftype : list of str 250 249 List of field types. 251 250 … … 273 272 Parameters 274 273 ---------- 275 step : :obj:`list` of :obj:`string` or :obj:`string`274 step : list of str or str 276 275 Specifies the forecast time step from forecast base time. 277 276 Valid values are hours (HH) from forecast base time. 278 277 279 mailfail : :obj:`list` of :obj:``string`278 mailfail : list of str 280 279 Contains all email addresses which should be notified. 281 280 It might also contain just the ecmwf user name which will trigger … … 284 283 Return 285 284 ------ 286 step : :obj:`list` of :obj:`string`285 step : list of str 287 286 List of forecast steps in format e.g. [001, 002, ...] 288 287 ''' … … 313 312 Parameters 314 313 ---------- 315 ftype : :obj:`list` of :obj:`string` or :obj:`string`314 ftype : list of str or str 316 315 List of field types. 317 316 318 steps : :obj:`string`317 steps : str 319 318 Specifies the forecast time step from forecast base time. 320 319 Valid values are hours (HH) from forecast base time. … … 322 321 Return 323 322 ------ 324 ftype : :obj:`list` of :obj:`string`323 ftype : list of str 325 324 List of field types. 326 325 ''' … … 340 339 Parameters 341 340 ---------- 342 ftime : :obj:`list` of :obj:`string` or :obj:`string`341 ftime : list of str or str 343 342 The time in hours of the field. 344 343 345 344 Return 346 345 ------ 347 ftime : :obj:`list` of :obj:`string`346 ftime : list of str 348 347 The time in hours of the field. 349 348 ''' … … 358 357 Parameters 359 358 ---------- 360 ftype : :obj:`list` of :obj:`string`359 ftype : list of str 361 360 List of field types. 362 361 363 ftime : :obj:`list` of :obj:`string` or :obj:`string`362 ftime : list of str or str 364 363 The time in hours of the field. 365 364 366 steps : :obj:`string`365 steps : str 367 366 Specifies the forecast time step from forecast base time. 368 367 Valid values are hours (HH) from forecast base time. 369 368 370 maxstep : :obj:`integer`369 maxstep : int 371 370 The maximum forecast time step in hours from the forecast base time. 372 371 This is the maximum step for non flux (accumulated) forecast data. 373 372 374 purefc : :obj:`integer`373 purefc : int 375 374 Switch for definition of pure forecast mode or not. 376 375 377 376 Return 378 377 ------ 379 ftype : :obj:`list` of :obj:`string`378 ftype : list of str 380 379 List of field types. 381 380 382 ftime : :obj:`list` of :obj:`string`381 ftime : list of str 383 382 The time in hours of the field. 384 383 385 steps : :obj:`string`384 steps : str 386 385 Specifies the forecast time step from forecast base time. 387 386 Valid values are hours (HH) from forecast base time. … … 410 409 Parameters 411 410 ---------- 412 mail : :obj:`list` of :obj:`string` or :obj:`string`411 mail : list of str or str 413 412 Contains email addresses for notifications. 414 413 It might also contain just the ecmwf user name which will trigger … … 417 416 Return 418 417 ------ 419 mail : :obj:`list` of :obj:``string`418 mail : list of str 420 419 Contains email addresses for notifications. 421 420 It might also contain just the ecmwf user name which will trigger … … 439 438 Parameters 440 439 ---------- 441 queue : :obj:`string`440 queue : str 442 441 Name of the queue if submitted to the ECMWF servers. 443 442 Used to check if ecuid, ecgid, gateway and destination 444 443 are set correctly and are not empty. 445 444 446 gateway : :obj:`string`445 gateway : str 447 446 The address of the gateway server. 448 447 449 destination : :obj:`string`448 destination : str 450 449 The name of the destination of the gateway server for data 451 450 transfer through ectrans. E.g. name@genericSftp 452 451 453 ecuid : :obj:`string`452 ecuid : str 454 453 ECMWF user id. 455 454 456 ecgid : :obj:`string`455 ecgid : str 457 456 ECMWF group id. 458 457 … … 475 474 Parameters 476 475 ---------- 477 idir : :obj:`string`476 idir : str 478 477 Path to the temporary directory for MARS retrieval data. 479 478 480 odir : :obj:`string`479 odir : str 481 480 Path to the final output directory where the FLEXPART input files 482 481 will be stored. 483 482 484 fpdir : :obj:`string`483 fpdir : str 485 484 Path to FLEXPART root directory. 486 485 487 fedir : :obj:`string`486 fedir : str 488 487 Path to flex_extract root directory. 489 488 490 489 Return 491 490 ------ 492 odir : :obj:`string`491 odir : str 493 492 Path to the final output directory where the FLEXPART input files 494 493 will be stored. 495 494 496 fpdir : :obj:`string`495 fpdir : str 497 496 Path to FLEXPART root directory. 498 497 … … 513 512 Parameters 514 513 ---------- 515 start : :obj:`string`514 start : str 516 515 The start date of the retrieval job. 517 516 518 end : :obj:`string`517 end : str 519 518 The end date of the retrieval job. 520 519 521 520 Return 522 521 ------ 523 start : :obj:`string`522 start : str 524 523 The start date of the retrieval job. 525 524 526 end : :obj:`string`525 end : str 527 526 The end date of the retrieval job. 528 527 … … 555 554 Parameters 556 555 ---------- 557 maxstep : :obj:`string`556 maxstep : str 558 557 The maximum forecast time step in hours from the forecast base time. 559 558 This is the maximum step for non flux (accumulated) forecast data. 560 559 561 steps : :obj:`string`560 steps : str 562 561 Specifies the forecast time step from forecast base time. 563 562 Valid values are hours (HH) from forecast base time. … … 565 564 Return 566 565 ------ 567 maxstep : :obj:`integer`566 maxstep : int 568 567 The maximum forecast time step in hours from the forecast base time. 569 568 This is the maximum step for non flux (accumulated) forecast data. … … 588 587 Parameters 589 588 ---------- 590 basetime : :obj:``589 basetime : str 591 590 The time for a half day retrieval. The 12 hours upfront are to be 592 591 retrieved. … … 607 606 Parameters 608 607 ---------- 609 request : :obj:`integer`608 request : int 610 609 Selects the mode of retrieval. 611 610 0: Retrieves the data from ECMWF. … … 613 612 2: Retrieves the data and prints the mars request. 614 613 615 marsfile : :obj:`string`614 marsfile : str 616 615 Path to the mars request file. 617 616 … … 631 630 Parameters 632 631 ---------- 633 public : :obj:`ìnteger`632 public : int 634 633 Specifies if public data are to be retrieved or not. 635 634 636 dataset : :obj:`string`635 dataset : str 637 636 Specific name which identifies the public dataset. 638 637 … … 653 652 Parameters 654 653 ---------- 655 acctype : :obj:`string`654 acctype : str 656 655 The field type for the accumulated forecast fields. 657 656 658 ftype : :obj:`list` of :obj:`string`657 ftype : list of str 659 658 List of field types. 660 659 661 660 Return 662 661 ------ 663 acctype : :obj:`string`662 acctype : str 664 663 The field type for the accumulated forecast fields. 665 664 ''' … … 690 689 Parameters 691 690 ---------- 692 acctime : :obj:`string`691 acctime : str 693 692 The starting time from the accumulated forecasts. 694 693 695 acctype : :obj:`string`694 acctype : str 696 695 The field type for the accumulated forecast fields. 697 696 698 purefc : :obj:`integer`697 purefc : int 699 698 Switch for definition of pure forecast mode or not. 700 699 701 700 Return 702 701 ------ 703 acctime : :obj:`string`702 acctime : str 704 703 The starting time from the accumulated forecasts. 705 704 ''' 706 705 if not acctime: 707 706 print('... Control parameter ACCTIME was not defined.') 708 print('... Value will be set depending on field type: '707 print('... Value will be set depending on field type:\n ' 709 708 '\t\t EA=06/18\n\t\t EI/OD=00/12\n\t\t EP=18') 710 709 if acctype.upper() == 'EA': # Era 5 … … 726 725 Parameters 727 726 ---------- 728 accmaxstep : :obj:`string`727 accmaxstep : str 729 728 The maximum forecast step for the accumulated forecast fields. 730 729 731 acctype : :obj:`string`730 acctype : str 732 731 The field type for the accumulated forecast fields. 733 732 734 purefc : :obj:`integer`733 purefc : int 735 734 Switch for definition of pure forecast mode or not. 736 735 737 maxstep : :obj:`string`736 maxstep : str 738 737 The maximum forecast time step in hours from the forecast base time. 739 738 This is the maximum step for non flux (accumulated) forecast data. … … 741 740 Return 742 741 ------ 743 accmaxstep : :obj:`string`742 accmaxstep : str 744 743 The maximum forecast step for the accumulated forecast fields. 745 744 ''' … … 776 775 Parameters 777 776 ---------- 778 addpar : :obj:`string` or :obj:'list' of :obj:'string'777 addpar : str or list of str 779 778 List of additional parameters to be retrieved. 780 779 781 780 Return 782 781 ------ 783 addpar : :obj:'string'782 addpar : str 784 783 List of additional parameters to be retrieved. 785 784 ''' … … 798 797 799 798 def check_job_chunk(job_chunk): 800 '''Checks that the job chunknumber is positive and non zero.801 802 Parameters 803 ---------- 804 job_chunk : :obj:`integer`799 '''Checks that if job chunk is set, the number is positive and non zero. 800 801 Parameters 802 ---------- 803 job_chunk : int 805 804 The number of days for a single job script. 806 805 807 806 Return 808 807 ------ 809 job_chunk : :obj:`integer`808 job_chunk : int 810 809 The number of days for a single job script. 811 810 ''' 811 if not job_chunk: 812 return job_chunk 813 812 814 if job_chunk < 0: 813 815 raise ValueError('ERROR: The number of job chunk is negative!\n' -
source/python/mods/disaggregation.py
r0e2f93e r6f951ca 17 17 # - outsourced the disaggregation functions dapoly and darain 18 18 # to a new module named disaggregation 19 # - added the new disaggregation method for precipitation 19 20 # 20 21 # @License: 21 # (C) Copyright 2015-2018. 22 # 23 # This software is licensed under the terms of the Apache Licence Version 2.0 24 # which can be obtained at http://www.apache.org/licenses/LICENSE-2.0. 25 # 26 # @Module Description: 27 # disaggregation of deaccumulated flux data from an ECMWF model FG field. 28 # Initially the flux data to be concerned are: 29 # - large-scale precipitation 30 # - convective precipitation 31 # - surface sensible heat flux 32 # - surface solar radiation 33 # - u stress 34 # - v stress 35 # Different versions of disaggregation is provided for rainfall 36 # data (darain, modified linear) and the surface fluxes and 37 # stress data (dapoly, cubic polynomial). 38 # 39 # @Module Content: 22 # (C) Copyright 2014-2019. 23 # Anne Philipp, Leopold Haimberger 24 # 25 # This work is licensed under the Creative Commons Attribution 4.0 26 # International License. To view a copy of this license, visit 27 # http://creativecommons.org/licenses/by/4.0/ or send a letter to 28 # Creative Commons, PO Box 1866, Mountain View, CA 94042, USA. 29 # 30 # @Methods: 40 31 # - dapoly 41 32 # - darain 42 33 # - IA3 43 #44 34 #******************************************************************************* 35 '''Disaggregation of deaccumulated flux data from an ECMWF model FG field. 36 37 Initially the flux data to be concerned are: 38 - large-scale precipitation 39 - convective precipitation 40 - surface sensible heat flux 41 - surface solar radiation 42 - u stress 43 - v stress 44 45 Different versions of disaggregation is provided for rainfall 46 data (darain, modified linear) and the surface fluxes and 47 stress data (dapoly, cubic polynomial). 48 ''' 45 49 46 50 # ------------------------------------------------------------------------------ … … 65 69 Parameters 66 70 ---------- 67 alist : :obj:`list` of :obj:`array` of :obj:`float`71 alist : list of array of float 68 72 List of 4 timespans as 2-dimensional, horizontal fields. 69 73 E.g. [[array_t1], [array_t2], [array_t3], [array_t4]] … … 71 75 Return 72 76 ------ 73 nfield : :obj:`array` of :obj:`float`77 nfield : array of float 74 78 Interpolated flux at central point of accumulation timespan. 75 79 … … 109 113 Parameters 110 114 ---------- 111 alist : :obj:`list` of :obj:`array` of :obj:`float`115 alist : list of array of float 112 116 List of 4 timespans as 2-dimensional, horizontal fields. 113 117 E.g. [[array_t1], [array_t2], [array_t3], [array_t4]] … … 115 119 Return 116 120 ------ 117 nfield : :obj:`array` of :obj:`float`121 nfield : array of float 118 122 Interpolated flux at central point of accumulation timespan. 119 123 … … 160 164 Note 161 165 ---- 162 Copyright 2017166 (C) Copyright 2017-2019 163 167 Sabine Hittmeir, Anne Philipp, Petra Seibert 164 168 … … 170 174 Parameters 171 175 ---------- 172 g : :obj:`list` of :obj:`float`176 g : list of float 173 177 Complete data series that will be interpolated having 174 178 the dimension of the original raw series. … … 176 180 Return 177 181 ------ 178 f : :obj:`list` of :obj:`float`182 f : list of float 179 183 The interpolated data series with additional subgrid points. 180 184 Its dimension is equal to the length of the input data series -
source/python/mods/get_mars_data.py
r9aefaad r6f951ca 22 22 # online documentation) 23 23 # - use of UIFiles class for file selection and deletion 24 # 24 # - seperated get_mars_data function into several smaller pieces: 25 # write_reqheader, mk_server, mk_dates, remove_old, do_retrievment 25 26 # 26 27 # @License: 27 # (C) Copyright 2014-2018. 28 # 29 # This software is licensed under the terms of the Apache Licence Version 2.0 30 # which can be obtained at http://www.apache.org/licenses/LICENSE-2.0. 31 # 32 # @Program Functionality: 33 # This program can be used as a module in the whole flex_extract process 34 # or can be run by itself to just extract MARS data from ECMWF. To do so, 35 # a couple of necessary parameters has to be passed with the program call. 36 # See documentation for more details. 37 # 38 # @Program Content: 39 # - main 40 # - get_mars_data 41 # - do_retrievement 42 # 28 # (C) Copyright 2014-2019. 29 # Anne Philipp, Leopold Haimberger 30 # 31 # This work is licensed under the Creative Commons Attribution 4.0 32 # International License. To view a copy of this license, visit 33 # http://creativecommons.org/licenses/by/4.0/ or send a letter to 34 # Creative Commons, PO Box 1866, Mountain View, CA 94042, USA. 43 35 #******************************************************************************* 44 """ToDo: Name of litte program 45 46 ToDo: Add desccription 47 48 ToDo: Add Conditions 49 50 This script requires that `...` be installed within the Python 51 environment you are running this script in. 36 '''This script extracts MARS data from ECMWF servers. 37 38 At first, the necessary parameters from command line and CONTROL files are 39 extracted. They define the data set to be extracted from MARS. 52 40 53 41 This file can also be imported as a module and contains the following 54 42 functions: 55 43 56 * get_mars_data -57 * do_retrievement -58 44 * main - the main function of the script 59 """ 45 * get_mars_data - overall control of ECMWF data retrievment 46 * write_reqheader - writes the header into the mars_request file 47 * mk_server - creates the server connection to ECMWF servers 48 * mk_dates - defines the start and end date 49 * remove_old - deletes old retrieved grib files 50 * do_retrievement - creates individual retrievals 51 52 Type: get_mars_data.py --help 53 to get information about command line parameters. 54 Read the documentation for usage instructions. 55 ''' 60 56 # ------------------------------------------------------------------------------ 61 57 # MODULES … … 67 63 68 64 # software specific classes and modules from flex_extract 65 # add path to local main python path for flex_extract to get full access 69 66 sys.path.append(os.path.dirname(os.path.abspath( 70 67 inspect.getfile(inspect.currentframe()))) + '/../') … … 120 117 Parameters 121 118 ---------- 122 c : :obj:`ControlFile`119 c : ControlFile 123 120 Contains all the parameters of CONTROL file and 124 121 command line. … … 166 163 Parameters 167 164 ---------- 168 marsfile : :obj:`string`165 marsfile : str 169 166 Path to the mars request file. 170 167 … … 189 186 Parameters 190 187 ---------- 191 c : :obj:`ControlFile`188 c : ControlFile 192 189 Contains all the parameters of CONTROL file and 193 190 command line. … … 195 192 Return 196 193 ------ 197 server : :obj:`ECMWFDataServer` or :obj:`ECMWFService`194 server : ECMWFDataServer or ECMWFService 198 195 Connection to ECMWF server via python interface ECMWF WebAPI. 199 196 … … 230 227 Parameters 231 228 ---------- 232 c : :obj:`ControlFile`229 c : ControlFile 233 230 Contains all the parameters of CONTROL file and 234 231 command line. 235 232 236 fluxes : :obj:`boolean`, optional233 fluxes : boolean, optional 237 234 Decides if the flux parameter settings are stored or 238 235 the rest of the parameter list. … … 241 238 Return 242 239 ------ 243 start : :obj:`datetime`240 start : datetime 244 241 The start date of the retrieving data set. 245 242 246 end : :obj:`datetime`243 end : datetime 247 244 The end date of the retrieving data set. 248 245 249 chunk : :obj:`datetime`246 chunk : datetime 250 247 Time period in days for one single mars retrieval. 251 248 … … 271 268 Parameters 272 269 ---------- 273 pattern : :obj:`string`270 pattern : str 274 271 The sub string pattern which identifies the files to be deleted. 275 272 276 inputdir : :obj:`string`, optional273 inputdir : str, optional 277 274 Path to the directory where the retrieved data is stored. 278 275 … … 295 292 Parameters 296 293 ---------- 297 c : :obj:`ControlFile`294 c : ControlFile 298 295 Contains all the parameters of CONTROL file and 299 296 command line. 300 297 301 server : :obj:`ECMWFService`298 server : ECMWFService or ECMWFDataServer 302 299 The server connection to ECMWF. 303 300 304 start : :obj:`datetime`301 start : datetime 305 302 The start date of the retrieval. 306 303 307 end : :obj:`datetime`304 end : datetime 308 305 The end date of the retrieval. 309 306 310 delta_t : :obj:`datetime`307 delta_t : datetime 311 308 Delta_t + 1 is the maximal time period of a single 312 309 retrieval. 313 310 314 fluxes : :obj:`boolean`, optional311 fluxes : boolean, optional 315 312 Decides if the flux parameters are to be retrieved or 316 313 the rest of the parameter list. -
source/python/mods/prepare_flexpart.py
r9aefaad r6f951ca 22 22 # - added documentation 23 23 # - minor changes in programming style for consistence 24 # - BUG : removed call of clean_up-Function after call of24 # - BUGFIX: removed call of clean_up-Function after call of 25 25 # prepareFlexpart in main since it is already called in 26 26 # prepareFlexpart at the end! … … 29 29 # 30 30 # @License: 31 # (C) Copyright 2014-2018. 31 # (C) Copyright 2014-2019. 32 # Anne Philipp, Leopold Haimberger 32 33 # 33 # This software is licensed under the terms of the Apache Licence Version 2.0 34 # which can be obtained at http://www.apache.org/licenses/LICENSE-2.0. 35 # 36 # @Program Functionality: 37 # This program prepares the final version of the grib files which are 38 # then used by FLEXPART. It converts the bunch of grib files extracted 39 # via get_mars_data by doing for example the necessary conversion to get 40 # consistent grids or the disaggregation of flux data. Finally, the 41 # program combines the data fields in files per available hour with the 42 # naming convention xxYYMMDDHH, where xx should be 2 arbitrary letters 43 # (mostly xx is chosen to be "EN"). 44 # 45 # @Program Content: 46 # - main 47 # - prepare_flexpart 48 # 34 # This work is licensed under the Creative Commons Attribution 4.0 35 # International License. To view a copy of this license, visit 36 # http://creativecommons.org/licenses/by/4.0/ or send a letter to 37 # Creative Commons, PO Box 1866, Mountain View, CA 94042, USA. 49 38 #******************************************************************************* 39 '''This script prepares the final version of the grib files which are 40 then used by FLEXPART. 41 42 It converts the bunch of grib files extracted via get_mars_data before, 43 by doing the necessary conversion to get consistent grids or the 44 disaggregation of flux data. Finally, the data fields are combined 45 in files per available hour with the naming convention xxYYMMDDHH, 46 where xx should be 2 arbitrary letters (mostly xx is chosen to be "EN"). 47 48 This file can also be imported as a module which then contains the following 49 functions: 50 51 * main 52 * prepare_flexpart 53 54 Type: prepare_flexpart.py --help 55 to get information about command line parameters. 56 Read the documentation for usage instructions. 57 ''' 50 58 51 59 # ------------------------------------------------------------------------------ … … 59 67 60 68 # software specific classes and modules from flex_extract 61 69 # add path to local main python path for flex_extract to get full access 62 70 sys.path.append(os.path.dirname(os.path.abspath( 63 71 inspect.getfile(inspect.currentframe()))) + '/../') … … 115 123 Parameters 116 124 ---------- 117 ppid : :obj:`int`125 ppid : int 118 126 Contains the ppid number of the current ECMWF job. It will be None if 119 127 the method was called within this module. 120 128 121 c : :obj:`ControlFile`129 c : ControlFile 122 130 Contains all the parameters of CONTROL file and 123 131 command line. -
source/python/mods/tools.py
rf2616a3 r6f951ca 14 14 # my_error, normal_exit, init128, to_param_id 15 15 # 16 # April 2018 - Anne Philipp (University of Vienna):16 # April - December 2018 - Anne Philipp (University of Vienna): 17 17 # - applied PEP8 style guide 18 18 # - added documentation 19 # - moved all functions from file Flexparttools to this file tools 20 # - added function get_list_as_string 19 # - moved all non class methods from former file Flexparttools in here 21 20 # - seperated args and control interpretation 21 # - added functions get_list_as_string, read_ecenv, send_mail, make_dir, 22 # put_file_to_ecserver, submit_job_to_ecserver, get_informations, 23 # get_dimensions, execute_subprocess, none_or_int, none_or_str 22 24 # 23 25 # @License: 24 # (C) Copyright 2014-2018. 26 # (C) Copyright 2014-2019. 27 # Anne Philipp, Leopold Haimberger 25 28 # 26 # This software is licensed under the terms of the Apache Licence Version 2.0 27 # which can be obtained at http://www.apache.org/licenses/LICENSE-2.0. 29 # This work is licensed under the Creative Commons Attribution 4.0 30 # International License. To view a copy of this license, visit 31 # http://creativecommons.org/licenses/by/4.0/ or send a letter to 32 # Creative Commons, PO Box 1866, Mountain View, CA 94042, USA. 28 33 # 29 # @Modul Description: 30 # This module contains a couple of helpful functions which are 31 # used in different places in flex_extract. 32 # 33 # @Module Content: 34 # - get_cmdline_args 35 # - clean_up 36 # - my_error 37 # - normal_exit 38 # - product 39 # - silent_remove 40 # - init128 41 # - to_param_id 42 # - get_list_as_string 43 # - make_dir 44 # 34 # @Methods: 35 # none_or_str 36 # none_or_int 37 # get_cmdline_args 38 # read_ecenv 39 # clean_up 40 # my_error 41 # send_mail 42 # normal_exit 43 # product 44 # silent_remove 45 # init128 46 # to_param_id 47 # get_list_as_string 48 # make_dir 49 # put_file_to_ecserver 50 # submit_job_to_ecserver 51 # get_informations 52 # get_dimensions 53 # execute_subprocess 45 54 #******************************************************************************* 55 '''This module contains a collection of diverse tasks within flex_extract. 56 ''' 46 57 47 58 # ------------------------------------------------------------------------------ … … 59 70 60 71 # ------------------------------------------------------------------------------ 61 # FUNCTIONS72 # METHODS 62 73 # ------------------------------------------------------------------------------ 63 74 … … 68 79 Parameters 69 80 ---------- 70 value : :obj:`string`81 value : str 71 82 String to be checked for the "None" word. 72 83 … … 87 98 Parameters 88 99 ---------- 89 value : :obj:`string`100 value : str 90 101 String to be checked for the "None" word. 91 102 … … 110 121 Return 111 122 ------ 112 args : :obj:`Namespace`123 args : Namespace 113 124 Contains the commandline arguments from script/program call. 114 125 ''' … … 201 212 Parameters 202 213 ---------- 203 filepath : :obj:`string`214 filepath : str 204 215 Path to file where the ECMWF environment parameters are stored. 205 216 206 217 Return 207 218 ------ 208 envs : :obj:`dictionary`219 envs : dict 209 220 Contains the environment parameter ecuid, ecgid, gateway 210 221 and destination for ECMWF server environments. … … 233 244 Parameters 234 245 ---------- 235 c : :obj:`ControlFile`246 c : ControlFile 236 247 Contains all the parameters of CONTROL file and 237 248 command line. … … 265 276 Parameters 266 277 ---------- 267 user : :obj:`list` of :obj:`string`278 user : list of str 268 279 Contains all email addresses which should be notified. 269 280 It might also contain just the ecmwf user name which wil trigger 270 281 mailing to the associated email address for this user. 271 282 272 message : :obj:`string`, optional283 message : str, optional 273 284 Error message. Default value is "ERROR". 274 285 … … 295 306 Parameters 296 307 ---------- 297 users : :obj:`list` of :obj:`string`308 users : list of str 298 309 Contains all email addresses which should be notified. 299 310 It might also contain just the ecmwf user name which wil trigger 300 311 mailing to the associated email address for this user. 301 312 302 success_mode : :obj:``string`313 success_mode : str 303 314 States the exit mode of the program to put into 304 315 the mail subject line. 305 316 306 message : :obj:`string`, optional317 message : str, optional 307 318 Message for exiting program. Default value is "Done!". 308 319 … … 341 352 Parameters 342 353 ---------- 343 message : :obj:`string`, optional354 message : str, optional 344 355 Message for exiting program. Default value is "Done!". 345 356 … … 374 385 Parameters 375 386 ---------- 376 \*args : :obj:`list` or :obj:`string`387 \*args : list or str 377 388 Positional arguments (arbitrary number). 378 389 379 \*\*kwds : :obj:`dictionary`390 \*\*kwds : dict 380 391 Contains all the keyword arguments from \*args. 381 392 … … 405 416 Parameters 406 417 ---------- 407 filename : :obj:`string`418 filename : str 408 419 The name of the file to be removed without notification. 409 420 … … 429 440 Parameters 430 441 ---------- 431 filepath : :obj:`string`442 filepath : str 432 443 Path to file of ECMWF grib table number 128. 433 444 434 445 Return 435 446 ------ 436 table128 : :obj:`dictionary`447 table128 : dict 437 448 Contains the ECMWF grib table 128 information. 438 449 The key is the parameter number and the value is the … … 462 473 Parameters 463 474 ---------- 464 pars : :obj:`string`475 pars : str 465 476 Addpar argument from CONTROL file in the format of 466 477 parameter names instead of ids. The parameter short … … 468 479 one single string. 469 480 470 table : :obj:`dictionary`481 table : dict 471 482 Contains the ECMWF grib table 128 information. 472 483 The key is the parameter number and the value is the … … 475 486 Return 476 487 ------ 477 ipar : :obj:`list` of :obj:`integer`488 ipar : list of int 478 489 List of addpar parameters from CONTROL file transformed to 479 490 parameter ids in the format of integer. … … 501 512 Parameters 502 513 ---------- 503 list_obj : :obj:`list`514 list_obj : list of * 504 515 A list with arbitrary content. 505 516 506 concatenate_sign : :obj:`string`, optional517 concatenate_sign : str, optional 507 518 A string which is used to concatenate the single 508 519 list elements. Default value is ", ". … … 510 521 Return 511 522 ------ 512 str_of_list : :obj:`string`523 str_of_list : str 513 524 The content of the list as a single string. 514 525 ''' … … 528 539 Parameters 529 540 ---------- 530 directory : :obj:`string`541 directory : str 531 542 The path to directory which should be created. 532 543 … … 557 568 Parameters 558 569 ---------- 559 ecd : :obj:`string`570 ecd : str 560 571 The path were the file is stored. 561 572 562 filename : :obj:`string`573 filename : str 563 574 The name of the file to send to the ECMWF server. 564 575 565 target : :obj:`string`576 target : str 566 577 The target queue where the file should be sent to. 567 578 568 ecuid : :obj:`string`579 ecuid : str 569 580 The user id on ECMWF server. 570 581 571 ecgid : :obj:`string`582 ecgid : str 572 583 The group id on ECMWF server. 573 584 … … 610 621 Parameters 611 622 ---------- 612 target : :obj:`string`623 target : str 613 624 The target where the file should be sent to, e.g. the queue. 614 625 615 jobname : :obj:`string`626 jobname : str 616 627 The name of the jobfile to be submitted to the ECMWF server. 617 628 618 629 Return 619 630 ------ 620 job_id : :obj:`int`631 job_id : int 621 632 The id number of the job as a reference at the ecmwf server. 622 633 ''' … … 650 661 Parameters 651 662 ---------- 652 filename : :obj:`string`663 filename : str 653 664 Name of the file which will be opened to extract basic information. 654 665 655 666 Return 656 667 ------ 657 data : :obj:`dictionary`668 data : dict 658 669 Contains basic informations of the ECMWF grib files, e.g. 659 670 'Ni', 'Nj', 'latitudeOfFirstGridPointInDegrees', … … 702 713 Parameters 703 714 ---------- 704 info : :obj:`dictionary`715 info : dict 705 716 Contains basic informations of the ECMWF grib files, e.g. 706 717 'Ni', 'Nj', 'latitudeOfFirstGridPointInDegrees', … … 709 720 'iDirectionIncrementInDegrees', 'missingValue' 710 721 711 purefc : :obj:`integer`722 purefc : int 712 723 Switch for definition of pure forecast mode or not. 713 724 714 dtime : :obj:`string`725 dtime : str 715 726 Time step in hours. 716 727 717 index_vals : :obj:`list`728 index_vals : list of list of str 718 729 Contains the values from the keys used for a distinct selection 719 730 of grib messages in processing the grib files. … … 723 734 index_vals[2]: ('0', '12', '3', '6', '9') ; stepRange 724 735 725 start_date : :obj:`string`736 start_date : str 726 737 The start date of the retrieval job. 727 738 728 end_date : :obj:`string`739 end_date : str 729 740 The end date of the retrieval job. 730 741 731 742 Return 732 743 ------ 733 (ix, jy, it) : :obj:`tuple` of :obj:`integer`744 (ix, jy, it) : tuple of int 734 745 Dimension in x-direction, y-direction and in time. 735 746 ''' … … 755 766 Parameters 756 767 ---------- 757 cmd_list : :obj:`list` of `:obj:`string`768 cmd_list : list of str 758 769 A list of the components for the command line execution. Each 759 770 list entry is a single part of the command which is seperated from … … 763 774 Return 764 775 ------ 765 error_msg : :obj:`string`, optional776 error_msg : str, optional 766 777 The possible error message if the subprocess failed. 767 778 By default it will just tell "SUBPROCESS FAILED!".
Note: See TracChangeset
for help on using the changeset viewer.
