Changeset 8209738 in flex_extract.git


Ignore:
Timestamp:
May 7, 2020, 3:32:03 PM (4 years ago)
Author:
anphi <anne.philipp@…>
Branches:
master, ctbto, dev
Children:
b936fd3
Parents:
05e9362
Message:

language corrections in comment sections and print commands

Location:
Source/Python
Files:
5 edited

Legend:

Unmodified
Added
Removed
  • Source/Python/Mods/checks.py

    r50f9ca6 r8209738  
    119119    '''Defines the correct area string.
    120120
    121     Checks on the format of the four area components. Wether it is of
    122     the order of 1000 or 1. Also checks wether area was already set by command
    123     line, then the four components are overwritten.
     121    Checks the format of the four area components wether it is on
     122    the order of 1000 or 1.
     123    Also checks wether area was already set on command line,
     124    then the four components are overwritten.
    124125    Convert to correct format of the order of magnitude "1" and sets the
    125126    area parameter (North/West/South/East).
     
    135136
    136137    upper : str
    137         The northern most latitude.
     138        The northernmost latitude.
    138139
    139140    lower : str
    140         The souther most latitude.
     141        The southernmost latitude.
    141142
    142143    left : str
    143         The western most longitude.
     144        The westernmost longitude.
    144145
    145146    right : str
    146         The eastern most longiude.
     147        The easternmost longitude.
    147148
    148149    Return
    149150    ------
    150151    grid : str
    151         Contains grid in format Lat/lon. E.g. 0.1/0.1
     152        Contains grid in format lat/lon. E.g. 0.1/0.1
    152153    '''
    153154    if 'N' in grid:  # Gaussian output grid
     
    180181                                    float(right))
    181182    else:
    182         raise ValueError('The area components have different '
     183        raise ValueError('Area components have inconsisten or unrecognised '
    183184                         'formats (upper, lower, left, right): '
    184185                         '{}/{}/{}/{}'.format(str(upper), str(lower),
     
    209210    level : str
    210211        Specifies the maximum level. It has to be one of the
    211         available maximum level number as contained in the variable
    212         MAX_LEVEL_LIST in "_config". E.g. [16, 19, 31, 40, 50, 60, 62, 91, 137]
     212        available maximum level numbers as contained in the variable
     213        MAX_LEVEL_LIST in "_config": [16, 19, 31, 40, 50, 60, 62, 91, 137]
    213214
    214215    '''
     
    346347    for i, val in enumerate(ftype):
    347348        if ftype[i] == 'AN' and int(steps[i]) != 0:
    348             print('Analysis retrievals must have STEP = 0 (now set to 0)')
     349            print('For analysis data, STEP = 0 is needed. Setting to 0 now.)')
    349350            ftype[i] = 0
    350351
     
    386387    maxstep : int
    387388        The maximum forecast time step in hours from the forecast base time.
    388         This is the maximum step for non flux (accumulated) forecast data.
     389        This is the maximum step for non-flux (not accumulated) forecast data.
    389390
    390391    purefc : int
     
    405406    if not len(ftype) == len(ftime) == len(steps):
    406407        raise ValueError('ERROR: The number of field types, times and steps '
    407                          'are not the same! Please check the setting in the '
     408                         'are not the same! Please check the settings in the '
    408409                         'CONTROL file!')
    409410
     
    423424
    424425def check_mail(mail):
    425     '''Check the string of mail addresses, seperate them and convert to a list.
     426    '''Check the string of mail addresses, separate them and convert to list.
    426427
    427428    Parameters
     
    529530
    530531def check_dates(start, end):
    531     '''Checks if there is at least a start date for a one day retrieval.
     532    '''Checks if there is at least a start date for a one-day retrieval.
    532533
    533534    Checks if end date lies after start date and end date is set.
     
    628629
    629630def check_request(request, marsfile):
    630     '''Check if there is an old mars request file and remove it.
     631    '''Check if there is an old MARS request file; if so, remove it.
    631632
    632633    Parameters
     
    651652
    652653def check_public(public, dataset):
    653     '''Check wether the dataset parameter is set for a
    654     public data set retrieval.
     654    '''Check wether the dataset parameter is set to a
     655    public data set.
    655656
    656657    Parameters
     
    667668    '''
    668669    if public and not dataset:
    669         raise ValueError('ERROR: If public mars data wants to be retrieved, '
    670                          'the "dataset"-parameter has to be set too!')
     670        raise ValueError('ERROR: If public MARS data are to be retrieved, '
     671                         'the "dataset"-parameter has to be set, too!')
    671672    return
    672673
     
    674675    '''Guarantees that the accumulation field type is set.
    675676
    676     If not set, it is derivated as in the old method (TYPE[1]).
     677    If not set, it is derived with the old method (TYPE[1]).
    677678
    678679    Parameters
     
    710711    '''Guarantees that the accumulation forecast times were set.
    711712
    712     If it is not set, it tries to set the value for some of the
    713     most commonly used data sets. Otherwise it raises an error.
     713    If not set, setting the value to some of the most commonly used data sets
     714    is attempted. Otherwise, an eror is raised.
    714715
    715716    Parameters
    716717    ----------
    717718    acctime : str
    718         The starting time from the accumulated forecasts.
     719        The starting time for the accumulated forecasts.
    719720
    720721    marsclass : str
     
    727728    ------
    728729    acctime : str
    729         The starting time from the accumulated forecasts.
     730        The starting time for the accumulated forecasts.
    730731    '''
    731732
    732733    if not acctime:
    733         print('... Control parameter ACCTIME was not defined.')
     734        print('... Control parameter ACCTIME was not set.')
    734735        print('... Value will be set depending on field type:\n '
    735736              '\t\t EA=06/18\n\t\t EI/OD=00/12\n\t\t EP=18')
     
    745746            acctime = time[0]
    746747        else:
    747             raise ValueError('ERROR: Accumulation forecast time can not '
    748                              'automatically be derived!')
     748            raise ValueError('ERROR: Accumulation forecast time can not be'
     749                             'derived automatically!')
    749750    return acctime
    750751
    751752def check_accmaxstep(accmaxstep, marsclass, purefc, maxstep):
    752     '''Guarantees that the accumulation forecast step were set.
     753    '''Guarantees that the accumulation forecast step was set.
    753754
    754755    Parameters
     
    765766    maxstep : str
    766767        The maximum forecast time step in hours from the forecast base time.
    767         This is the maximum step for non flux (accumulated) forecast data.
     768        This is the maximum step for non-flux (accumulated) forecast data.
    768769
    769770    Return
     
    773774    '''
    774775    if not accmaxstep:
    775         print('... Control parameter ACCMAXSTEP was not defined.')
     776        print('... Control parameter ACCMAXSTEP was not set.')
    776777        print('... Value will be set depending on field type/time: '
    777778              '\n\t\t EA/EI/OD=12\n\t\t EP=24')
     
    785786            print('... For pure forecast mode, the accumulated forecast must '
    786787                  'have the same maxstep as the normal forecast fields!\n'
    787                   '\t\t Accmaxstep was set to maxstep!')
     788                  '\t\t ACCMAXSTEP was set to MAXSTEP!')
    788789        else:
    789             raise ValueError('ERROR: Accumulation forecast step can not '
    790                              'automatically be derived!')
     790            raise ValueError('ERROR: Accumulation forecast step can not be'
     791                             'derived automatically!')
    791792    else:
    792793        if purefc and int(accmaxstep) != int(maxstep):
     
    794795            print('... For pure forecast mode, the accumulated forecast must '
    795796                  'have the same maxstep as the normal forecast fields!\n'
    796                   '\t\t Accmaxstep was set to maxstep!')
     797                  '\t\t ACCMAXSTEP was set to MAXSTEP!')
    797798    return accmaxstep
    798799
    799800def check_addpar(addpar):
    800801    '''Check that addpar has correct format of additional parameters in
    801     a single string, so that it can be easily appended to the hard coded
    802     parameters that are retrieved in any case.
     802    a single string, so that it can be easily appended to the hard-coded
     803    parameters retrieved in any case.
    803804
    804805    Parameters
     
    826827
    827828def check_job_chunk(job_chunk):
    828     '''Checks that if job chunk is set, the number is positive and non zero.
     829    '''Checks that if job chunk is set, the number is positive and nonzero.
    829830
    830831    Parameters
     
    844845
    845846    if job_chunk < 0:
    846         raise ValueError('ERROR: The number of job chunk is negative!\n'
     847        raise ValueError('ERROR: The number of job chunks is negative!\n'
    847848                         'It has to be a positive number!')
    848849    elif job_chunk == 0:
  • Source/Python/Mods/get_mars_data.py

    r0f89116 r8209738  
    1010#    November 2015 - Leopold Haimberger (University of Vienna):
    1111#        - moved the getEIdata program into a function "get_mars_data"
    12 #        - moved the AgurmentParser into a seperate function
    13 #        - adatpted the function for the use in flex_extract
    14 #        - renamed file to get_mars_data
     12#        - moved the AgurmentParser into a separate function
     13#        - adapted the function for use in flex_extract
     14#        - renamed source file to get_mars_data
    1515#
    1616#    February 2018 - Anne Philipp (University of Vienna):
     
    2222#          online documentation)
    2323#        - use of UIFiles class for file selection and deletion
    24 #        - seperated get_mars_data function into several smaller pieces:
     24#        - separated get_mars_data function into several smaller pieces:
    2525#          write_reqheader, mk_server, mk_dates, remove_old, do_retrievment
    2626#
     
    3636#    Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.
    3737#*******************************************************************************
    38 '''This script extracts MARS data from ECMWF servers.
     38'''This script extracts MARS data from ECMWF.
    3939
    4040At first, the necessary parameters from command line and CONTROL files are
     
    4444functions:
    4545
    46     * main - the main function of the script
    47     * get_mars_data - overall control of ECMWF data retrievment
     46    * main            - the main function of the script
     47    * get_mars_data   - overall control of ECMWF data retrievment
    4848    * write_reqheader - writes the header into the mars_request file
    49     * mk_server - creates the server connection to ECMWF servers
    50     * mk_dates - defines the start and end date
    51     * remove_old - deletes old retrieved grib files
    52     * do_retrievement - creates individual retrievals
    53 
    54 Type: get_mars_data.py --help
     49    * mk_server       - creates the server connection to ECMWF servers
     50    * mk_dates        - defines the start and end date
     51    * remove_old      - deletes old retrieved grib files
     52    * do_retrieval    - creates individual retrievals
     53
     54Type get_mars_data.py --help
    5555to get information about command line parameters.
    5656Read the documentation for usage instructions.
     
    6666from datetime import datetime, timedelta
    6767
    68 # software specific classes and modules from flex_extract
    69 # add path to local main python path for flex_extract to get full access
     68# software-specific classes and modules from flex_extract
     69# add path to local main Python path for flex_extract to get full access
    7070sys.path.append(os.path.dirname(os.path.abspath(
    7171    inspect.getfile(inspect.currentframe()))) + '/../')
     
    9494# ------------------------------------------------------------------------------
    9595def main():
    96     '''Controls the program to get data out of mars.
    97 
    98     This is done if it is called directly from command line.
    99     Then it also takes program call arguments and control file input.
     96    '''Controls the program to retrieve data from MARS.
     97
     98    This is done if called directly from command line.
     99    Then, arguments and control file are taken as input.
    100100
    101101    Parameters
     
    114114
    115115def get_mars_data(c):
    116     '''Retrieves the EC data needed for a FLEXPART simulation.
    117 
    118     Start and end dates for retrieval period is set. Retrievals
    119     are divided into smaller periods if necessary and datechunk parameter
     116    '''Retrieves the ECMWF data required for a FLEXPART simulation.
     117
     118    Start and end dates for retrieval period are set. Retrievals
     119    are divided into shorter periods if necessary and if datechunk parameter
    120120    is set.
    121121
     
    137137
    138138    if c.request == 0:
    139         print("Retrieving EC data!")
     139        print("Retrieving ECMWF data!")
    140140    else:
    141141        if c.request == 1:
    142             print("Printing mars requests!")
     142            print("Printing MARS requests!")
    143143        elif c.request == 2:
    144             print("Retrieving EC data and printing mars request!")
     144            print("Retrieving ECMWF data and printing MARS request!")
    145145        write_reqheader(os.path.join(c.inputdir, _config.FILE_MARS_REQUESTS))
    146146
     
    165165
    166166def write_reqheader(marsfile):
    167     '''Writes header with column names into mars request file.
     167    '''Writes header with column names into MARS request file.
    168168
    169169    Parameters
    170170    ----------
    171171    marsfile : str
    172         Path to the mars request file.
     172        Path to the MARS request file.
    173173
    174174    Return
     
    188188
    189189def mk_server(c):
    190     '''Creates a server connection with available python API.
    191 
    192     Which API is used depends on availability and the dataset to be retrieved.
    193     The CDS API is used for ERA5 dataset no matter if the user is a member or
    194     a public user. ECMWF WebAPI is used for all other available datasets.
     190    '''Creates a server connection with available Python API.
     191
     192    The API selected depends on availability and the data set to be retrieved.
     193    The CDS API is used for ERA5 data, no matter whether the user is a
     194    member-state or a public user.
     195    ECMWF WebAPI is used for all other available datasets.
    195196
    196197    Parameters
     
    247248
    248249def mk_dates(c, fluxes):
    249     '''Prepares start and end date depending on flux or non flux data.
    250 
    251     If forecast for maximum one day (upto 24h) are to be retrieved, then
     250    '''Prepares start and end date depending on flux or non-flux type of data.
     251
     252    If forecasts for a maximum of one day (24 h) are to be retrieved, then
    252253    collect accumulation data (flux data) with additional days in the
    253     beginning and at the end (used for complete disaggregation of
     254    beginning and at the end (needed for complete disaggregation of
    254255    original period)
    255256
    256     If forecast data longer than 24h are to be retrieved, then
     257    If forecast data for more than +24 h are to be retrieved, then
    257258    collect accumulation data (flux data) with the exact start and end date
    258259    (disaggregation will be done for the exact time period with
     
    314315    ----------
    315316    pattern : str
    316         The sub string pattern which identifies the files to be deleted.
     317        The substring pattern which identifies the files to be deleted.
    317318
    318319    inputdir : str, optional
    319         Path to the directory where the retrieved data is stored.
    320 
    321     Return
    322     ------
    323 
    324     '''
    325     print('... removing old content of ' + inputdir)
     320        Path to the directory where the retrieved data are stored.
     321
     322    Return
     323    ------
     324
     325    '''
     326    print('... removing old files in ' + inputdir)
    326327
    327328    tobecleaned = UioFiles(inputdir, pattern)
     
    332333
    333334def do_retrievement(c, server, start, end, delta_t, fluxes=False):
    334     '''Divides the complete retrieval period in smaller chunks and
     335    '''Divides the total retrieval period into smaller chunks and
    335336    retrieves the data from MARS.
    336337
     
    351352
    352353    delta_t : datetime
    353         Delta_t + 1 is the maximal time period of a single
    354         retrieval.
     354        Delta_t + 1 is the maximum time period of a single retrieval.
    355355
    356356    fluxes : boolean, optional
     
    365365
    366366    # since actual day also counts as one day,
    367     # we only need to add datechunk - 1 days to retrieval
    368     # for a period
     367    # we only need to add datechunk - 1 days to retrieval for a period
    369368    delta_t_m1 = delta_t - timedelta(days=1)
    370369
  • Source/Python/Mods/prepare_flexpart.py

    r0f89116 r8209738  
    1313#        - job templates suitable for twice daily operational dissemination
    1414#        - dividing retrievals of longer periods into digestable chunks
    15 #        - retrieve also longer term forecasts, not only analyses and
    16 #          short term forecast data
     15#        - retrieve also longer-term forecasts, not only analyses and
     16#          short-term forecast data
    1717#        - conversion into GRIB2
    18 #        - conversion into .fp format for faster execution of FLEXPART
     18#       
    1919#
    2020#    February 2018 - Anne Philipp (University of Vienna):
     
    4545
    4646It converts the bunch of grib files extracted via get_mars_data before,
    47 by doing the necessary conversion to get consistent grids or the
     47by doing the necessary conversion to obtain consistent grids or the
    4848disaggregation of flux data. Finally, the data fields are combined
    49 in files per available hour with the naming convention xxYYMMDDHH,
     49in files per hour available with the naming convention xxYYMMDDHH,
    5050where xx should be 2 arbitrary letters (mostly xx is chosen to be "EN").
    5151
     
    8888# ------------------------------------------------------------------------------
    8989def main():
    90     '''Controls the program to prepare flexpart input files from mars data.
     90    '''Controls the program to prepare FLEXPART input files from MARS data.
    9191
    92     This is done if it is called directly from command line.
    93     Then it also takes program call arguments and control file input.
     92    This is done if called directly from the command line.
     93    Then, arguments and control file are taken as input.
    9494
    9595    Parameters
     
    108108
    109109def prepare_flexpart(ppid, c):
    110     '''Converts the mars data into flexpart ready input files.
     110    '''Converts the MARS data into files ready as input for FLEXPART.
    111111
    112     Specific data fields are converted to a different grid and the flux
    113     data are going to be disaggregated. The data fields are collected by
    114     hour and stored in a file with a specific FLEXPART relevant naming
    115     convention.
     112    Certain fields are converted to a different grid and the flux
     113    data are disaggregated. Fields are collected by hour and stored in a file
     114    with a specific naming convention.
    116115
    117116    Parameters
  • Source/Python/Mods/tools.py

    rd1bfa24 r8209738  
    1818#        - added documentation
    1919#        - moved all non class methods from former file Flexparttools in here
    20 #        - seperated args and control interpretation
     20#        - separated args and control interpretation
    2121#        - added functions get_list_as_string, read_ecenv, send_mail, make_dir,
    2222#          put_file_to_ecserver, submit_job_to_ecserver, get_informations,
     
    118118
    119119def none_or_str(value):
    120     '''Converts the input string into pythons None-type if the string
    121     contains string "None".
     120    '''Converts the input string into Pythons None type if it
     121    contains the string "None".
    122122
    123123    Parameters
     
    130130    None or value:
    131131        Return depends on the content of the input value. If it was "None",
    132         then the python type None is returned. Otherwise the string itself.
     132        then the Python type None is returned, otherwise the string itself.
    133133    '''
    134134    if value == 'None':
     
    137137
    138138def none_or_int(value):
    139     '''Converts the input string into pythons None-type if the string
    140     contains string "None". Otherwise it is converted to an integer value.
     139    '''Converts the input string into Pythons None-type if it
     140    contains string "None"; otherwise it is converted to an integer value.
    141141
    142142    Parameters
     
    158158def get_cmdline_args():
    159159    '''Decomposes the command line arguments and assigns them to variables.
    160     Apply default values for non mentioned arguments.
     160    Apply default values for arguments not present.
    161161
    162162    Parameters
     
    166166    ------
    167167    args : Namespace
    168         Contains the commandline arguments from script/program call.
     168        Contains the command line arguments from the script / program call.
    169169    '''
    170170
     
    191191    parser.add_argument("--basetime", dest="basetime",
    192192                        type=none_or_int, default=None,
    193                         help="base such as 0 or 12 (for half day retrievals)")
     193                        help="base time such as 0 or 12 (for half day retrievals)")
    194194    parser.add_argument("--step", dest="step",
    195195                        type=none_or_str, default=None,
     
    200200    parser.add_argument("--area", dest="area",
    201201                        type=none_or_str, default=None,
    202                         help="area defined as north/west/south/east")
     202                        help="area, defined by north/west/south/east")
    203203
    204204    # some switches
    205205    parser.add_argument("--debug", dest="debug",
    206206                        type=none_or_int, default=None,
    207                         help="debug mode - leave temporary files intact")
     207                        help="debug mode - temporary files will be conserved")
    208208    parser.add_argument("--oper", dest="oper",
    209209                        type=none_or_int, default=None,
    210                         help='operational mode - prepares dates with '
     210                        help='operational mode - prepares dates from '
    211211                        'environment variables')
    212212    parser.add_argument("--request", dest="request",
    213213                        type=none_or_int, default=None,
    214                         help="list all mars requests in file mars_requests.dat")
     214                        help="list all MARS requests in file mars_requests.dat")
    215215    parser.add_argument("--public", dest="public",
    216216                        type=none_or_int, default=None,
    217                         help="public mode - retrieves the public datasets")
     217                        help="public mode - retrieves public datasets")
    218218    parser.add_argument("--rrint", dest="rrint",
    219219                        type=none_or_int, default=None,
    220                         help='Selection of old or new precipitation '
    221                         'interpolation:\n'
     220                        help='Selection of old or new '
     221                        'interpolation method for precipitation:\n'
    222222                        '     0 - old method\n'
    223223                        '     1 - new method (additional subgrid points)')
     
    226226    parser.add_argument("--inputdir", dest="inputdir",
    227227                        type=none_or_str, default=None,
    228                         help='Path to the temporary directory for the '
    229                         'retrieval grib files and other processing files.')
     228                        help='Path to temporary directory for '
     229                        'retrieved grib files and other processing files.')
    230230    parser.add_argument("--outputdir", dest="outputdir",
    231231                        type=none_or_str, default=None,
    232                         help='Path to the final directory where the final '
    233                         'FLEXPART ready input files are stored.')
     232                        help='Path to final directory where '
     233                        'FLEXPART input files will be stored.')
    234234
    235235    # this is only used by prepare_flexpart.py to rerun a postprocessing step
    236236    parser.add_argument("--ppid", dest="ppid",
    237237                        type=none_or_str, default=None,
    238                         help='This is the specify parent process id of a '
     238                        help='This is the specify the parent process id of a '
    239239                        'single flex_extract run to identify the files. '
    240240                        'It is the second number in the GRIB files.')
     
    243243    parser.add_argument("--job_template", dest='job_template',
    244244                        type=none_or_str, default="job.temp",
    245                         help='The job template file which are adapted to be '
    246                         'submitted to the batch system on ECMWF server.')
     245                        help='Job template file. Will be used for submission '
     246                        'to the batch system on the ECMWF server after '
     247                        'modification.')
    247248    parser.add_argument("--queue", dest="queue",
    248249                        type=none_or_str, default=None,
    249                         help='The ECMWF server name for submission of the '
    250                         'job script to the batch system '
     250                        help='The name of the ECMWF server name where the'
     251                        'job script is to be submitted '
    251252                        '(e.g. ecgate | cca | ccb)')
    252253
     
    289290
    290291    It keeps the final FLEXPART input files if program runs without
    291     ECMWF Api and keywords "ectrans" or "ecstorage" are set to "1".
     292    ECMWF API and keywords "ectrans" or "ecstorage" are set to "1".
    292293
    293294    Parameters
     
    410411
    411412    This method combines the single characters of the passed arguments
    412     with each other. So that each character of each argument value
     413    with each other in a way that each character of each argument value
    413414    will be combined with each character of the other arguments as a tuple.
    414415
     
    519520    pars : str
    520521        Addpar argument from CONTROL file in the format of
    521         parameter names instead of ids. The parameter short
    522         names are sepearted with "/" and they are passed as
    523         one single string.
     522        parameter names instead of IDs. The parameter short
     523        names are separated by "/" and passed as one single string.
    524524
    525525    table : dict
     
    555555
    556556def to_param_id_with_tablenumber(pars, table):
    557     '''Transform parameter names to parameter ids and add table id.
     557    '''Transform parameter names to parameter IDs and add table ID.
    558558
    559559    Conversion with ECMWF grib table 128.
     
    563563    pars : str
    564564        Addpar argument from CONTROL file in the format of
    565         parameter names instead of ids. The parameter short
    566         names are sepearted with "/" and they are passed as
    567         one single string.
     565        parameter names instead of ID. The parameter short
     566        names are separated by "/" and passed as one single string.
    568567
    569568    table : dict
     
    576575    spar : str
    577576        List of addpar parameters from CONTROL file transformed to
    578         parameter ids in the format of integer.
     577        parameter IDs in the format of integer.
    579578    '''
    580579    if not pars:
     
    598597
    599598def get_list_as_string(list_obj, concatenate_sign=', '):
    600     '''Converts a list of arbitrary content into a single string.
     599    '''Converts a list of arbitrary content into a single string using a given
     600    concatenation character.
    601601
    602602    Parameters
     
    624624    '''Creates a directory.
    625625
    626     It gives a warning if the directory already exists and skips process.
    627     The program stops only if there is another problem.
     626    If the directory already exists, an information is printed and the creation
     627    skipped. The program stops only if there is another problem.
    628628
    629629    Parameters
     
    641641        # errno.EEXIST = directory already exists
    642642        if e.errno == errno.EEXIST:
    643             print('WARNING: Directory {0} already exists!'.format(directory))
     643            print('INFORMATION: Directory {0} already exists!'.format(directory))
    644644        else:
    645645            raise # re-raise exception if a different error occured
     
    720720    ------
    721721    job_id : int
    722         The id number of the job as a reference at the ecmwf server.
     722        The id number of the job as a reference at the ECMWF server.
    723723    '''
    724724
     
    732732
    733733        print('\n... Do you have a valid ecaccess certification key?')
    734         sys.exit('... ECACCESS-JOB-SUBMIT FAILED!')
     734        sys.exit('... ecaccess-job-submit FAILED!')
    735735    except OSError as e:
    736736        print('... ERROR CODE: ' + str(e.errno))
     
    738738
    739739        print('\n... Most likely the ECACCESS library is not available!')
    740         sys.exit('... ECACCESS-JOB-SUBMIT FAILED!')
     740        sys.exit('... ecaccess-job-submit FAILED!')
    741741
    742742    return job_id.decode()
     
    744744
    745745def get_informations(filename):
    746     '''Gets basic information from an example grib file.
    747 
    748     These information are important for later use and the
    749     initialization of numpy arrays for data storing.
     746    '''Extracts basic information from a sample grib file.
     747
     748    This information is needed for later use and the
     749    initialization of numpy arrays where data are stored.
    750750
    751751    Parameters
     
    768768
    769769    # --- open file ---
    770     print("Opening file for getting information data --- %s" % filename)
     770    print("Opening grib file for extraction of information --- %s" % filename)
    771771    with open(filename, 'rb') as f:
    772772        # load first message from file
     
    785785               ]
    786786
    787         print('\nInformations are: ')
     787        print('\nInformation extracted: ')
    788788        for key in keys:
    789789            # Get the value of the key in a grib message.
     
    798798
    799799def get_dimensions(info, purefc, dtime, index_vals, start_date, end_date):
    800     '''This function specifies the correct dimensions for x, y and t.
     800    '''This function specifies the correct dimensions for x, y, and t.
    801801
    802802    Parameters
     
    817817    index_vals : list of list of str
    818818        Contains the values from the keys used for a distinct selection
    819         of grib messages in processing the grib files.
     819        of GRIB messages in processing the grib files.
    820820        Content looks like e.g.:
    821821        index_vals[0]: ('20171106', '20171107', '20171108') ; date
     
    849849
    850850def execute_subprocess(cmd_list, error_msg='SUBPROCESS FAILED!'):
    851     '''Executes a command line instruction via a subprocess.
     851    '''Executes a command via a subprocess.
    852852
    853853    Error handling is done if an error occures.
     
    856856    ----------
    857857    cmd_list : list of str
    858         A list of the components for the command line execution. Each
    859         list entry is a single part of the command which is seperated from
    860         the rest by a blank space.
    861         E.g. ['mv', file1, file2]
     858        A list of the components for the command line execution.
     859        They will be concatenated with blank space for the command
     860        to be submitted, like ['mv', file1, file2] for mv file1 file2.
    862861
    863862    Return
    864863    ------
    865864    error_msg : str, optional
    866         The possible error message if the subprocess failed.
    867         By default it will just tell "SUBPROCESS FAILED!".
     865        Error message if the subprocess fails.
     866        By default it will just say "SUBPROCESS FAILED!".
    868867    '''
    869868
  • Source/Python/install.py

    r50f9ca6 r8209738  
    4646
    4747Depending on the selected installation environment (locally or on the
    48 ECMWF server ecgate or cca) the program extracts the commandline
     48ECMWF server ecgate or cca) the program extracts the command line
    4949arguments and the CONTROL file parameter and prepares the corresponding
    5050environment.
    51 The necessary files are collected in a tar-ball and placed
    52 at the target location. There its untared, the environment variables will
    53 be set and the Fortran code will be compiled.
    54 If the ECMWF environment is selected a job script is prepared and submitted
    55 for the remaining configurations after putting the tar-ball to the
     51The necessary files are collected in a tar ball and placed
     52at the target location. There, is is untared, the environment variables are
     53set, and the Fortran code is compiled.
     54If the ECMWF environment is selected, a job script is prepared and submitted
     55for the remaining configurations after putting the tar ball on the
    5656target ECMWF server.
    5757
     
    107107def get_install_cmdline_args():
    108108    '''Decomposes the command line arguments and assigns them to variables.
    109     Apply default values for non mentioned arguments.
     109    Apply default values for arguments not present.
    110110
    111111    Parameters
     
    127127    parser.add_argument("--makefile", dest="makefile",
    128128                        type=none_or_str, default=None,
    129                         help='Name of Makefile to use for compiling the '
     129                        help='Name of makefile for compiling the '
    130130                        'Fortran program')
    131131    parser.add_argument("--ecuid", dest="ecuid",
    132132                        type=none_or_str, default=None,
    133                         help='The user id at ECMWF.')
     133                        help='User id at ECMWF')
    134134    parser.add_argument("--ecgid", dest="ecgid",
    135135                        type=none_or_str, default=None,
    136                         help='The group id at ECMWF.')
     136                        help='Group id at ECMWF')
    137137    parser.add_argument("--gateway", dest="gateway",
    138138                        type=none_or_str, default=None,
    139                         help='The name of the local gateway server.')
     139                        help='Name of the local gateway server')
    140140    parser.add_argument("--destination", dest="destination",
    141141                        type=none_or_str, default=None,
    142                         help='The ecaccess association, e.g. '
     142                        help='ecaccess association, e.g. '
    143143                        'myUser@genericSftp')
    144144
    145145    parser.add_argument("--installdir", dest="installdir",
    146146                        type=none_or_str, default=None,
    147                         help='Root directory where '
    148                         'flex_extract will be installed to.')
     147                        help='Root directory of the '
     148                        'flex_extract installation')
    149149
    150150    # arguments for job submission to ECMWF, only needed by submit.py
    151151    parser.add_argument("--job_template", dest='job_template',
    152152                        type=none_or_str, default="job.template",
    153                         help='The rudimentary template file to create a batch '
    154                         'job template for submission to ECMWF servers.')
     153                        help='Rudimentary template file to create a batch '
     154                        'job template for submission to ECMWF servers')
    155155
    156156    parser.add_argument("--controlfile", dest="controlfile",
    157157                        type=none_or_str, default='CONTROL_EA5',
    158                         help="The file with all CONTROL parameters.")
     158                        help="A file that contains all CONTROL parameters.")
    159159
    160160    args = parser.parse_args()
     
    164164
    165165def install_via_gateway(c):
    166     '''Prepare data transfer to remote gate and submit a job script which will
    167     install everything on the remote gate.
     166    '''Prepare data transfer to remote gateway and submit a job script which will
     167    install everything on the remote gateway.
    168168
    169169    Parameters
     
    259259
    260260def check_install_conditions(c):
    261     '''Checks a couple of necessary attributes and conditions
    262     for the installation such as if they exist and contain values.
     261    '''Checks necessary attributes and conditions
     262    for the installation, e.g. whether they exist and contain values.
    263263    Otherwise set default values.
    264264
     
    310310    '''Creates a tarball with all necessary files which need to be sent to the
    311311    installation directory.
    312     It does not matter if this is local or remote.
    313     Collects all python files, the Fortran source and makefiles,
     312    It does not matter whether this is local or remote.
     313    Collects all Python files, the Fortran source and makefiles,
    314314    the ECMWF_ENV file, the CONTROL files as well as the
    315315    template files.
     
    497497    ----------
    498498    makefile : str
    499         Name of the makefile which should be used to compile FORTRAN
    500         CONVERT2 program.
     499        Name of the makefile which should be used to compile the Fortran
     500        program.
    501501
    502502    ecuid : str
     
    686686            print(perr.decode())
    687687            print('Please edit ' + makefile +
    688                   ' or try another Makefile in the src directory.')
     688                  ' or try another makefile in the src directory.')
    689689            print('Most likely GRIB_API_INCLUDE_DIR, GRIB_API_LIB '
    690690                  'and EMOSLIB must be adapted.')
    691             print('Available Makefiles:')
    692             print(UioFiles(src_path, 'Makefile*'))
     691            print('Available makefiles:')
     692            print(UioFiles(src_path, 'makefile*'))
    693693            sys.exit('Compilation failed!')
    694694    except ValueError as e:
    695         print('ERROR: Makefile call failed:')
     695        print('ERROR: make of Fortran code failed:')
    696696        print(e)
    697697    else:
    698         execute_subprocess(['ls', '-l',
     698        execute_subprocess(['ls', '-l', 
    699699                            os.path.join(src_path, _config.FORTRAN_EXECUTABLE)],
    700700                           error_msg='FORTRAN EXECUTABLE COULD NOT BE FOUND!')
Note: See TracChangeset for help on using the changeset viewer.
hosted by ZAMG