Changeset 0690a6c in flex_extract.git for Source


Ignore:
Timestamp:
Aug 13, 2019, 12:35:02 PM (5 years ago)
Author:
Anne Philipp <anne.philipp@…>
Branches:
master, ctbto, dev
Children:
30f7911
Parents:
c77630a (diff), 2d1c338 (diff)
Note: this is a merge changeset, the changes displayed below correspond to the merge itself.
Use the (diff) links above to see all the changes relative to each parent.
Message:

Merge branch 'dev' of ​https://www.flexpart.eu/git/flex_extract into dev

merging changes from Petra: FORD update to 6.0 and moving directory Test to Testing plus minor stuff

Location:
Source
Files:
1 added
14 edited
4 moved

Legend:

Unmodified
Added
Removed

  • Source/Python/Classes/ControlFile.py

    rba99230 r44174de  
    1 #!/usr/bin/env python
     1#!/usr/bin/env python3
    22# -*- coding: utf-8 -*-
    33#*******************************************************************************
    … …  
    2424#    Anne Philipp, Leopold Haimberger
    2525#
     26#    SPDX-License-Identifier: CC-BY-4.0
     27#
    2628#    This work is licensed under the Creative Commons Attribution 4.0
    2729#    International License. To view a copy of this license, visit
    2830#    http://creativecommons.org/licenses/by/4.0/ or send a letter to
    2931#    Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.
    30 #
    31 # @Class Methods:
    32 #    __init__
    33 #    _read_controlfile
    34 #    __str__
    35 #    assign_args_to_control
    36 #    assign_envs_to_control
    37 #    check_conditions
    38 #    check_install_conditions
    39 #    to_list
    4032#*******************************************************************************
    4133

  • Source/Python/Classes/EcFlexpart.py

    rba99230 rc77630a  
    1 #!/usr/bin/env python
     1#!/usr/bin/env python3
    22# -*- coding: utf-8 -*-
    33#*******************************************************************************
    … …  
    4242#    Anne Philipp, Leopold Haimberger
    4343#
     44#    SPDX-License-Identifier: CC-BY-4.0
     45#
    4446#    This work is licensed under the Creative Commons Attribution 4.0
    4547#    International License. To view a copy of this license, visit
    … …  
    415417        # -----------------------------------------------------------------------
    416418        if wrf:
    417             self.params['OG__ML'][0] += '/Z/VO'
    418             if '/D' not in self.params['OG__ML'][0]:
    419                 self.params['OG__ML'][0] += '/D'
    420 
    421             wrf_sfc = ['SP','SKT','SST','CI','STL1','STL2', 'STL3','STL4',
    422                        'SWVL1','SWVL2','SWVL3','SWVL4']
    423             for par in wrf_sfc:
    424                 if par not in self.params['OG__SL'][0]:
    425                     self.params['OG__SL'][0] += '/' + par
     419            # @WRF
     420            # THIS IS NOT YET CORRECTLY IMPLEMENTED !!!
     421            #
     422            # UNDER CONSTRUCTION !!!
     423            #
     424
     425            print('WRF VERSION IS UNDER CONSTRUCTION!') # dummy argument
     426
     427            #self.params['OG__ML'][0] += '/Z/VO'
     428            #if '/D' not in self.params['OG__ML'][0]:
     429            #    self.params['OG__ML'][0] += '/D'
     430
     431            #wrf_sfc = ['SP','SKT','SST','CI','STL1','STL2', 'STL3','STL4',
     432            #           'SWVL1','SWVL2','SWVL3','SWVL4']
     433            #for par in wrf_sfc:
     434            #    if par not in self.params['OG__SL'][0]:
     435            #        self.params['OG__SL'][0] += '/' + par
    426436
    427437        return
    … …  
    846856            if area[1] > area[3]:
    847857                area[1] -= 360
    848             maxl = round((area[3] - area[1]) / grid[1]) + 1
    849             maxb = round((area[0] - area[2]) / grid[0]) + 1
     858            maxl = int(round((area[3] - area[1]) / grid[1])) + 1
     859            maxb = int(round((area[0] - area[2]) / grid[0])) + 1
    850860
    851861            stream = namelist_template.generate(
    … …  
    10841094
    10851095            print("outputfile = " + fnout)
    1086             f_handle = open(fnout, 'w')
    1087             h_handle = open(hnout, 'w')
    1088             g_handle = open(gnout, 'w')
     1096            f_handle = open(fnout, 'wb')
     1097            h_handle = open(hnout, 'wb')
     1098            g_handle = open(gnout, 'wb')
    10891099
    10901100            # read message for message and store relevant data fields, where
    … …  
    13921402            # write original time step to flux file as usual
    13931403            fluxfile = GribUtil(os.path.join(c.inputdir, fluxfilename))
    1394             fluxfile.set_keys(tmpfile, filemode='a', strict=True,
     1404            fluxfile.set_keys(tmpfile, filemode='ab', strict=True,
    13951405                              wherekeynames=['paramId'], wherekeyvalues=[142],
    13961406                              keynames=['perturbationNumber','date','time','stepRange','values'],
    … …  
    13981408                                         date.hour*100, 0, lsp_new_np[inumb,:,it]],
    13991409                             )
    1400             fluxfile.set_keys(tmpfile, filemode='a', strict=True,
     1410            fluxfile.set_keys(tmpfile, filemode='ab', strict=True,
    14011411                              wherekeynames=['paramId'], wherekeyvalues=[143],
    14021412                              keynames=['perturbationNumber','date','time','stepRange','values'],
    … …  
    14061416
    14071417            # rr for first subgrid point is identified by step = 1
    1408             fluxfile.set_keys(tmpfile, filemode='a', strict=True,
     1418            fluxfile.set_keys(tmpfile, filemode='ab', strict=True,
    14091419                              wherekeynames=['paramId'], wherekeyvalues=[142],
    14101420                              keynames=['perturbationNumber','date','time','stepRange','values'],
    … …  
    14121422                                         date.hour*100, '1', lsp_new_np[inumb,:,it+1]]
    14131423                              )
    1414             fluxfile.set_keys(tmpfile, filemode='a', strict=True,
     1424            fluxfile.set_keys(tmpfile, filemode='ab', strict=True,
    14151425                              wherekeynames=['paramId'], wherekeyvalues=[143],
    14161426                              keynames=['perturbationNumber','date','time','stepRange','values'],
    … …  
    14201430
    14211431            # rr for second subgrid point is identified by step = 2
    1422             fluxfile.set_keys(tmpfile, filemode='a', strict=True,
     1432            fluxfile.set_keys(tmpfile, filemode='ab', strict=True,
    14231433                              wherekeynames=['paramId'], wherekeyvalues=[142],
    14241434                              keynames=['perturbationNumber','date','time','stepRange','values'],
    … …  
    14261436                                         date.hour*100, '2', lsp_new_np[inumb,:,it+2]]
    14271437                              )
    1428             fluxfile.set_keys(tmpfile, filemode='a', strict=True,
     1438            fluxfile.set_keys(tmpfile, filemode='ab', strict=True,
    14291439                              wherekeynames=['paramId'], wherekeyvalues=[143],
    14301440                              keynames=['perturbationNumber','date','time','stepRange','values'],
    … …  
    14561466
    14571467        gribfile.copy_dummy_msg(ifile, keynames=['paramId'],
    1458                       keyvalues=[142], filemode='w')
     1468                      keyvalues=[142], filemode='wb')
    14591469
    14601470        gribfile.copy_dummy_msg(ifile, keynames=['paramId'],
    1461                       keyvalues=[143], filemode='a')
     1471                      keyvalues=[143], filemode='ab')
    14621472
    14631473        return
    … …  
    15021512        end_period = end_period + timedelta(hours=int(c.step[-1]))
    15031513
    1504         if c.wrf:
    1505             table128 = init128(_config.PATH_GRIBTABLE)
    1506             wrfpars = to_param_id('sp/mslp/skt/2t/10u/10v/2d/z/lsm/sst/ci/sd/\
    1507                                    stl1/stl2/stl3/stl4/swvl1/swvl2/swvl3/swvl4',
    1508                                   table128)
     1514        # @WRF
     1515        # THIS IS NOT YET CORRECTLY IMPLEMENTED !!!
     1516        #
     1517        # UNDER CONSTRUCTION !!!
     1518        #
     1519        #if c.wrf:
     1520        #    table128 = init128(_config.PATH_GRIBTABLE)
     1521        #    wrfpars = to_param_id('sp/mslp/skt/2t/10u/10v/2d/z/lsm/sst/ci/sd/\
     1522        #                           stl1/stl2/stl3/stl4/swvl1/swvl2/swvl3/swvl4',
     1523        #                          table128)
    15091524
    15101525        # these numbers are indices for the temporary files "fort.xx"
    … …  
    15581573                fortfile = os.path.join(c.inputdir, 'fort.' + k)
    15591574                silent_remove(fortfile)
    1560                 fdict[k] = open(fortfile, 'w')
     1575                fdict[k] = open(fortfile, 'wb')
    15611576#============================================================================================
    15621577            # create correct timestamp from the three time informations
    … …  
    15851600                    continue
    15861601
    1587             if c.wrf:
    1588                 if 'olddate' not in locals() or cdate != olddate:
    1589                     fwrf = open(os.path.join(c.outputdir,
    1590                                 'WRF' + cdate + '.' + ctime + '.000.grb2'), 'w')
    1591                     olddate = cdate[:]
     1602            # @WRF
     1603            # THIS IS NOT YET CORRECTLY IMPLEMENTED !!!
     1604            #
     1605            # UNDER CONSTRUCTION !!!
     1606            #
     1607            #if c.wrf:
     1608            #    if 'olddate' not in locals() or cdate != olddate:
     1609            #        fwrf = open(os.path.join(c.outputdir,
     1610            #                    'WRF' + cdate + '.' + ctime + '.000.grb2'), 'wb')
     1611            #        olddate = cdate[:]
    15921612#============================================================================================
    15931613            # savedfields remembers which fields were already used.
    … …  
    16271647                        codes_write(gid, fdict['22'])
    16281648                        scwc = None
    1629                 elif c.wrf and paramId in [129, 138, 155] and \
    1630                       levtype == 'hybrid': # Z, VO, D
    1631                     # do not do anything right now
    1632                     # these are specific parameter for WRF
    1633                     pass
     1649                # @WRF
     1650                # THIS IS NOT YET CORRECTLY IMPLEMENTED !!!
     1651                #
     1652                # UNDER CONSTRUCTION !!!
     1653                #
     1654                #elif c.wrf and paramId in [129, 138, 155] and \
     1655                #      levtype == 'hybrid': # Z, VO, D
     1656                #    # do not do anything right now
     1657                #    # these are specific parameter for WRF
     1658                #    pass
    16341659                else:
    16351660                    if paramId not in savedfields:
    … …  
    16401665                    else:
    16411666                        print('duplicate ' + str(paramId) + ' not written')
    1642 
    1643                 try:
    1644                     if c.wrf:
    1645                         # model layer
    1646                         if levtype == 'hybrid' and \
    1647                            paramId in [129, 130, 131, 132, 133, 138, 155]:
    1648                             codes_write(gid, fwrf)
    1649                         # sfc layer
    1650                         elif paramId in wrfpars:
    1651                             codes_write(gid, fwrf)
    1652                 except AttributeError:
    1653                     pass
     1667                # @WRF
     1668                # THIS IS NOT YET CORRECTLY IMPLEMENTED !!!
     1669                #
     1670                # UNDER CONSTRUCTION !!!
     1671                #
     1672                #try:
     1673                #    if c.wrf:
     1674                #        # model layer
     1675                #        if levtype == 'hybrid' and \
     1676                #           paramId in [129, 130, 131, 132, 133, 138, 155]:
     1677                #            codes_write(gid, fwrf)
     1678                #        # sfc layer
     1679                #        elif paramId in wrfpars:
     1680                #            codes_write(gid, fwrf)
     1681                #except AttributeError:
     1682                #    pass
    16541683
    16551684                codes_release(gid)
    … …  
    17211750                                            'rb'), fout)
    17221751# ============================================================================================
    1723         if c.wrf:
    1724             fwrf.close()
     1752
     1753        # @WRF
     1754        # THIS IS NOT YET CORRECTLY IMPLEMENTED !!!
     1755        #
     1756        # UNDER CONSTRUCTION !!!
     1757        #
     1758        #if c.wrf:
     1759        #    fwrf.close()
    17251760
    17261761        codes_index_release(iid)
    … …  
    17561791        # get a list of all prepared output files with control forecast (CF)
    17571792        CF_filelist = UioFiles(path, prefix + '*.N000')
     1793        CF_filelist.files = sorted(CF_filelist.files)
    17581794
    17591795        for cffile in CF_filelist.files:
    … …  
    17691805            filename = cffile.split('N000')[0]
    17701806            for i in range(1, maxnum + 1):
    1771 
    17721807                # read an ensemble member
    17731808                g = open(filename + 'N{:0>3}'.format(i), 'rb')

  • Source/Python/Classes/GribUtil.py

    rba99230 rc77630a  
    1 #!/usr/bin/env python
     1#!/usr/bin/env python3
    22# -*- coding: utf-8 -*-
    33#*******************************************************************************
    … …  
    1313#
    1414# @License:
    15 #    (C) Copyright 2014-2018.
    16 #
    17 #    This software is licensed under the terms of the Apache Licence Version 2.0
    18 #    which can be obtained at http://www.apache.org/licenses/LICENSE-2.0.
    19 #
    20 # @Class Description:
    21 #    The GRIB API provides all necessary tools to work directly with the
    22 #    grib files. Nevertheless, the GRIB API tools are very basic and are in
    23 #    direct connection with the grib files. This class provides some higher
    24 #    functions which apply a set of GRIB API tools together in the respective
    25 #    context. So, the class initially contains a list of grib files (their
    26 #    names) and the using program then applies the methods directly on the
    27 #    class objects without having to think about how the actual GRIB API
    28 #    tools have to be arranged.
    29 #
    30 # @Class Content:
    31 #    - __init__
    32 #    - get_keys
    33 #    - set_keys
    34 #    - copy_dummy_msg
    35 #    - index
    36 #
    37 # @Class Attributes:
    38 #    - filenames
    39 #
     15#    (C) Copyright 2014-2019.
     16#    Anne Philipp, Leopold Haimberger
     17#
     18#    SPDX-License-Identifier: CC-BY-4.0
     19#
     20#    This work is licensed under the Creative Commons Attribution 4.0
     21#    International License. To view a copy of this license, visit
     22#    http://creativecommons.org/licenses/by/4.0/ or send a letter to
     23#    Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.
    4024#*******************************************************************************
    4125
    … …  
    5337    '''
    5438    Class for GRIB utilities (new methods) based on GRIB API
     39
     40    The GRIB API provides all necessary tools to work directly with the
     41    grib files. Nevertheless, the GRIB API tools are very basic and are in
     42    direct connection with the grib files. This class provides some higher
     43    functions which apply a set of GRIB API tools together in the respective
     44    context. So, the class initially contains a list of grib files (their
     45    names) and the using program then applies the methods directly on the
     46    class objects without having to think about how the actual GRIB API
     47    tools have to be arranged.
    5548    '''
    5649    # --------------------------------------------------------------------------
    … …  
    9891                             codes_release)
    9992
    100         fileid = open(self.filenames, 'r')
    101 
    10293        return_list = []
    10394
    104         while 1:
     95        with open(self.filenames, 'rb') as fileid:
     96
    10597            gid = codes_new_from_file(fileid)
    106 
    107             if gid is None:
    108                 break
    10998
    11099            if len(wherekeynames) != len(wherekeyvalues):
    … …  
    130119            codes_release(gid)
    131120
    132         fileid.close()
    133 
    134121        return return_list
    135122
    136123
    137124    def set_keys(self, fromfile, keynames, keyvalues, wherekeynames=[],
    138                  wherekeyvalues=[], strict=False, filemode='w'):
     125                 wherekeyvalues=[], strict=False, filemode='wb'):
    139126        '''Opens the file to read the grib messages and then write
    140127        the selected messages (with wherekeys) to a new output file.
    … …  
    168155
    169156        filemode : :obj:`string`, optional
    170             Sets the mode for the output file. Default is "w".
     157            Sets the mode for the output file. Default is "wb".
    171158
    172159        Return
    … …  
    182169
    183170        fout = open(self.filenames, filemode)
    184         fin = open(fromfile)
    185 
    186         while 1:
     171
     172        with open(fromfile, 'rb') as fin:
    187173            gid = codes_grib_new_from_file(fin)
    188 
    189             if gid is None:
    190                 break
    191174
    192175            select = True
    … …  
    213196            codes_release(gid)
    214197
    215         fin.close()
    216198        fout.close()
    217199
    … …  
    219201
    220202    def copy_dummy_msg(self, filename_in, selectWhere=True,
    221                  keynames=[], keyvalues=[], filemode='w'):
     203                 keynames=[], keyvalues=[], filemode='wb'):
    222204        '''Add the content of another input grib file to the objects file but
    223205        only messages corresponding to keys/values passed to the function.
    … …  
    242224
    243225        filemode : :obj:`string`, optional
    244             Sets the mode for the output file. Default is "w".
     226            Sets the mode for the output file. Default is "wb".
    245227
    246228        Return
    … …  
    254236            raise Exception("Give a value for each keyname!")
    255237
    256         fin = open(filename_in, 'rb')
     238
    257239        fout = open(self.filenames, filemode)
    258240
    259241        fields = 0
    260242
    261         while fields < 1:
     243        with open(filename_in, 'rb') as fin:
     244            if fields >= 1:
     245                fout.close()
     246                return
     247
    262248            gid = codes_grib_new_from_file(fin)
    263 
    264             if gid is None:
    265                 break
    266249
    267250            select = True
    … …  
    285268            codes_release(gid)
    286269
    287         fin.close()
    288270        fout.close()
    289271

  • Source/Python/Classes/MarsRetrieval.py

    rba99230 rc77630a  
    1 #!/usr/bin/env python
     1#!/usr/bin/env python3
    22# -*- coding: utf-8 -*-
    33#*******************************************************************************
    … …  
    2222#    (C) Copyright 2014-2019.
    2323#    Anne Philipp, Leopold Haimberger
     24#
     25#    SPDX-License-Identifier: CC-BY-4.0
    2426#
    2527#    This work is licensed under the Creative Commons Attribution 4.0
    … …  
    486488            del attrs[key]
    487489
    488         attrs['ppengine'] = 'emos'
     490#        attrs['ppengine'] = 'emos'
    489491
    490492        # MARS request via Python script
    … …  
    518520                request_str = request_str + ',' + key + '=' + str(value)
    519521            request_str += ',target="' + target + '"'
    520             p = subprocess.Popen(['mars', '-e'],
     522            p = subprocess.Popen(['mars'], #'-e'],
    521523                                 stdin=subprocess.PIPE,
    522524                                 stdout=subprocess.PIPE,

  • Source/Python/Classes/UioFiles.py

    rba99230 r44174de  
    1 #!/usr/bin/env python
     1#!/usr/bin/env python3
    22# -*- coding: utf-8 -*-
    33#*******************************************************************************
    … …  
    2525#    (C) Copyright 2014-2019.
    2626#    Anne Philipp, Leopold Haimberger
     27#
     28#    SPDX-License-Identifier: CC-BY-4.0
    2729#
    2830#    This work is licensed under the Creative Commons Attribution 4.0

  • Source/Python/Mods/checks.py

    rba99230 r44174de  
    1 #!/usr/bin/env python
     1#!/usr/bin/env python3
    22# -*- coding: utf-8 -*-
    33#*******************************************************************************
    … …  
    1111#    (C) Copyright 2014-2019.
    1212#    Anne Philipp, Leopold Haimberger
     13#
     14#    SPDX-License-Identifier: CC-BY-4.0
    1315#
    1416#    This work is licensed under the Creative Commons Attribution 4.0

  • Source/Python/Mods/disaggregation.py

    rba99230 r44174de  
    1 #!/usr/bin/env python
     1#!/usr/bin/env python3
    22# -*- coding: utf-8 -*-
    33#*******************************************************************************
    … …  
    2222#    (C) Copyright 2014-2019.
    2323#    Anne Philipp, Leopold Haimberger
     24#
     25#    SPDX-License-Identifier: CC-BY-4.0
    2426#
    2527#    This work is licensed under the Creative Commons Attribution 4.0

  • Source/Python/Mods/get_mars_data.py

    rba99230 r44174de  
    1 #!/usr/bin/env python
     1#!/usr/bin/env python3
    22# -*- coding: utf-8 -*-
    33#*******************************************************************************
    … …  
    2828#    (C) Copyright 2014-2019.
    2929#    Anne Philipp, Leopold Haimberger
     30#
     31#    SPDX-License-Identifier: CC-BY-4.0
    3032#
    3133#    This work is licensed under the Creative Commons Attribution 4.0

  • Source/Python/Mods/prepare_flexpart.py

    rba99230 r44174de  
    1 #!/usr/bin/env python
     1#!/usr/bin/env python3
    22# -*- coding: utf-8 -*-
    33#*******************************************************************************
    … …  
    3131#    (C) Copyright 2014-2019.
    3232#    Anne Philipp, Leopold Haimberger
     33#
     34#    SPDX-License-Identifier: CC-BY-4.0
    3335#
    3436#    This work is licensed under the Creative Commons Attribution 4.0

  • Source/Python/Mods/profiling.py

    rba99230 r8463d78  
    1 #!/usr/bin/env python
     1#!/usr/bin/env python3
    22# -*- coding: utf-8 -*-
    33#************************************************************************

  • Source/Python/Mods/tools.py

    rba99230 rc77630a  
    1 #!/usr/bin/env python
     1#!/usr/bin/env python3
    22# -*- coding: utf-8 -*-
    33#*******************************************************************************
    … …  
    2626#    (C) Copyright 2014-2019.
    2727#    Anne Philipp, Leopold Haimberger
     28#
     29#    SPDX-License-Identifier: CC-BY-4.0
    2830#
    2931#    This work is licensed under the Creative Commons Attribution 4.0
    … …  
    761763    # --- open file ---
    762764    print("Opening file for getting information data --- %s" % filename)
    763     with open(filename) as f:
     765    with open(filename, 'rb') as f:
    764766        # load first message from file
    765767        gid = codes_grib_new_from_file(f)

  • Source/Python/_config.py

    rba99230 r44174de  
    1 #!/usr/bin/env python
     1#!/usr/bin/env python3
    22# -*- coding: utf-8 -*-
    33#*******************************************************************************
    … …  
    1111#    (C) Copyright 2014-2019.
    1212#    Anne Philipp, Leopold Haimberger
     13#
     14#    SPDX-License-Identifier: CC-BY-4.0
    1315#
    1416#    This work is licensed under the Creative Commons Attribution 4.0

  • Source/Python/install.py

    rba99230 r44174de  
    1 #!/usr/bin/env python
     1#!/usr/bin/env python3
    22# -*- coding: utf-8 -*-
    33#*******************************************************************************
    … …  
    2121#    (C) Copyright 2014-2019.
    2222#    Anne Philipp, Leopold Haimberger
     23#
     24#    SPDX-License-Identifier: CC-BY-4.0
    2325#
    2426#    This work is licensed under the Creative Commons Attribution 4.0

  • Source/Python/submit.py

    rba99230 r44174de  
    1 #!/usr/bin/env python
     1#!/usr/bin/env python3
    22# -*- coding: utf-8 -*-
    33#*******************************************************************************
    … …  
    2525#    (C) Copyright 2014-2019.
    2626#    Anne Philipp, Leopold Haimberger
     27#
     28#    SPDX-License-Identifier: CC-BY-4.0
    2729#
    2830#    This work is licensed under the Creative Commons Attribution 4.0
Note: See TracChangeset for help on using the changeset viewer.
hosted by ZAMG