Ignore:
Timestamp:
Jan 15, 2019, 1:03:37 AM (5 years ago)
Author:
Anne Philipp <anne.philipp@…>
Branches:
master, ctbto, dev
Children:
6f951ca
Parents:
0629ba8
Message:

new build of sphinx for updated api

File:
1 edited

Legend:

Unmodified
Added
Removed
  • documentation/Sphinx/build/html/_modules/get_mars_data.html

    r274f9ef r2625ca8  
    175175<span class="c1">#          online documentation)</span>
    176176<span class="c1">#        - use of UIFiles class for file selection and deletion</span>
    177 <span class="c1">#</span>
     177<span class="c1">#        - seperated get_mars_data function into several smaller pieces:</span>
     178<span class="c1">#          write_reqheader, mk_server, mk_dates, remove_old, do_retrievment</span>
    178179<span class="c1">#</span>
    179180<span class="c1"># @License:</span>
    180 <span class="c1">#    (C) Copyright 2014-2018.</span>
     181<span class="c1">#    (C) Copyright 2014-2019.</span>
     182<span class="c1">#    Anne Philipp, Leopold Haimberger</span>
    181183<span class="c1">#</span>
    182 <span class="c1">#    This software is licensed under the terms of the Apache Licence Version 2.0</span>
    183 <span class="c1">#    which can be obtained at http://www.apache.org/licenses/LICENSE-2.0.</span>
    184 <span class="c1">#</span>
    185 <span class="c1"># @Program Functionality:</span>
    186 <span class="c1">#    This program can be used as a module in the whole flex_extract process</span>
    187 <span class="c1">#    or can be run by itself to just extract MARS data from ECMWF. To do so,</span>
    188 <span class="c1">#    a couple of necessary parameters has to be passed with the program call.</span>
    189 <span class="c1">#    See documentation for more details.</span>
    190 <span class="c1">#</span>
    191 <span class="c1"># @Program Content:</span>
    192 <span class="c1">#    - main</span>
    193 <span class="c1">#    - get_mars_data</span>
    194 <span class="c1">#    - do_retrievement</span>
    195 <span class="c1">#</span>
     184<span class="c1">#    This work is licensed under the Creative Commons Attribution 4.0</span>
     185<span class="c1">#    International License. To view a copy of this license, visit</span>
     186<span class="c1">#    http://creativecommons.org/licenses/by/4.0/ or send a letter to</span>
     187<span class="c1">#    Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.</span>
    196188<span class="c1">#*******************************************************************************</span>
    197 
     189<span class="sd">&#39;&#39;&#39;This script extracts MARS data from ECMWF servers.</span>
     190
     191<span class="sd">At first, the necessary parameters from command line and CONTROL files are</span>
     192<span class="sd">extracted. They define the data set to be extracted from MARS.</span>
     193
     194<span class="sd">This file can also be imported as a module and contains the following</span>
     195<span class="sd">functions:</span>
     196
     197<span class="sd">    * main - the main function of the script</span>
     198<span class="sd">    * get_mars_data - overall control of ECMWF data retrievment</span>
     199<span class="sd">    * write_reqheader - writes the header into the mars_request file</span>
     200<span class="sd">    * mk_server - creates the server connection to ECMWF servers</span>
     201<span class="sd">    * mk_dates - defines the start and end date</span>
     202<span class="sd">    * remove_old - deletes old retrieved grib files</span>
     203<span class="sd">    * do_retrievement - creates individual retrievals</span>
     204
     205<span class="sd">Type: get_mars_data.py --help</span>
     206<span class="sd">to get information about command line parameters.</span>
     207<span class="sd">Read the documentation for usage instructions.</span>
     208<span class="sd">&#39;&#39;&#39;</span>
    198209<span class="c1"># ------------------------------------------------------------------------------</span>
    199210<span class="c1"># MODULES</span>
     
    205216
    206217<span class="c1"># software specific classes and modules from flex_extract</span>
     218<span class="c1"># add path to local main python path for flex_extract to get full access</span>
    207219<span class="n">sys</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">dirname</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">abspath</span><span class="p">(</span>
    208220    <span class="n">inspect</span><span class="o">.</span><span class="n">getfile</span><span class="p">(</span><span class="n">inspect</span><span class="o">.</span><span class="n">currentframe</span><span class="p">())))</span> <span class="o">+</span> <span class="s1">&#39;/../&#39;</span><span class="p">)</span>
    209221<span class="kn">import</span> <span class="nn">_config</span>
    210 <span class="kn">from</span> <span class="nn">tools</span> <span class="k">import</span> <span class="p">(</span><span class="n">my_error</span><span class="p">,</span> <span class="n">normal_exit</span><span class="p">,</span> <span class="n">get_cmdline_arguments</span><span class="p">,</span>
     222<span class="kn">from</span> <span class="nn">tools</span> <span class="k">import</span> <span class="p">(</span><span class="n">my_error</span><span class="p">,</span> <span class="n">normal_exit</span><span class="p">,</span> <span class="n">get_cmdline_args</span><span class="p">,</span>
    211223                   <span class="n">read_ecenv</span><span class="p">,</span> <span class="n">make_dir</span><span class="p">)</span>
    212224<span class="kn">from</span> <span class="nn">classes.EcFlexpart</span> <span class="k">import</span> <span class="n">EcFlexpart</span>
    213225<span class="kn">from</span> <span class="nn">classes.UioFiles</span> <span class="k">import</span> <span class="n">UioFiles</span>
     226<span class="kn">from</span> <span class="nn">classes.MarsRetrieval</span> <span class="k">import</span> <span class="n">MarsRetrieval</span>
    214227
    215228<span class="k">try</span><span class="p">:</span>
     
    235248<span class="sd">    &#39;&#39;&#39;</span>
    236249
    237     <span class="n">args</span> <span class="o">=</span> <span class="n">get_cmdline_arguments</span><span class="p">()</span>
     250    <span class="n">args</span> <span class="o">=</span> <span class="n">get_cmdline_args</span><span class="p">()</span>
    238251    <span class="n">c</span> <span class="o">=</span> <span class="n">ControlFile</span><span class="p">(</span><span class="n">args</span><span class="o">.</span><span class="n">controlfile</span><span class="p">)</span>
    239252
     
    244257
    245258    <span class="n">get_mars_data</span><span class="p">(</span><span class="n">c</span><span class="p">)</span>
    246     <span class="n">normal_exit</span><span class="p">(</span><span class="n">c</span><span class="o">.</span><span class="n">mailfail</span><span class="p">,</span> <span class="s1">&#39;Done!&#39;</span><span class="p">)</span>
     259    <span class="n">normal_exit</span><span class="p">(</span><span class="n">c</span><span class="o">.</span><span class="n">mailops</span><span class="p">,</span> <span class="n">c</span><span class="o">.</span><span class="n">queue</span><span class="p">,</span> <span class="s1">&#39;Done!&#39;</span><span class="p">)</span>
    247260
    248261    <span class="k">return</span></div>
     
    257270<span class="sd">    Parameters</span>
    258271<span class="sd">    ----------</span>
    259 <span class="sd">    c : :obj:`ControlFile`</span>
     272<span class="sd">    c : ControlFile</span>
    260273<span class="sd">        Contains all the parameters of CONTROL file and</span>
    261274<span class="sd">        command line.</span>
     
    265278
    266279<span class="sd">    &#39;&#39;&#39;</span>
     280    <span class="n">c</span><span class="o">.</span><span class="n">ecapi</span> <span class="o">=</span> <span class="n">ecapi</span>
    267281
    268282    <span class="k">if</span> <span class="ow">not</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">exists</span><span class="p">(</span><span class="n">c</span><span class="o">.</span><span class="n">inputdir</span><span class="p">):</span>
    269283        <span class="n">make_dir</span><span class="p">(</span><span class="n">c</span><span class="o">.</span><span class="n">inputdir</span><span class="p">)</span>
    270284
    271     <span class="k">if</span> <span class="n">c</span><span class="o">.</span><span class="n">request</span> <span class="o">==</span> <span class="mi">0</span> <span class="ow">or</span> <span class="n">c</span><span class="o">.</span><span class="n">request</span> <span class="o">==</span> <span class="mi">2</span><span class="p">:</span>
     285    <span class="k">if</span> <span class="n">c</span><span class="o">.</span><span class="n">request</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
    272286        <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Retrieving EC data!&quot;</span><span class="p">)</span>
    273     <span class="k">elif</span> <span class="n">c</span><span class="o">.</span><span class="n">request</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
    274         <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Printing mars requests!&quot;</span><span class="p">)</span>
     287    <span class="k">else</span><span class="p">:</span>
     288        <span class="k">if</span> <span class="n">c</span><span class="o">.</span><span class="n">request</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
     289            <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Printing mars requests!&quot;</span><span class="p">)</span>
     290        <span class="k">elif</span> <span class="n">c</span><span class="o">.</span><span class="n">request</span> <span class="o">==</span> <span class="mi">2</span><span class="p">:</span>
     291            <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Retrieving EC data and printing mars request!&quot;</span><span class="p">)</span>
     292        <span class="n">write_reqheader</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">c</span><span class="o">.</span><span class="n">inputdir</span><span class="p">,</span> <span class="n">_config</span><span class="o">.</span><span class="n">FILE_MARS_REQUESTS</span><span class="p">))</span>
    275293
    276294    <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;start date </span><span class="si">%s</span><span class="s2"> &quot;</span> <span class="o">%</span> <span class="p">(</span><span class="n">c</span><span class="o">.</span><span class="n">start_date</span><span class="p">))</span>
    277295    <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;end date </span><span class="si">%s</span><span class="s2"> &quot;</span> <span class="o">%</span> <span class="p">(</span><span class="n">c</span><span class="o">.</span><span class="n">end_date</span><span class="p">))</span>
    278296
    279     <span class="k">if</span> <span class="n">ecapi</span><span class="p">:</span>
     297    <span class="n">server</span> <span class="o">=</span> <span class="n">mk_server</span><span class="p">(</span><span class="n">c</span><span class="p">)</span>
     298
     299    <span class="c1"># if data are to be retrieved, clean up any old grib files</span>
     300    <span class="k">if</span> <span class="n">c</span><span class="o">.</span><span class="n">request</span> <span class="o">==</span> <span class="mi">0</span> <span class="ow">or</span> <span class="n">c</span><span class="o">.</span><span class="n">request</span> <span class="o">==</span> <span class="mi">2</span><span class="p">:</span>
     301        <span class="n">remove_old</span><span class="p">(</span><span class="s1">&#39;*grb&#39;</span><span class="p">,</span> <span class="n">c</span><span class="o">.</span><span class="n">inputdir</span><span class="p">)</span>
     302
     303    <span class="c1"># --------------  flux data ------------------------------------------------</span>
     304    <span class="n">start</span><span class="p">,</span> <span class="n">end</span><span class="p">,</span> <span class="n">datechunk</span> <span class="o">=</span> <span class="n">mk_dates</span><span class="p">(</span><span class="n">c</span><span class="p">,</span> <span class="n">fluxes</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
     305    <span class="n">do_retrievement</span><span class="p">(</span><span class="n">c</span><span class="p">,</span> <span class="n">server</span><span class="p">,</span> <span class="n">start</span><span class="p">,</span> <span class="n">end</span><span class="p">,</span> <span class="n">datechunk</span><span class="p">,</span> <span class="n">fluxes</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
     306
     307    <span class="c1"># --------------  non flux data --------------------------------------------</span>
     308    <span class="n">start</span><span class="p">,</span> <span class="n">end</span><span class="p">,</span> <span class="n">datechunk</span> <span class="o">=</span> <span class="n">mk_dates</span><span class="p">(</span><span class="n">c</span><span class="p">,</span> <span class="n">fluxes</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
     309    <span class="n">do_retrievement</span><span class="p">(</span><span class="n">c</span><span class="p">,</span> <span class="n">server</span><span class="p">,</span> <span class="n">start</span><span class="p">,</span> <span class="n">end</span><span class="p">,</span> <span class="n">datechunk</span><span class="p">,</span> <span class="n">fluxes</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
     310
     311    <span class="k">return</span></div>
     312
     313<div class="viewcode-block" id="write_reqheader"><a class="viewcode-back" href="../api.html#get_mars_data.write_reqheader">[docs]</a><span class="k">def</span> <span class="nf">write_reqheader</span><span class="p">(</span><span class="n">marsfile</span><span class="p">):</span>
     314    <span class="sd">&#39;&#39;&#39;Writes header with column names into mars request file.</span>
     315
     316<span class="sd">    Parameters</span>
     317<span class="sd">    ----------</span>
     318<span class="sd">    marsfile : str</span>
     319<span class="sd">        Path to the mars request file.</span>
     320
     321<span class="sd">    Return</span>
     322<span class="sd">    ------</span>
     323
     324<span class="sd">    &#39;&#39;&#39;</span>
     325    <span class="n">MR</span> <span class="o">=</span> <span class="n">MarsRetrieval</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
     326    <span class="n">attrs</span> <span class="o">=</span> <span class="nb">vars</span><span class="p">(</span><span class="n">MR</span><span class="p">)</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
     327    <span class="k">del</span> <span class="n">attrs</span><span class="p">[</span><span class="s1">&#39;server&#39;</span><span class="p">]</span>
     328    <span class="k">del</span> <span class="n">attrs</span><span class="p">[</span><span class="s1">&#39;public&#39;</span><span class="p">]</span>
     329    <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">marsfile</span><span class="p">,</span> <span class="s1">&#39;w&#39;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
     330        <span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s1">&#39;request_number&#39;</span> <span class="o">+</span> <span class="s1">&#39;, &#39;</span><span class="p">)</span>
     331        <span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s1">&#39;, &#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">key</span><span class="p">)</span> <span class="k">for</span> <span class="n">key</span> <span class="ow">in</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">attrs</span><span class="o">.</span><span class="n">iterkeys</span><span class="p">())))</span>
     332        <span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s1">&#39;</span><span class="se">\n</span><span class="s1">&#39;</span><span class="p">)</span>
     333
     334    <span class="k">return</span></div>
     335
     336<div class="viewcode-block" id="mk_server"><a class="viewcode-back" href="../api.html#get_mars_data.mk_server">[docs]</a><span class="k">def</span> <span class="nf">mk_server</span><span class="p">(</span><span class="n">c</span><span class="p">):</span>
     337    <span class="sd">&#39;&#39;&#39;Creates server connection if ECMWF WebAPI is available.</span>
     338
     339<span class="sd">    Parameters</span>
     340<span class="sd">    ----------</span>
     341<span class="sd">    c : ControlFile</span>
     342<span class="sd">        Contains all the parameters of CONTROL file and</span>
     343<span class="sd">        command line.</span>
     344
     345<span class="sd">    Return</span>
     346<span class="sd">    ------</span>
     347<span class="sd">    server : ECMWFDataServer or ECMWFService</span>
     348<span class="sd">        Connection to ECMWF server via python interface ECMWF WebAPI.</span>
     349
     350<span class="sd">    &#39;&#39;&#39;</span>
     351    <span class="k">if</span> <span class="n">c</span><span class="o">.</span><span class="n">ecapi</span><span class="p">:</span>
    280352        <span class="k">if</span> <span class="n">c</span><span class="o">.</span><span class="n">public</span><span class="p">:</span>
    281353            <span class="n">server</span> <span class="o">=</span> <span class="n">ecmwfapi</span><span class="o">.</span><span class="n">ECMWFDataServer</span><span class="p">()</span>
     
    285357        <span class="n">server</span> <span class="o">=</span> <span class="kc">False</span>
    286358
    287     <span class="n">c</span><span class="o">.</span><span class="n">ecapi</span> <span class="o">=</span> <span class="n">ecapi</span>
    288359    <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;Using ECMWF WebAPI: &#39;</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="n">c</span><span class="o">.</span><span class="n">ecapi</span><span class="p">))</span>
    289360
    290     <span class="c1"># basetime geht rückwärts</span>
    291 
    292     <span class="c1"># if basetime 00</span>
    293     <span class="c1"># dann wird von 12 am vortag bis 00 am start tag geholt</span>
    294     <span class="c1"># aber ohne 12 selbst sondern 12 + step</span>
    295 
    296     <span class="c1"># if basetime 12</span>
    297     <span class="c1"># dann wird von 00 + step bis 12 am start tag geholt</span>
    298 
    299     <span class="c1"># purer forecast wird vorwärts bestimmt.</span>
    300     <span class="c1"># purer forecast mode ist dann wenn  größer 24 stunden</span>
    301     <span class="c1"># wie kann das noch festgestellt werden ????</span>
    302     <span class="c1"># nur FC und steps mehr als 24 ?</span>
    303     <span class="c1"># die einzige problematik beim reinen forecast ist die benennung der files!</span>
    304     <span class="c1"># also sobald es Tagesüberschneidungen gibt</span>
    305     <span class="c1"># allerdings ist das relevant und ersichtlich an den NICHT FLUSS DATEN</span>
    306 
     361    <span class="k">return</span> <span class="n">server</span></div>
     362
     363
     364<div class="viewcode-block" id="mk_dates"><a class="viewcode-back" href="../api.html#get_mars_data.mk_dates">[docs]</a><span class="k">def</span> <span class="nf">mk_dates</span><span class="p">(</span><span class="n">c</span><span class="p">,</span> <span class="n">fluxes</span><span class="p">):</span>
     365    <span class="sd">&#39;&#39;&#39;Prepares start and end date depending on flux or non flux data.</span>
     366
     367<span class="sd">    If forecast for maximum one day (upto 24h) are to be retrieved, then</span>
     368<span class="sd">    collect accumulation data (flux data) with additional days in the</span>
     369<span class="sd">    beginning and at the end (used for complete disaggregation of</span>
     370<span class="sd">    original period)</span>
     371
     372<span class="sd">    If forecast data longer than 24h are to be retrieved, then</span>
     373<span class="sd">    collect accumulation data (flux data) with the exact start and end date</span>
     374<span class="sd">    (disaggregation will be done for the exact time period with</span>
     375<span class="sd">    boundary conditions)</span>
     376
     377<span class="sd">    Since for basetime the extraction contains the 12 hours upfront,</span>
     378<span class="sd">    if basetime is 0, the starting date has to be the day before and</span>
     379
     380<span class="sd">    Parameters</span>
     381<span class="sd">    ----------</span>
     382<span class="sd">    c : ControlFile</span>
     383<span class="sd">        Contains all the parameters of CONTROL file and</span>
     384<span class="sd">        command line.</span>
     385
     386<span class="sd">    fluxes : boolean, optional</span>
     387<span class="sd">        Decides if the flux parameter settings are stored or</span>
     388<span class="sd">        the rest of the parameter list.</span>
     389<span class="sd">        Default value is False.</span>
     390
     391<span class="sd">    Return</span>
     392<span class="sd">    ------</span>
     393<span class="sd">    start : datetime</span>
     394<span class="sd">        The start date of the retrieving data set.</span>
     395
     396<span class="sd">    end : datetime</span>
     397<span class="sd">        The end date of the retrieving data set.</span>
     398
     399<span class="sd">    chunk : datetime</span>
     400<span class="sd">        Time period in days for one single mars retrieval.</span>
     401
     402<span class="sd">    &#39;&#39;&#39;</span>
    307403    <span class="n">start</span> <span class="o">=</span> <span class="n">datetime</span><span class="o">.</span><span class="n">strptime</span><span class="p">(</span><span class="n">c</span><span class="o">.</span><span class="n">start_date</span><span class="p">,</span> <span class="s1">&#39;%Y%m</span><span class="si">%d</span><span class="s1">&#39;</span><span class="p">)</span>
    308404    <span class="n">end</span> <span class="o">=</span> <span class="n">datetime</span><span class="o">.</span><span class="n">strptime</span><span class="p">(</span><span class="n">c</span><span class="o">.</span><span class="n">end_date</span><span class="p">,</span> <span class="s1">&#39;%Y%m</span><span class="si">%d</span><span class="s1">&#39;</span><span class="p">)</span>
    309     <span class="c1"># time period for one single retrieval</span>
    310     <span class="n">datechunk</span> <span class="o">=</span> <span class="n">timedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="nb">int</span><span class="p">(</span><span class="n">c</span><span class="o">.</span><span class="n">date_chunk</span><span class="p">))</span>
    311 
    312     <span class="k">if</span> <span class="n">c</span><span class="o">.</span><span class="n">basetime</span> <span class="o">==</span> <span class="s1">&#39;00&#39;</span><span class="p">:</span>
     405    <span class="n">chunk</span> <span class="o">=</span> <span class="n">timedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="nb">int</span><span class="p">(</span><span class="n">c</span><span class="o">.</span><span class="n">date_chunk</span><span class="p">))</span>
     406
     407    <span class="k">if</span> <span class="n">c</span><span class="o">.</span><span class="n">basetime</span><span class="p">:</span>
     408        <span class="k">if</span> <span class="n">c</span><span class="o">.</span><span class="n">basetime</span> <span class="o">==</span> <span class="s1">&#39;00&#39;</span><span class="p">:</span>
     409            <span class="n">start</span> <span class="o">=</span> <span class="n">start</span> <span class="o">-</span> <span class="n">timedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
     410
     411    <span class="k">if</span> <span class="ow">not</span> <span class="n">c</span><span class="o">.</span><span class="n">purefc</span> <span class="ow">and</span> <span class="n">fluxes</span><span class="p">:</span>
    313412        <span class="n">start</span> <span class="o">=</span> <span class="n">start</span> <span class="o">-</span> <span class="n">timedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
    314 
    315     <span class="k">if</span> <span class="n">c</span><span class="o">.</span><span class="n">maxstep</span> <span class="o">&lt;=</span> <span class="mi">24</span><span class="p">:</span>
    316         <span class="n">startm1</span> <span class="o">=</span> <span class="n">start</span> <span class="o">-</span> <span class="n">timedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
    317 
    318     <span class="k">if</span> <span class="n">c</span><span class="o">.</span><span class="n">basetime</span> <span class="o">==</span> <span class="s1">&#39;00&#39;</span> <span class="ow">or</span> <span class="n">c</span><span class="o">.</span><span class="n">basetime</span> <span class="o">==</span> <span class="s1">&#39;12&#39;</span><span class="p">:</span>
    319         <span class="c1"># endp1 = end + timedelta(days=1)</span>
    320         <span class="n">endp1</span> <span class="o">=</span> <span class="n">end</span>
    321     <span class="k">else</span><span class="p">:</span>
    322         <span class="c1"># endp1 = end + timedelta(days=2)</span>
    323         <span class="n">endp1</span> <span class="o">=</span> <span class="n">end</span> <span class="o">+</span> <span class="n">timedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
    324 
    325     <span class="c1"># --------------  flux data ------------------------------------------------</span>
    326     <span class="k">if</span> <span class="n">c</span><span class="o">.</span><span class="n">request</span> <span class="o">==</span> <span class="mi">0</span> <span class="ow">or</span> <span class="n">c</span><span class="o">.</span><span class="n">request</span> <span class="o">==</span> <span class="mi">2</span><span class="p">:</span>
    327         <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;... removing old flux content of &#39;</span> <span class="o">+</span> <span class="n">c</span><span class="o">.</span><span class="n">inputdir</span><span class="p">)</span>
    328         <span class="n">tobecleaned</span> <span class="o">=</span> <span class="n">UioFiles</span><span class="p">(</span><span class="n">c</span><span class="o">.</span><span class="n">inputdir</span><span class="p">,</span>
    329                                <span class="s1">&#39;*_acc_*.&#39;</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">getppid</span><span class="p">())</span> <span class="o">+</span> <span class="s1">&#39;.*.grb&#39;</span><span class="p">)</span>
    330         <span class="n">tobecleaned</span><span class="o">.</span><span class="n">delete_files</span><span class="p">()</span>
    331 
    332     <span class="c1"># if forecast for maximum one day (upto 24h) are to be retrieved,</span>
    333     <span class="c1"># collect accumulation data (flux data)</span>
    334     <span class="c1"># with additional days in the beginning and at the end</span>
    335     <span class="c1"># (used for complete disaggregation of original period)</span>
    336     <span class="k">if</span> <span class="n">c</span><span class="o">.</span><span class="n">maxstep</span> <span class="o">&lt;=</span> <span class="mi">24</span><span class="p">:</span>
    337         <span class="n">do_retrievement</span><span class="p">(</span><span class="n">c</span><span class="p">,</span> <span class="n">server</span><span class="p">,</span> <span class="n">startm1</span><span class="p">,</span> <span class="n">endp1</span><span class="p">,</span> <span class="n">datechunk</span><span class="p">,</span> <span class="n">fluxes</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
    338 
    339     <span class="c1"># if forecast data longer than 24h are to be retrieved,</span>
    340     <span class="c1"># collect accumulation data (flux data)</span>
    341     <span class="c1"># with the exact start and end date</span>
    342     <span class="c1"># (disaggregation will be done for the</span>
    343     <span class="c1"># exact time period with boundary conditions)</span>
    344     <span class="k">else</span><span class="p">:</span>
    345         <span class="n">do_retrievement</span><span class="p">(</span><span class="n">c</span><span class="p">,</span> <span class="n">server</span><span class="p">,</span> <span class="n">start</span><span class="p">,</span> <span class="n">end</span><span class="p">,</span> <span class="n">datechunk</span><span class="p">,</span> <span class="n">fluxes</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
    346 
    347     <span class="c1"># --------------  non flux data --------------------------------------------</span>
    348     <span class="k">if</span> <span class="n">c</span><span class="o">.</span><span class="n">request</span> <span class="o">==</span> <span class="mi">0</span> <span class="ow">or</span> <span class="n">c</span><span class="o">.</span><span class="n">request</span> <span class="o">==</span> <span class="mi">2</span><span class="p">:</span>
    349         <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;... removing old non flux content of &#39;</span> <span class="o">+</span> <span class="n">c</span><span class="o">.</span><span class="n">inputdir</span><span class="p">)</span>
    350         <span class="n">tobecleaned</span> <span class="o">=</span> <span class="n">UioFiles</span><span class="p">(</span><span class="n">c</span><span class="o">.</span><span class="n">inputdir</span><span class="p">,</span>
    351                                <span class="s1">&#39;*__*.&#39;</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">getppid</span><span class="p">())</span> <span class="o">+</span> <span class="s1">&#39;.*.grb&#39;</span><span class="p">)</span>
    352         <span class="n">tobecleaned</span><span class="o">.</span><span class="n">delete_files</span><span class="p">()</span>
    353 
    354     <span class="n">do_retrievement</span><span class="p">(</span><span class="n">c</span><span class="p">,</span> <span class="n">server</span><span class="p">,</span> <span class="n">start</span><span class="p">,</span> <span class="n">end</span><span class="p">,</span> <span class="n">datechunk</span><span class="p">,</span> <span class="n">fluxes</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
     413        <span class="n">end</span> <span class="o">=</span> <span class="n">end</span> <span class="o">+</span> <span class="n">timedelta</span><span class="p">(</span><span class="n">days</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
     414
     415    <span class="k">return</span> <span class="n">start</span><span class="p">,</span> <span class="n">end</span><span class="p">,</span> <span class="n">chunk</span></div>
     416
     417<div class="viewcode-block" id="remove_old"><a class="viewcode-back" href="../api.html#get_mars_data.remove_old">[docs]</a><span class="k">def</span> <span class="nf">remove_old</span><span class="p">(</span><span class="n">pattern</span><span class="p">,</span> <span class="n">inputdir</span><span class="p">):</span>
     418    <span class="sd">&#39;&#39;&#39;Deletes old retrieval files from current input directory</span>
     419<span class="sd">    matching the pattern.</span>
     420
     421<span class="sd">    Parameters</span>
     422<span class="sd">    ----------</span>
     423<span class="sd">    pattern : str</span>
     424<span class="sd">        The sub string pattern which identifies the files to be deleted.</span>
     425
     426<span class="sd">    inputdir : str, optional</span>
     427<span class="sd">        Path to the directory where the retrieved data is stored.</span>
     428
     429<span class="sd">    Return</span>
     430<span class="sd">    ------</span>
     431
     432<span class="sd">    &#39;&#39;&#39;</span>
     433    <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;... removing old content of &#39;</span> <span class="o">+</span> <span class="n">inputdir</span><span class="p">)</span>
     434
     435    <span class="n">tobecleaned</span> <span class="o">=</span> <span class="n">UioFiles</span><span class="p">(</span><span class="n">inputdir</span><span class="p">,</span> <span class="n">pattern</span><span class="p">)</span>
     436    <span class="n">tobecleaned</span><span class="o">.</span><span class="n">delete_files</span><span class="p">()</span>
    355437
    356438    <span class="k">return</span></div>
     439
    357440
    358441<div class="viewcode-block" id="do_retrievement"><a class="viewcode-back" href="../api.html#get_mars_data.do_retrievement">[docs]</a><span class="k">def</span> <span class="nf">do_retrievement</span><span class="p">(</span><span class="n">c</span><span class="p">,</span> <span class="n">server</span><span class="p">,</span> <span class="n">start</span><span class="p">,</span> <span class="n">end</span><span class="p">,</span> <span class="n">delta_t</span><span class="p">,</span> <span class="n">fluxes</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
     
    362445<span class="sd">    Parameters</span>
    363446<span class="sd">    ----------</span>
    364 <span class="sd">    c : :obj:`ControlFile`</span>
     447<span class="sd">    c : ControlFile</span>
    365448<span class="sd">        Contains all the parameters of CONTROL file and</span>
    366449<span class="sd">        command line.</span>
    367450
    368 <span class="sd">    server : :obj:`ECMWFService`</span>
     451<span class="sd">    server : ECMWFService or ECMWFDataServer</span>
    369452<span class="sd">            The server connection to ECMWF.</span>
    370453
    371 <span class="sd">    start : :obj:`datetime`</span>
     454<span class="sd">    start : datetime</span>
    372455<span class="sd">        The start date of the retrieval.</span>
    373456
    374 <span class="sd">    end : :obj:`datetime`</span>
     457<span class="sd">    end : datetime</span>
    375458<span class="sd">        The end date of the retrieval.</span>
    376459
    377 <span class="sd">    delta_t : :obj:`datetime`</span>
     460<span class="sd">    delta_t : datetime</span>
    378461<span class="sd">        Delta_t + 1 is the maximal time period of a single</span>
    379462<span class="sd">        retrieval.</span>
    380463
    381 <span class="sd">    fluxes : :obj:`boolean`, optional</span>
     464<span class="sd">    fluxes : boolean, optional</span>
    382465<span class="sd">        Decides if the flux parameters are to be retrieved or</span>
    383466<span class="sd">        the rest of the parameter list.</span>
     
    430513  <div role="contentinfo">
    431514    <p>
    432         &copy; Copyright 2018, Anne Philipp and Leopold Haimberger.
     515        &copy; Copyright 2018, Anne Philipp and Leopold Haimberger
    433516
    434517    </p>
     
    449532 
    450533
    451     <script type="text/javascript">
    452         var DOCUMENTATION_OPTIONS = {
    453             URL_ROOT:'../',
    454             VERSION:'7.1 alpha',
    455             LANGUAGE:'None',
    456             COLLAPSE_INDEX:false,
    457             FILE_SUFFIX:'.html',
    458             HAS_SOURCE:  true,
    459             SOURCELINK_SUFFIX: '.txt'
    460         };
    461     </script>
    462       <script type="text/javascript" src="../_static/jquery.js"></script>
    463       <script type="text/javascript" src="../_static/underscore.js"></script>
    464       <script type="text/javascript" src="../_static/doctools.js"></script>
    465       <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
     534   
     535   
     536      <script type="text/javascript" id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
     537        <script type="text/javascript" src="../_static/jquery.js"></script>
     538        <script type="text/javascript" src="../_static/underscore.js"></script>
     539        <script type="text/javascript" src="../_static/doctools.js"></script>
     540   
    466541
    467542 
Note: See TracChangeset for help on using the changeset viewer.
hosted by ZAMG