Ignore:
Timestamp:
Mar 8, 2019, 10:00:34 AM (5 years ago)
Author:
Anne Philipp <anne.philipp@…>
Branches:
master, ctbto, dev
Children:
b4a4777
Parents:
8778c5a
Message:

deleted jobscripts for tracking and updated Sphinx doc

File:
1 edited

Legend:

Unmodified
Added
Removed
  • documentation/Sphinx/build/html/_modules/MarsRetrieval.html

    r2625ca8 rf66f990  
    188188<span class="kn">import</span> <span class="nn">sys</span>
    189189<span class="kn">import</span> <span class="nn">subprocess</span>
     190<span class="kn">import</span> <span class="nn">traceback</span>
    190191
    191192<span class="c1"># software specific classes and modules from flex_extract</span>
    192193<span class="n">sys</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s1">&#39;../&#39;</span><span class="p">)</span>
    193194<span class="kn">import</span> <span class="nn">_config</span>
     195<span class="k">try</span><span class="p">:</span>
     196    <span class="n">ec_api</span> <span class="o">=</span> <span class="kc">True</span>
     197    <span class="kn">import</span> <span class="nn">ecmwfapi</span>
     198<span class="k">except</span> <span class="ne">ImportError</span><span class="p">:</span>
     199    <span class="n">ec_api</span> <span class="o">=</span> <span class="kc">False</span>
     200
     201<span class="k">try</span><span class="p">:</span>
     202    <span class="n">cds_api</span> <span class="o">=</span> <span class="kc">True</span>
     203    <span class="kn">import</span> <span class="nn">cdsapi</span>
     204<span class="k">except</span> <span class="ne">ImportError</span><span class="p">:</span>
     205    <span class="n">cds_api</span> <span class="o">=</span> <span class="kc">False</span>
    194206<span class="c1"># ------------------------------------------------------------------------------</span>
    195207<span class="c1"># CLASS</span>
     
    283295<span class="sd">    &#39;&#39;&#39;</span>
    284296
    285     <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">server</span><span class="p">,</span> <span class="n">public</span><span class="p">,</span> <span class="n">marsclass</span><span class="o">=</span><span class="s2">&quot;ei&quot;</span><span class="p">,</span> <span class="n">dataset</span><span class="o">=</span><span class="s2">&quot;&quot;</span><span class="p">,</span> <span class="nb">type</span><span class="o">=</span><span class="s2">&quot;&quot;</span><span class="p">,</span>
     297    <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">server</span><span class="p">,</span> <span class="n">public</span><span class="p">,</span> <span class="n">marsclass</span><span class="o">=</span><span class="s2">&quot;EA&quot;</span><span class="p">,</span> <span class="n">dataset</span><span class="o">=</span><span class="s2">&quot;&quot;</span><span class="p">,</span> <span class="nb">type</span><span class="o">=</span><span class="s2">&quot;&quot;</span><span class="p">,</span>
    286298                 <span class="n">levtype</span><span class="o">=</span><span class="s2">&quot;&quot;</span><span class="p">,</span> <span class="n">levelist</span><span class="o">=</span><span class="s2">&quot;&quot;</span><span class="p">,</span> <span class="n">repres</span><span class="o">=</span><span class="s2">&quot;&quot;</span><span class="p">,</span> <span class="n">date</span><span class="o">=</span><span class="s2">&quot;&quot;</span><span class="p">,</span> <span class="n">resol</span><span class="o">=</span><span class="s2">&quot;&quot;</span><span class="p">,</span>
    287299                 <span class="n">stream</span><span class="o">=</span><span class="s2">&quot;&quot;</span><span class="p">,</span> <span class="n">area</span><span class="o">=</span><span class="s2">&quot;&quot;</span><span class="p">,</span> <span class="n">time</span><span class="o">=</span><span class="s2">&quot;&quot;</span><span class="p">,</span> <span class="n">step</span><span class="o">=</span><span class="s2">&quot;&quot;</span><span class="p">,</span> <span class="n">expver</span><span class="o">=</span><span class="s2">&quot;1&quot;</span><span class="p">,</span>
     
    311323<span class="sd">        marsclass : str, optional</span>
    312324<span class="sd">            Characterisation of dataset. E.g. EI (ERA-Interim),</span>
    313 <span class="sd">            E4 (ERA40), OD (Operational archive), ea (ERA5).</span>
    314 <span class="sd">            Default is the ERA-Interim dataset &quot;ei&quot;.</span>
     325<span class="sd">            E4 (ERA40), OD (Operational archive), EA (ERA5).</span>
     326<span class="sd">            Default is the ERA5 dataset &quot;EA&quot;.</span>
    315327
    316328<span class="sd">        dataset : str, optional</span>
     
    582594<div class="viewcode-block" id="MarsRetrieval.data_retrieve"><a class="viewcode-back" href="../api.html#MarsRetrieval.MarsRetrieval.data_retrieve">[docs]</a>    <span class="k">def</span> <span class="nf">data_retrieve</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
    583595        <span class="sd">&#39;&#39;&#39;Submits a MARS retrieval. Depending on the existence of</span>
    584 <span class="sd">        ECMWF Web-API it is submitted via Python or a</span>
     596<span class="sd">        ECMWF Web-API or CDS API it is submitted via Python or a</span>
    585597<span class="sd">        subprocess in the Shell. The parameter for the mars retrieval</span>
    586598<span class="sd">        are taken from the defined class attributes.</span>
     
    605617        <span class="n">attrs</span><span class="p">[</span><span class="s1">&#39;class&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">mclass</span>
    606618
    607         <span class="c1"># prepare target variable as needed for the Web API mode</span>
     619        <span class="c1"># prepare target variable as needed for the Web API or CDS API mode</span>
    608620        <span class="c1"># within the dictionary for full access</span>
    609621        <span class="c1"># as a single variable for public access</span>
     
    628640        <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">server</span><span class="p">:</span>
    629641            <span class="k">try</span><span class="p">:</span>
    630                 <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">public</span><span class="p">:</span>
    631                     <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;RETRIEVE PUBLIC DATA!&#39;</span><span class="p">)</span>
     642                <span class="k">if</span> <span class="n">cds_api</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">server</span><span class="p">,</span> <span class="n">cdsapi</span><span class="o">.</span><span class="n">Client</span><span class="p">):</span>
     643                    <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;RETRIEVE ERA5 WITH CDS API!&#39;</span><span class="p">)</span>
     644                    <span class="bp">self</span><span class="o">.</span><span class="n">server</span><span class="o">.</span><span class="n">retrieve</span><span class="p">(</span><span class="n">_config</span><span class="o">.</span><span class="n">CDS_DATASET</span><span class="p">,</span>
     645                                         <span class="n">attrs</span><span class="p">,</span> <span class="n">target</span><span class="p">)</span>
     646                <span class="k">elif</span> <span class="n">ec_api</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">server</span><span class="p">,</span> <span class="n">ecmwfapi</span><span class="o">.</span><span class="n">ECMWFDataServer</span><span class="p">):</span>
     647                    <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;RETRIEVE PUBLIC DATA (NOT ERA5)!&#39;</span><span class="p">)</span>
    632648                    <span class="bp">self</span><span class="o">.</span><span class="n">server</span><span class="o">.</span><span class="n">retrieve</span><span class="p">(</span><span class="n">attrs</span><span class="p">)</span>
     649                <span class="k">elif</span> <span class="n">ec_api</span> <span class="ow">and</span> <span class="nb">isinstance</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">server</span><span class="p">,</span> <span class="n">ecmwfapi</span><span class="o">.</span><span class="n">ECMWFService</span><span class="p">):</span>
     650                    <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;EXECUTE NON-PUBLIC RETRIEVAL (NOT ERA5)!&#39;</span><span class="p">)</span>
     651                    <span class="bp">self</span><span class="o">.</span><span class="n">server</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">attrs</span><span class="p">,</span> <span class="n">target</span><span class="p">)</span>
    633652                <span class="k">else</span><span class="p">:</span>
    634                     <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;EXECUTE NON-PUBLIC RETRIEVAL!&#39;</span><span class="p">)</span>
    635                     <span class="bp">self</span><span class="o">.</span><span class="n">server</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="n">attrs</span><span class="p">,</span> <span class="n">target</span><span class="p">)</span>
    636             <span class="k">except</span><span class="p">:</span>
    637                 <span class="n">e</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">exc_info</span><span class="p">()[</span><span class="mi">0</span><span class="p">]</span>
    638                 <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;ERROR: &quot;</span><span class="p">,</span> <span class="n">e</span><span class="p">)</span>
    639                 <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;MARS Request failed!&#39;</span><span class="p">)</span>
    640                 <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">public</span> <span class="ow">and</span> <span class="n">os</span><span class="o">.</span><span class="n">stat</span><span class="p">(</span><span class="n">target</span><span class="p">)</span><span class="o">.</span><span class="n">st_size</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
    641                     <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;MARS Request returned no data - &#39;</span>
    642                           <span class="s1">&#39;please check request&#39;</span><span class="p">)</span>
     653                    <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;ERROR:&#39;</span><span class="p">)</span>
     654                    <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;No match for Web API instance!&#39;</span><span class="p">)</span>
    643655                    <span class="k">raise</span> <span class="ne">IOError</span>
    644                 <span class="k">elif</span> <span class="bp">self</span><span class="o">.</span><span class="n">public</span> <span class="ow">and</span> <span class="n">os</span><span class="o">.</span><span class="n">stat</span><span class="p">(</span><span class="n">target</span><span class="p">)</span><span class="o">.</span><span class="n">st_size</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span>
    645                     <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;Public MARS Request returned no data - &#39;</span>
    646                           <span class="s1">&#39;please check request&#39;</span><span class="p">)</span>
    647                     <span class="k">raise</span> <span class="ne">IOError</span>
    648                 <span class="k">else</span><span class="p">:</span>
    649                     <span class="k">raise</span> <span class="ne">IOError</span>
    650         <span class="c1"># MARS request via extra process in shell</span>
     656            <span class="k">except</span> <span class="ne">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
     657                <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;</span><span class="se">\n\n</span><span class="s1">MARS Request failed!&#39;</span><span class="p">)</span>
     658                <span class="nb">print</span><span class="p">(</span><span class="n">e</span><span class="p">)</span>
     659                <span class="n">tb</span> <span class="o">=</span> <span class="n">sys</span><span class="o">.</span><span class="n">exc_info</span><span class="p">()[</span><span class="mi">2</span><span class="p">]</span>
     660                <span class="nb">print</span><span class="p">(</span><span class="n">traceback</span><span class="o">.</span><span class="n">format_exc</span><span class="p">())</span>
     661                <span class="n">sys</span><span class="o">.</span><span class="n">exit</span><span class="p">()</span>
     662
     663        <span class="c1"># MARS request via call in shell</span>
    651664        <span class="k">else</span><span class="p">:</span>
    652665            <span class="n">request_str</span> <span class="o">=</span> <span class="s1">&#39;ret&#39;</span>
     
    654667                <span class="n">request_str</span> <span class="o">=</span> <span class="n">request_str</span> <span class="o">+</span> <span class="s1">&#39;,&#39;</span> <span class="o">+</span> <span class="n">key</span> <span class="o">+</span> <span class="s1">&#39;=&#39;</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="n">value</span><span class="p">)</span>
    655668            <span class="n">request_str</span> <span class="o">+=</span> <span class="s1">&#39;,target=&quot;&#39;</span> <span class="o">+</span> <span class="n">target</span> <span class="o">+</span> <span class="s1">&#39;&quot;&#39;</span>
    656             <span class="n">p</span> <span class="o">=</span> <span class="n">subprocess</span><span class="o">.</span><span class="n">Popen</span><span class="p">([</span><span class="s1">&#39;mars&#39;</span><span class="p">],</span>
     669            <span class="n">p</span> <span class="o">=</span> <span class="n">subprocess</span><span class="o">.</span><span class="n">Popen</span><span class="p">([</span><span class="s1">&#39;mars&#39;</span><span class="p">,</span> <span class="s1">&#39;-p&#39;</span><span class="p">],</span>
    657670                                 <span class="n">stdin</span><span class="o">=</span><span class="n">subprocess</span><span class="o">.</span><span class="n">PIPE</span><span class="p">,</span>
    658671                                 <span class="n">stdout</span><span class="o">=</span><span class="n">subprocess</span><span class="o">.</span><span class="n">PIPE</span><span class="p">,</span>
     
    682695  <div role="contentinfo">
    683696    <p>
    684         &copy; Copyright 2018, Anne Philipp and Leopold Haimberger
     697        &copy; Copyright 2018, Anne Philipp and Leopold Haimberger.
    685698
    686699    </p>
     
    701714 
    702715
    703    
    704    
    705       <script type="text/javascript" id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
    706         <script type="text/javascript" src="../_static/jquery.js"></script>
    707         <script type="text/javascript" src="../_static/underscore.js"></script>
    708         <script type="text/javascript" src="../_static/doctools.js"></script>
    709    
     716    <script type="text/javascript">
     717        var DOCUMENTATION_OPTIONS = {
     718            URL_ROOT:'../',
     719            VERSION:'7.1 alpha',
     720            LANGUAGE:'None',
     721            COLLAPSE_INDEX:false,
     722            FILE_SUFFIX:'.html',
     723            HAS_SOURCE:  true,
     724            SOURCELINK_SUFFIX: '.txt'
     725        };
     726    </script>
     727      <script type="text/javascript" src="../_static/jquery.js"></script>
     728      <script type="text/javascript" src="../_static/underscore.js"></script>
     729      <script type="text/javascript" src="../_static/doctools.js"></script>
     730      <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
    710731
    711732 
Note: See TracChangeset for help on using the changeset viewer.
hosted by ZAMG