Changeset 5920b7f in flex_extract.git for Documentation/html/Installation/local.html


Ignore:
Timestamp:
May 30, 2020, 9:35:50 AM (4 years ago)
Author:
anphi <anne.philipp@…>
Branches:
master, ctbto, dev
Children:
a9d33f6
Parents:
550435b
Message:

New compilation of Sphinx after extensive review in language/content/understanding

File:
1 edited

Legend:

Unmodified
Added
Removed

  • Documentation/html/Installation/local.html

    rb1674ed r5920b7f  
    9292<li class="toctree-l1 current"><a class="reference internal" href="../installation.html">Installation</a><ul class="current">
    9393<li class="toctree-l2"><a class="reference internal" href="../installation.html#registration-at-ecmwf">Registration at ECMWF</a></li>
    94 <li class="toctree-l2"><a class="reference internal" href="../installation.html#licence-agreement-for-public-datasets">Licence agreement for public datasets</a></li>
     94<li class="toctree-l2"><a class="reference internal" href="../installation.html#accept-licences-for-public-datasets">Accept licences for public datasets</a></li>
    9595<li class="toctree-l2"><a class="reference internal" href="../installation.html#download-flex-extract">Download <code class="docutils literal notranslate"><span class="pre">flex_extract</span></code></a></li>
    9696<li class="toctree-l2"><a class="reference internal" href="../installation.html#dependencies">Dependencies</a></li>
    … …  
    100100<li class="toctree-l3 current"><a class="current reference internal" href="#">Local mode installation</a><ul>
    101101<li class="toctree-l4"><a class="reference internal" href="#local-mode-dependencies">Local mode - dependencies</a></li>
    102 <li class="toctree-l4"><a class="reference internal" href="#prepare-local-environment">Prepare local environment</a></li>
    103 <li class="toctree-l4"><a class="reference internal" href="#test-local-environment">Test local environment</a></li>
     102<li class="toctree-l4"><a class="reference internal" href="#preparing-the-local-environment">Preparing the local environment</a></li>
     103<li class="toctree-l4"><a class="reference internal" href="#testing-the-local-environment">Testing the local environment</a></li>
    104104<li class="toctree-l4"><a class="reference internal" href="#local-installation">Local installation</a></li>
    105105</ul>
    … …  
    193193<span id="ref-req-local"></span><span id="ref-local-mode"></span><h2>Local mode - dependencies<a class="headerlink" href="#local-mode-dependencies" title="Permalink to this headline">¶</a></h2>
    194194<p>The installation is the same for the access modes <strong>member</strong> and <strong>public</strong>.</p>
    195 <p>The environment on your local system has to provide these software packages
     195<p>The environment on your local system has to provide the following software packages
    196196and libraries, since the preparation of the extraction and the post-processing is done on the local machine:</p>
    197197<table class="docutils align-default">
    … …  
    204204<td><p>Fortran part</p></td>
    205205</tr>
    206 <tr class="row-even"><td><ul class="simple">
     206<tr class="row-even"><td><ol class="arabic simple">
    207207<li><p><a class="reference external" href="https://www.python.org/downloads/">Python3</a></p></li>
    208208<li><p><a class="reference external" href="http://www.numpy.org/">numpy</a></p></li>
    … …  
    211211<li><p><a class="reference external" href="https://confluence.ecmwf.int/display/WEBAPI/ECMWF+Web+API+Home">ecmwf-api-client</a> (everything except ERA5)</p></li>
    212212<li><p><a class="reference external" href="https://cds.climate.copernicus.eu/api-how-to">cdsapi</a> (just for ERA5 and member user)</p></li>
    213 </ul>
     213</ol>
    214214</td>
    215 <td><ul class="simple">
     215<td><ol class="arabic simple">
    216216<li><p><a class="reference external" href="https://gcc.gnu.org/wiki/GFortran">gfortran</a></p></li>
    217217<li><p><a class="reference external" href="http://www.fftw.org">fftw3</a></p></li>
    218218<li><p><a class="reference external" href="https://software.ecmwf.int/wiki/display/ECC">eccodes</a></p></li>
    219219<li><p><a class="reference external" href="https://software.ecmwf.int/wiki/display/EMOS/Emoslib">emoslib</a></p></li>
    220 </ul>
     220</ol>
    221221</td>
    222222</tr>
    … …  
    224224</table>
    225225</div>
    226 <div class="section" id="prepare-local-environment">
    227 <span id="ref-prep-local"></span><h2>Prepare local environment<a class="headerlink" href="#prepare-local-environment" title="Permalink to this headline">¶</a></h2>
    228 <p>The easiest way to install all required packages is to use the package management system of your Linux distribution  which requires admin rights.
     226<div class="section" id="preparing-the-local-environment">
     227<span id="ref-prep-local"></span><h2>Preparing the local environment<a class="headerlink" href="#preparing-the-local-environment" title="Permalink to this headline">¶</a></h2>
     228<p>The easiest way to install all required packages is to use the package management system of your Linux distribution which requires admin rights.
    229229The installation was tested on a <em>Debian GNU/Linux buster</em> and an <em>Ubuntu 18.04 Bionic Beaver</em> system.</p>
    230 <div class="highlight-sh notranslate"><div class="highlight"><pre><span></span><span class="c1"># On a Debian or Debian-derived sytem (e. g. Ubuntu) system you may use the following commands (or equivalent commands of your preferred package manager):</span>
    231 <span class="c1"># (if not already available):</span>
     230<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span><span class="c1"># On a Debian or Debian-derived (e. g. Ubuntu) system,</span>
     231<span class="c1"># you may use the following commands (or equivalent commands of your preferred package manager):</span>
     232<span class="c1"># (if respective packages are not already available):</span>
    232233 apt-get install python3 <span class="o">(</span>usually already available on GNU/Linux systems<span class="o">)</span>
    233234 apt-get install python3-eccodes
    … …  
    238239 apt-get install libeccodes-dev
    239240 apt-get install libemos-dev
    240 <span class="c1"># Some of these packages will pull in further packages as dependencies. This is fine, and some are even needed by ``flex_extract&#39;&#39;.</span>
    241 
    242 
    243 <span class="c1"># As currently the CDS and ECMWF API packages are not available as Debian packages, they need to be installed outside of the Debian (Ubuntu etc.) package management system. The recommended way is:</span>
     241<span class="c1"># Some of these packages will pull in further packages as dependencies.</span>
     242<span class="c1"># This is fine, and some are even needed by ``flex_extract&#39;&#39;.</span>
     243
     244<span class="c1"># As currently the CDS and ECMWF API packages are not available as Debian packages,</span>
     245<span class="c1"># they need to be installed outside of the Debian (Ubuntu etc.) package management system.</span>
     246<span class="c1"># The recommended way is:</span>
    244247 apt-get install pip
    245248 pip install cdsapi
    … …  
    249252<div class="admonition note">
    250253<p class="admonition-title">Note</p>
    251 <p>In case you would like to use Anaconda Python we recommend you follow the installation instructions of
    252 <a class="reference external" href="https://docs.anaconda.com/anaconda/install/linux/">Anaconda Python Installation for Linux</a> and then install the
    253 <code class="docutils literal notranslate"><span class="pre">eccodes</span></code> package from <code class="docutils literal notranslate"><span class="pre">conda</span></code> with:</p>
     254<p>If you are using Anaconda Python, we recommend to follow the installation instructions of
     255<a class="reference external" href="https://docs.anaconda.com/anaconda/install/linux/">Anaconda Python Installation for Linux</a>
     256and then install the <code class="docutils literal notranslate"><span class="pre">eccodes</span></code> package from <code class="docutils literal notranslate"><span class="pre">conda</span></code> with:</p>
    254257<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>conda install conda-forge::python-eccodes
    255258</pre></div>
    256259</div>
    257260</div>
    258 <p>The CDS API (cdsapi) is required for ERA5 data and the ECMWF Web API (ecmwf-api-client) for all other public datasets.</p>
     261<p>The CDS API (<code class="docutils literal notranslate"><span class="pre">cdsapi</span></code>) is required for ERA5 data and the ECMWF Web API (ecmwf-api-client) for all other public datasets.</p>
    259262<div class="admonition note">
    260263<p class="admonition-title">Note</p>
    261 <p>Since <strong>public users</strong> currently don’t have access to the full <em>ERA5</em> dataset they can skip the installation of the <code class="docutils literal notranslate"><span class="pre">CDS</span> <span class="pre">API</span></code>.</p>
    262 </div>
    263 <p>Both user groups have to provide keys with their credentials for the Web API’s in their home directory. Therefore, follow these instructions:</p>
     264<p>Since <strong>public users</strong> currently don’t have access to the full <em>ERA5</em> dataset, they can skip the installation of the CDS API.</p>
     265</div>
     266<p>Both user groups have to provide keys with their credentials for the Web APIs in their home directory, following these instructions:</p>
    264267<dl class="simple">
    265 <dt>ECMWF Web API:</dt><dd><p>Go to <a class="reference external" href="https://confluence.ecmwf.int//display/WEBAPI/Access+MARS">MARS access</a> website and log in with your credentials. Afterwards, on this site in section “Install ECMWF KEY” the key for the ECMWF Web API should be listed. Please follow the instructions in this section under 1 (save the key in a file <cite>.ecmwfapirc</cite> in your home directory).</p>
     268<dt>ECMWF Web API:</dt><dd><p>Go to the <a class="reference external" href="https://confluence.ecmwf.int//display/WEBAPI/Access+MARS">MARS access</a> website and log in with your credentials. Afterwards, go to the section “Install ECMWF KEY”, where the key for the ECMWF Web API should be listed. Please follow the instructions in this section under 1 (save the key in a file <code class="docutils literal notranslate"><span class="pre">.ecmwfapirc</span></code> in your home directory).</p>
    266269</dd>
    267 <dt>CDS API:</dt><dd><p>Go to <a class="reference external" href="https://cds.climate.copernicus.eu/user/register">CDS API registration</a> and register there too. Log in at the <a class="reference external" href="https://cds.climate.copernicus.eu/api-how-to">cdsapi</a> website and follow the instructions at section “Install the CDS API key” to save your credentials in a <cite>.cdsapirc</cite> file.</p>
     270<dt>CDS API:</dt><dd><p>Go to <a class="reference external" href="https://cds.climate.copernicus.eu/user/register">CDS API registration</a> and register there, too. Log in on the <a class="reference external" href="https://cds.climate.copernicus.eu/api-how-to">cdsapi</a> website and follow the instructions in the section “Install the CDS API key” to save your credentials in file <code class="docutils literal notranslate"><span class="pre">.cdsapirc</span></code>.</p>
    268271</dd>
    269272</dl>
    270273</div>
    271 <div class="section" id="test-local-environment">
    272 <span id="ref-test-local"></span><h2>Test local environment<a class="headerlink" href="#test-local-environment" title="Permalink to this headline">¶</a></h2>
    273 <p>Check the availability of the python packages by typing <code class="docutils literal notranslate"><span class="pre">python3</span></code> in a terminal window and run the <code class="docutils literal notranslate"><span class="pre">import</span></code> commands in the python shell. If there are no error messages, you succeeded in setting up the environment.</p>
    274 <div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># check in python3 console</span>
    275 <span class="kn">import</span> <span class="nn">eccodes</span>
    276 <span class="kn">import</span> <span class="nn">genshi</span>
    277 <span class="kn">import</span> <span class="nn">numpy</span>
    278 <span class="kn">import</span> <span class="nn">cdsapi</span>
    279 <span class="kn">import</span> <span class="nn">ecmwfapi</span>
    280 </pre></div>
    281 </div>
    282 <div class="section" id="test-the-web-api-s">
    283 <h3>Test the Web API’s<a class="headerlink" href="#test-the-web-api-s" title="Permalink to this headline">¶</a></h3>
     274<div class="section" id="testing-the-local-environment">
     275<span id="ref-test-local"></span><h2>Testing the local environment<a class="headerlink" href="#testing-the-local-environment" title="Permalink to this headline">¶</a></h2>
     276<p>Check the availability of the python packages by typing <code class="docutils literal notranslate"><span class="pre">python3</span></code> in a terminal window and run the <code class="docutils literal notranslate"><span class="pre">import</span></code> commands in the python shell:
     277.. code-block:: python</p>
     278<blockquote>
     279<div><p># check in python3 console
     280import eccodes
     281import genshi
     282import numpy
     283import cdsapi
     284import ecmwfapi</p>
     285</div></blockquote>
     286<p>If there are no error messages, you succeeded in setting up the environment.</p>
     287<div class="section" id="testing-the-web-apis">
     288<h3>Testing the Web APIs<a class="headerlink" href="#testing-the-web-apis" title="Permalink to this headline">¶</a></h3>
    284289<p>You can start very simple test retrievals for both Web APIs to be sure that everything works. This is recommended to minimise the range of possible errors using <code class="docutils literal notranslate"><span class="pre">flex_extract</span></code> later on.</p>
    285290<div class="section" id="ecmwf-web-api">
    … …  
    291296</colgroup>
    292297<tbody>
    293 <tr class="row-odd"><td><p>Please use this piece of Python code for <strong>Member user</strong>:</p></td>
    294 <td><p>Please use this piece of Python code for <strong>Public user</strong>:</p></td>
     298<tr class="row-odd"><td><p>Please use this Python code snippet as a <strong>Member user</strong>:</p></td>
     299<td><p>Please use this Python code snippet as a <strong>Public user</strong>:</p></td>
    295300</tr>
    296301<tr class="row-even"><td><div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">ecmwfapi</span> <span class="kn">import</span> <span class="n">ECMWFService</span>
    … …  
    341346<h4>CDS API<a class="headerlink" href="#cds-api" title="Permalink to this headline">¶</a></h4>
    342347<p>Extraction of ERA5 data via CDS API might take time as currently there is a high demand for ERA5 data. Therefore, as a simple test for the API just retrieve pressure-level data (even if that is NOT what we need for FLEXPART), as they are stored on disk and don’t need to be retrieved from MARS (which is the time-consuming action):</p>
    343 <p>Please use this piece of Python code to retrieve a small sample of <em>ERA5</em> pressure levels:</p>
     348<p>Please use the following Python code snippet to retrieve a small sample of <em>ERA5</em> pressure level data:</p>
    344349<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">cdsapi</span>
    345350
    … …  
    361366</div>
    362367<p>If you know that your CDS API works, you can try to extract some data from MARS.</p>
    363 <p>Please use this piece of Python code to retrieve a small <em>ERA5</em> data sample as a <strong>member-state user</strong>! The <strong>Public user</strong> do not have access to the full <em>ERA5</em> dataset!</p>
     368<p>Please use the following Python code snippet to retrieve a small <em>ERA5</em> data sample as a <strong>member-state user</strong>! The <strong>Public user</strong> do not have access to the full <em>ERA5</em> dataset!</p>
    364369<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">cdsapi</span>
    365370
    … …  
    389394<div class="section" id="local-installation">
    390395<span id="ref-install-local"></span><h2>Local installation<a class="headerlink" href="#local-installation" title="Permalink to this headline">¶</a></h2>
    391 <p>First prepare the Fortran <code class="docutils literal notranslate"><span class="pre">makefile</span></code> for your environment and set it in the <code class="docutils literal notranslate"><span class="pre">setup.sh</span></code> script. (See section <a class="reference internal" href="../Documentation/Input/fortran_makefile.html#ref-convert"><span class="std std-ref">Fortran Makefile</span></a> for more information.)
    392 <code class="docutils literal notranslate"><span class="pre">flex_extract</span></code> comes with two <code class="docutils literal notranslate"><span class="pre">makefiles</span></code> prepared for the <code class="docutils literal notranslate"><span class="pre">gfortran</span></code> compiler. One for the normal use <code class="docutils literal notranslate"><span class="pre">makefile_fast</span></code> and one for debugging <code class="docutils literal notranslate"><span class="pre">makefile_debug</span></code> which is usually only resonable for developers.</p>
    393 <p>They assume that <code class="docutils literal notranslate"><span class="pre">eccodes</span></code> and <code class="docutils literal notranslate"><span class="pre">emoslib</span></code> are installed as distribution packages and can be found at <code class="docutils literal notranslate"><span class="pre">flex_extract_vX.X/Source/Fortran</span></code>, where <code class="docutils literal notranslate"><span class="pre">vX.X</span></code> should be substituted with the current version number.</p>
     396<p>First, adapt the Fortran <code class="docutils literal notranslate"><span class="pre">makefile</span></code> for your environment (if necessary) and insert it into <code class="docutils literal notranslate"><span class="pre">setup.sh</span></code> script (see <a class="reference internal" href="../Documentation/Input/fortran_makefile.html#ref-convert"><span class="std std-ref">Fortran Makefile</span></a> for more information).
     397They can be found at <code class="docutils literal notranslate"><span class="pre">flex_extract_vX.X/Source/Fortran/</span></code>, where <code class="docutils literal notranslate"><span class="pre">vX.X</span></code> should be substituted by the current flex_extract version number.</p>
    394398<div class="admonition caution">
    395399<p class="admonition-title">Caution</p>
    … …  
    397401<code class="docutils literal notranslate"><span class="pre">makefiles</span></code> if other than standard paths are used.</p>
    398402</div>
    399 <p>So starting from the root directory of <code class="docutils literal notranslate"><span class="pre">flex_extract</span></code>,
    400 go to the <code class="docutils literal notranslate"><span class="pre">Fortran</span></code> source directory and open the <code class="docutils literal notranslate"><span class="pre">makefile</span></code> of your
    401 choice to modify with an editor of your choice. We use the <code class="docutils literal notranslate"><span class="pre">nedit</span></code> in this case.</p>
     403<p>Thus, go to the <code class="docutils literal notranslate"><span class="pre">Fortran</span></code> source directory and open the <code class="docutils literal notranslate"><span class="pre">makefile</span></code> of your
     404choice, and check / modify with an editor of your choice:</p>
    402405<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="nb">cd</span> flex_extract_vX.X/Source/Fortran
    403406nedit makefile_fast
    404407</pre></div>
    405408</div>
    406 <p>Edit the paths to the <code class="docutils literal notranslate"><span class="pre">eccodes</span></code> library on your local machine.</p>
     409<p>Set the paths to the <code class="docutils literal notranslate"><span class="pre">eccodes</span></code> library on your local machine, if necessary.</p>
    407410<div class="admonition caution">
    408411<p class="admonition-title">Caution</p>
    … …  
    417420<p>to find out the path to the <code class="docutils literal notranslate"><span class="pre">eccodes</span></code> library.</p>
    418421</div>
    419 <p>Substitute these paths in the <code class="docutils literal notranslate"><span class="pre">makefile</span></code> for parameters <strong>ECCODES_INCLUDE_DIR</strong>
    420 and <strong>ECCODES_LIB</strong> and save it.</p>
    421 <div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="c1"># these are the paths on a current Debian 10 Testing system (May 2019)</span>
     422<p>Assign these paths to the parameters <strong>ECCODES_INCLUDE_DIR</strong>
     423and <strong>ECCODES_LIB</strong> in the makefile, and save it.</p>
     424<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="c1"># these are the paths on Debian Buster:</span>
    422425<span class="nv">ECCODES_INCLUDE_DIR</span><span class="o">=</span>/usr/lib/x86_64-linux-gnu/fortran/gfortran-mod-15/
    423426<span class="nv">ECCODES_LIB</span><span class="o">=</span> -L/usr/lib -leccodes_f90 -leccodes -lm
    … …  
    425428</div>
    426429<p>The Fortran program called <code class="docutils literal notranslate"><span class="pre">calc_etadot</span></code> will be compiled during the
    427 installation process.Therefore the name of the <code class="docutils literal notranslate"><span class="pre">makefile</span></code> to be used needs to be given in  <code class="docutils literal notranslate"><span class="pre">setup.sh</span></code>.</p>
     430installation process. Therefore, the name of the <code class="docutils literal notranslate"><span class="pre">makefile</span></code> to be used needs to be given in  <code class="docutils literal notranslate"><span class="pre">setup.sh</span></code>.</p>
    428431<p>In the root directory of <code class="docutils literal notranslate"><span class="pre">flex_extract</span></code>, open the <code class="docutils literal notranslate"><span class="pre">setup.sh</span></code> script
    429 and adapt the installation parameters in the section labelled with
    430 “AVAILABLE COMMANDLINE ARGUMENTS TO SET” like shown below.</p>
     432with an editor and adapt the installation parameters in the section labelled with
     433“AVAILABLE COMMANDLINE ARGUMENTS TO SET” as shown below:</p>
    431434<div class="literal-block-wrapper docutils container" id="setup-sh">
    432435<div class="code-block-caption"><span class="caption-text">‘Example settings for a local installation.’</span><a class="headerlink" href="#setup-sh" title="Permalink to this code">¶</a></div>
Note: See TracChangeset for help on using the changeset viewer.
hosted by ZAMG