Ignore:
Timestamp:
Jul 9, 2020, 8:13:25 AM (4 years ago)
Author:
anphi <anne.philipp@…>
Branches:
master, ctbto, dev
Children:
82564d8
Parents:
3e13e02 (diff), 6931f61 (diff)
Note: this is a merge changeset, the changes displayed below correspond to the merge itself.
Use the (diff) links above to see all the changes relative to each parent.
Message:

merge of language editing branch into master

File:
1 edited

Legend:

Unmodified
Added
Removed

  • Documentation/html/_sources/Installation/local.rst.txt

    rb1674ed r6931f61  
    1111         
    1212   
    13 .. _Python 3: https://docs.python.org/3/
    14 .. _Python3: https://www.python.org/downloads/
    15 .. _Anaconda Python3: https://www.anaconda.com/distribution/#download-section
    16 
     13.. _Python3: https://www.python.org/
     14.. _Anaconda Python3: https://www.anaconda.com/distribution/
    1715.. _numpy: http://www.numpy.org/
    1816.. _ecmwf-api-client: https://confluence.ecmwf.int/display/WEBAPI/ECMWF+Web+API+Home
    1917.. _cdsapi: https://cds.climate.copernicus.eu/api-how-to
    2018.. _genshi: https://genshi.edgewall.org/
    21 .. _eccodes for python: https://packages.debian.org/sid/python3-eccodes
    22 .. _eccodes for conda: https://anaconda.org/conda-forge/eccodes
     19.. _eccodes for python: https://pypi.org/project/eccodes/
     20.. _eccodes for conda: https://anaconda.org/conda-forge/python-eccodes
    2321.. _gfortran: https://gcc.gnu.org/wiki/GFortran
    2422.. _fftw3: http://www.fftw.org
    … …  
    3028.. _ECMWF ectrans site: https://confluence.ecmwf.int/display/ECAC/Unattended+file+transfer+-+ectrans
    3129.. _ECaccess Presentation: https://confluence.ecmwf.int/download/attachments/45759146/ECaccess.pdf
    32 .. _ECMWF's instructions on gateway server: https://confluence.ecmwf.int/display/ECAC/ECaccess+Home
     30.. _ECMWF instructions on gateway servers: https://confluence.ecmwf.int/display/ECAC/ECaccess+Home
    3331.. _Computing Representative: https://www.ecmwf.int/en/about/contact-us/computing-representatives
    3432.. _MARS access: https://confluence.ecmwf.int//display/WEBAPI/Access+MARS
    35 
    3633.. _download section: https://www.flexpart.eu/downloads
    3734
    38  
    39    
    40    
    41    
     35     
    4236.. _ref-local-mode:
    43 
    44 
    4537
    4638.. _ref-req-local:
    … …  
    5143The installation is the same for the access modes **member** and **public**.
    5244
    53 The environment on your local system has to provide these software packages
     45The environment on your local system has to provide the following software packages
    5446and libraries, since the preparation of the extraction and the post-processing is done on the local machine:
    5547
    56 +------------------------------------------------+-----------------+
    57 |  Python part                                   | Fortran part    |
    58 +------------------------------------------------+-----------------+
    59 | * `Python3`_                                   | * `gfortran`_   |
    60 | * `numpy`_                                     | * `fftw3`_      |
    61 | * `genshi`_                                    | * `eccodes`_    |
    62 | * `eccodes for python`_                        | * `emoslib`_    |
    63 | * `ecmwf-api-client`_ (everything except ERA5) |                 |
    64 | * `cdsapi`_ (just for ERA5 and member user)    |                 |
    65 +------------------------------------------------+-----------------+
     48+------------------------------------------------+----------------+
     49|  Python code                                   | Fortran code   |
     50+------------------------------------------------+----------------+
     51| * `Python3`_                                   | * `gfortran`_  |
     52| * `numpy`_                                     | * `fftw3`_     |
     53| * `genshi`_                                    | * `eccodes`_   |
     54| * `eccodes for python`_                        | * `emoslib`_   |
     55| * `ecmwf-api-client`_ (everything except ERA5) |                |
     56| * `cdsapi`_ (just for ERA5 and member user)    |                |
     57+------------------------------------------------+----------------+
    6658
    6759
    6860.. _ref-prep-local:
    6961
    70 Prepare local environment
    71 =========================
    72 
    73 The easiest way to install all required packages is to use the package management system of your Linux distribution  which requires admin rights.
     62Preparing the local environment
     63===============================
     64
     65The easiest way to install all required packages is to use the package management system of your Linux distribution which requires admin rights.
    7466The installation was tested on a *Debian GNU/Linux buster* and an *Ubuntu 18.04 Bionic Beaver* system.
    7567
    7668.. code-block:: sh
    7769
    78   # On a Debian or Debian-derived sytem (e. g. Ubuntu) system you may use the following commands (or equivalent commands of your preferred package manager):
    79   # (if not already available):
     70  # On a Debian or Debian-derived (e. g. Ubuntu) system,
     71  # you may use the following commands (or equivalent commands of your preferred package manager):
     72  # (if respective packages are not already available):
    8073   apt-get install python3 (usually already available on GNU/Linux systems)
    8174   apt-get install python3-eccodes
    … …  
    8679   apt-get install libeccodes-dev
    8780   apt-get install libemos-dev
    88   # Some of these packages will pull in further packages as dependencies. This is fine, and some are even needed by ``flex_extract''.
    89  
    90 
    91   # As currently the CDS and ECMWF API packages are not available as Debian packages, they need to be installed outside of the Debian (Ubuntu etc.) package management system. The recommended way is:
     81  # Some of these packages will pull in further packages as dependencies.
     82  # This is fine, and some are even needed by ``flex_extract''.
     83
     84  # As currently the CDS and ECMWF API packages are not available as Debian packages,
     85  # they need to be installed outside of the Debian (Ubuntu etc.) package management system.
     86  # The recommended way is:
    9287   apt-get install pip
    9388   pip install cdsapi
    … …  
    9691.. note::
    9792
    98     In case you would like to use Anaconda Python we recommend you follow the installation instructions of
    99     `Anaconda Python Installation for Linux <https://docs.anaconda.com/anaconda/install/linux/>`_ and then install the
    100     ``eccodes`` package from ``conda`` with:
     93    If you are using Anaconda Python, we recommend to follow the installation instructions of
     94    `Anaconda Python Installation for Linux <https://docs.anaconda.com/anaconda/install/linux/>`_
     95    and then install the ``eccodes`` package from ``conda`` with:
    10196
    10297    .. code-block:: bash
    … …  
    10499       conda install conda-forge::python-eccodes   
    105100   
    106 The CDS API (cdsapi) is required for ERA5 data and the ECMWF Web API (ecmwf-api-client) for all other public datasets.   
     101The CDS API (``cdsapi``) is required for ERA5 data and the ECMWF Web API (ecmwf-api-client) for all other public datasets.   
    107102   
    108103.. note::
    109104
    110     Since **public users** currently don't have access to the full *ERA5* dataset they can skip the installation of the ``CDS API``.
    111 
    112 Both user groups have to provide keys with their credentials for the Web API's in their home directory. Therefore, follow these instructions:
     105    Since **public users** currently don't have access to the full *ERA5* dataset, they can skip the installation of the CDS API.
     106
     107Both user groups have to provide keys with their credentials for the Web APIs in their home directory, following these instructions:
    113108       
    114109ECMWF Web API:
    115    Go to `MARS access`_ website and log in with your credentials. Afterwards, on this site in section "Install ECMWF KEY" the key for the ECMWF Web API should be listed. Please follow the instructions in this section under 1 (save the key in a file `.ecmwfapirc` in your home directory).
     110   Go to the `MARS access`_ website and log in with your credentials. Afterwards, go to the section "Install ECMWF KEY", where the key for the ECMWF Web API should be listed. Please follow the instructions in this section under 1 (save the key in a file ``.ecmwfapirc`` in your home directory).
    116111     
    117112CDS API:
    118    Go to `CDS API registration`_ and register there too. Log in at the `cdsapi`_ website and follow the instructions at section "Install the CDS API key" to save your credentials in a `.cdsapirc` file.
     113   Go to `CDS API registration`_ and register there, too. Log in on the `cdsapi`_ website and follow the instructions in the section "Install the CDS API key" to save your credentials in file ``.cdsapirc``.
    119114
    120115   
    121116.. _ref-test-local:
    122117   
    123 Test local environment
    124 ======================
    125 
    126 Check the availability of the python packages by typing ``python3`` in a terminal window and run the ``import`` commands in the python shell. If there are no error messages, you succeeded in setting up the environment.
     118Testing the local environment
     119=============================
     120
     121Check the availability of the python packages by typing ``python3`` in a terminal window and run the ``import`` commands in the python shell:
    127122
    128123.. code-block:: python
    … …  
    135130   import ecmwfapi
    136131   
    137 
    138 
    139 Test the Web API's
    140 ------------------
     132If there are no error messages, you succeeded in setting up the environment.
     133
     134
     135Testing the Web APIs
     136--------------------
    141137
    142138You can start very simple test retrievals for both Web APIs to be sure that everything works. This is recommended to minimise the range of possible errors using ``flex_extract`` later on.
    … …  
    148144
    149145+----------------------------------------------------------+----------------------------------------------------------+
    150 |Please use this piece of Python code for **Member user**: |Please use this piece of Python code for **Public user**: |
     146|Please use this Python code snippet as a **Member user**: |Please use this Python code snippet as a **Public user**: |
    151147+----------------------------------------------------------+----------------------------------------------------------+
    152148|.. code-block:: python                                    |.. code-block:: python                                    |
    … …  
    178174Extraction of ERA5 data via CDS API might take time as currently there is a high demand for ERA5 data. Therefore, as a simple test for the API just retrieve pressure-level data (even if that is NOT what we need for FLEXPART), as they are stored on disk and don't need to be retrieved from MARS (which is the time-consuming action):
    179175
    180 Please use this piece of Python code to retrieve a small sample of *ERA5* pressure levels:
     176Please use the following Python code snippet to retrieve a small sample of *ERA5* pressure level data:
    181177
    182178.. code-block:: python
    … …  
    204200.. **Member-state user**
    205201
    206 Please use this piece of Python code to retrieve a small *ERA5* data sample as a **member-state user**! The **Public user** do not have access to the full *ERA5* dataset!
     202Please use the following Python code snippet to retrieve a small *ERA5* data sample as a **member-state user**! The **Public user** do not have access to the full *ERA5* dataset!
    207203
    208204.. code-block:: python
    … …  
    262258
    263259
    264 
    265260.. _ref-install-local:
    266261
    … …  
    268263==================
    269264
    270 First prepare the Fortran ``makefile`` for your environment and set it in the ``setup.sh`` script. (See section :ref:`Fortran Makefile <ref-convert>` for more information.)
    271 ``flex_extract`` comes with two ``makefiles`` prepared for the ``gfortran`` compiler. One for the normal use ``makefile_fast`` and one for debugging ``makefile_debug`` which is usually only resonable for developers.
     265   
     266The Fortran program called ``calc_etadot`` will be compiled during the
     267installation process. A suitable makefile (``makefile_local_gfortran``) for the compilation is set by default.
     268This may be overwritten by the ``MAKEFILE`` parameter in ``setup.sh``.
     269
     270However, you may have to adapt the makefile for your environment (the current default makefile works on Debian stretch and similar GNU/Linux distributions). If you use a new name for it, you will have to insert it into ``setup.sh``
     271For details on the makefile and how to adapt them, see :ref:`Fortran Makefile <ref-convert>`.
     272
    272273 
    273 They assume that ``eccodes`` and ``emoslib`` are installed as distribution packages and can be found at ``flex_extract_vX.X/Source/Fortran``, where ``vX.X`` should be substituted with the current version number.
    274 
    275 .. caution::   
    276    It is necessary to adapt **ECCODES_INCLUDE_DIR** and **ECCODES_LIB** in these
    277    ``makefiles`` if other than standard paths are used.
    278 
    279 So starting from the root directory of ``flex_extract``,
    280 go to the ``Fortran`` source directory and open the ``makefile`` of your
    281 choice to modify with an editor of your choice. We use the ``nedit`` in this case.
    282 
    283 .. code-block:: bash
    284 
    285    cd flex_extract_vX.X/Source/Fortran
    286    nedit makefile_fast
    287  
    288 Edit the paths to the ``eccodes`` library on your local machine.
    289 
    290 
    291 .. caution::
    292    This can vary from system to system.
    293    It is suggested to use a command like
    294 
    295    .. code-block:: bash
    296 
    297       # for the ECCODES_INCLUDE_DIR path do:
    298       $ dpkg -L libeccodes-dev | grep eccodes.mod
    299       # for the ECCODES_LIB path do:
    300       $ dpkg -L libeccodes-dev | grep libeccodes.so
    301      
    302    to find out the path to the ``eccodes`` library.
    303    
    304 Substitute these paths in the ``makefile`` for parameters **ECCODES_INCLUDE_DIR**
    305 and **ECCODES_LIB** and save it.
    306 
    307 .. code-block:: bash
    308 
    309    # these are the paths on a current Debian 10 Testing system (May 2019)
    310    ECCODES_INCLUDE_DIR=/usr/lib/x86_64-linux-gnu/fortran/gfortran-mod-15/
    311    ECCODES_LIB= -L/usr/lib -leccodes_f90 -leccodes -lm 
    312    
    313    
    314 The Fortran program called ``calc_etadot`` will be compiled during the
    315 installation process.Therefore the name of the ``makefile`` to be used needs to be given in  ``setup.sh``.
    316 
    317274In the root directory of ``flex_extract``, open the ``setup.sh`` script
    318 and adapt the installation parameters in the section labelled with
    319 "AVAILABLE COMMANDLINE ARGUMENTS TO SET" like shown below.
     275with an editor and adapt the installation parameters in the section labelled with
     276"AVAILABLE COMMANDLINE ARGUMENTS TO SET" as shown below:
    320277
    321278
    … …  
    331288   #
    332289   TARGET='local'
    333    MAKEFILE='makefile_fast'
     290   MAKEFILE=<name_of_your_makefile>
    334291   ECUID=None
    335292   ECGID=None
    … …  
    337294   DESTINATION=None
    338295   INSTALLDIR=None
    339    JOB_TEMPLATE='job.template'
     296   JOB_TEMPLATE=''
    340297   CONTROLFILE='CONTROL_EA5'
    341298   ...
    … …  
    358315        Install flex_extract_v7.1 software at local in directory <path-to-flex_extract>/flex_extract_v7.1
    359316
    360         Using makefile: makefile_fast
     317        Using makefile: makefile_local_gfortran
    361318        gfortran   -O3 -march=native -Bstatic -leccodes_f90 -leccodes -Bdynamic -lm -ljasper -lemosR64 -I. -I/usr/local/include -fdefault-real-8 -fopenmp -fconvert=big-endian   -c     ./rwgrib2.f90
    362319        gfortran   -O3 -march=native -Bstatic -leccodes_f90 -leccodes -Bdynamic -lm -ljasper -lemosR64 -I. -I/usr/local/include -fdefault-real-8 -fopenmp -fconvert=big-endian   -c     ./phgrreal.f90
Note: See TracChangeset for help on using the changeset viewer.
hosted by ZAMG