[41c9dbc] | 1 | ************ |
---|
| 2 | Installation |
---|
| 3 | ************ |
---|
| 4 | |
---|
| 5 | .. role:: underline |
---|
| 6 | :class: underline |
---|
| 7 | |
---|
| 8 | .. toctree:: |
---|
| 9 | :hidden: |
---|
| 10 | :maxdepth: 2 |
---|
[30f7911] | 11 | |
---|
[41c9dbc] | 12 | |
---|
| 13 | .. _Python 3: https://docs.python.org/3/ |
---|
| 14 | .. _Python3: https://www.python.org/downloads/ |
---|
| 15 | .. _Anaconda Python3: https://www.anaconda.com/distribution/#download-section |
---|
| 16 | |
---|
| 17 | .. _numpy: http://www.numpy.org/ |
---|
| 18 | .. _ecmwf-api-client: https://confluence.ecmwf.int/display/WEBAPI/ECMWF+Web+API+Home |
---|
| 19 | .. _cdsapi: https://cds.climate.copernicus.eu/api-how-to |
---|
| 20 | .. _genshi: https://genshi.edgewall.org/ |
---|
| 21 | .. _eccodes for python: https://packages.debian.org/sid/python3-eccodes |
---|
| 22 | .. _eccodes for conda: https://anaconda.org/conda-forge/eccodes |
---|
| 23 | .. _gfortran: https://gcc.gnu.org/wiki/GFortran |
---|
| 24 | .. _fftw3: http://www.fftw.org |
---|
| 25 | .. _eccodes: https://software.ecmwf.int/wiki/display/ECC |
---|
| 26 | .. _emoslib: https://software.ecmwf.int/wiki/display/EMOS/Emoslib |
---|
| 27 | .. _member state: https://www.ecmwf.int/en/about/who-we-are/member-states |
---|
| 28 | .. _registration form: https://apps.ecmwf.int/registration/ |
---|
| 29 | .. _CDS API registration: https://cds.climate.copernicus.eu/user/register |
---|
| 30 | .. _ECMWF ectrans site: https://confluence.ecmwf.int/display/ECAC/Unattended+file+transfer+-+ectrans |
---|
| 31 | .. _ECaccess Presentation: https://confluence.ecmwf.int/download/attachments/45759146/ECaccess.pdf |
---|
| 32 | .. _ECMWF's instructions on gateway server: https://confluence.ecmwf.int/display/ECAC/ECaccess+Home |
---|
| 33 | .. _Computing Representative: https://www.ecmwf.int/en/about/contact-us/computing-representatives |
---|
| 34 | .. _MARS access: https://confluence.ecmwf.int//display/WEBAPI/Access+MARS |
---|
| 35 | |
---|
| 36 | .. _download section: https://www.flexpart.eu/downloads |
---|
| 37 | |
---|
| 38 | |
---|
| 39 | |
---|
[30f7911] | 40 | The ``flex_extract`` software package contains python and shell scripts as well as a Fortran program. These components rely on several libraries which need to be available before starting the installation process. Currently, the software is only tested for a GNU/Linux environment. Feel free to try it out on other platforms. |
---|
[41c9dbc] | 41 | |
---|
[b936fd3] | 42 | At first, find out to which `user group <Ecmwf/access.html>`_ you belong and follow the instructions at :ref:`ref-registration` to obtain an account at ECMWF (if you don't have it already). Depending on the user group and the way to access the ECWMF MARS archive, there are four possible :doc:`Documentation/Overview/app_modes`: |
---|
[41c9dbc] | 43 | |
---|
[30f7911] | 44 | - Remote (member-state users only) :ref:`[installation]<ref-remote-mode>` |
---|
| 45 | - Gateway (member-state users only) :ref:`[installation]<ref-gateway-mode>` |
---|
| 46 | - Local, member-state user :ref:`[installation]<ref-local-mode>` |
---|
| 47 | - Local, public user :ref:`[installation]<ref-local-mode>` |
---|
[41c9dbc] | 48 | |
---|
[b936fd3] | 49 | More information can be found in :doc:`Documentation/Overview/app_modes`. |
---|
[41c9dbc] | 50 | |
---|
| 51 | .. note:: |
---|
| 52 | |
---|
| 53 | If you encounter any problems in the installation process, you can ask for :doc:`support`. |
---|
[b936fd3] | 54 | |
---|
| 55 | |
---|
| 56 | |
---|
| 57 | |
---|
[41c9dbc] | 58 | |
---|
| 59 | .. _ref-registration: |
---|
| 60 | |
---|
| 61 | Registration at ECMWF |
---|
| 62 | ===================== |
---|
| 63 | |
---|
[30f7911] | 64 | The registration depends on the :doc:`Documentation/Overview/app_modes` and in case of the local mode also on the data set you'd like to retrieve. The following table gives an overview where you need to register. |
---|
[41c9dbc] | 65 | |
---|
[30f7911] | 66 | +--------------+------------------------------------+--------------+ |
---|
| 67 | | | Member-state user | Public user | |
---|
| 68 | | +---------+----------+---------------+--------------+ |
---|
| 69 | | Data sets |Remote |Gateway |Local | Local | |
---|
| 70 | +--------------+---------+----------+---------------+--------------+ |
---|
| 71 | | Operational | 1 | 1 | 1,2 | - | |
---|
| 72 | +--------------+---------+----------+---------------+--------------+ |
---|
| 73 | | ERA-Interim | 1 | 1 | 1,2 | 2 | |
---|
| 74 | +--------------+---------+----------+---------------+--------------+ |
---|
| 75 | | CERA-20C | 1 | 1 | 1,2 | 2 | |
---|
| 76 | +--------------+---------+----------+---------------+--------------+ |
---|
| 77 | | ERA5 | 1 | 1 | 3 | - | |
---|
| 78 | +--------------+---------+----------+---------------+--------------+ |
---|
[41c9dbc] | 79 | |
---|
[0b00607] | 80 | |
---|
[30f7911] | 81 | |
---|
| 82 | Registration options: |
---|
| 83 | |
---|
| 84 | 1.) Access through a member-state user account granted by the `Computing Representative`_. The credentials have to be provided during installation. |
---|
[41c9dbc] | 85 | |
---|
[30f7911] | 86 | 2.) Access through the ECMWF Web API. Need to sign in at `ECMWF Web API <https://confluence.ecmwf.int/display/WEBAPI/ECMWF+Web+API+Home>`_ and configure the ECMWF key as described. Member-state users can sign in with their credentials from the `Computing Representative`_ and public users have to fill out the `registration form`_ to get an account. |
---|
[41c9dbc] | 87 | |
---|
[30f7911] | 88 | 3.) Access through the `CDS API <https://cds.climate.copernicus.eu/api-how-to>`_. Extra registration for member-state users is required at `Copernicus Climate Data Store <https://cds.climate.copernicus.eu/user/register>`_ including the configurations of the CDS key as described. This mode is currently not available for public users. See the note at `user group <Ecmwf/access.html>`_ for information. |
---|
| 89 | |
---|
[41c9dbc] | 90 | |
---|
| 91 | |
---|
| 92 | .. _ref-licence: |
---|
| 93 | |
---|
[b936fd3] | 94 | Accept licences for public datasets |
---|
[41c9dbc] | 95 | ===================================== |
---|
| 96 | |
---|
[b936fd3] | 97 | Each ECMWF :underline:`public` dataset has its own licence whose acceptance requires user activity, regardless of the user group. |
---|
[41c9dbc] | 98 | |
---|
| 99 | For the *ERA-Interim* and *CERA-20C* datasets this can be done at the ECMWF website `Available ECMWF Public Datasets <https://confluence.ecmwf.int/display/WEBAPI/Available+ECMWF+Public+Datasets>`_. Log in and follow the licence links on the right side for each dataset and accept it. |
---|
| 100 | |
---|
[30f7911] | 101 | For the *ERA5* dataset this has to be done at the `Climate Data Store (CDS) website <https://cds.climate.copernicus.eu/cdsapp#!/search?type=dataset>`_. Log in and select, on the left panel, product type "Reanalysis" for finding *ERA5* datasets. Then follow any link with *ERA5* to the full dataset record, click on tab "Download data" and scroll down. There is a section "Terms of use" where you have to click the :underline:`Accept terms` button. |
---|
[41c9dbc] | 102 | |
---|
| 103 | |
---|
| 104 | |
---|
| 105 | |
---|
| 106 | |
---|
| 107 | .. _ref-download: |
---|
| 108 | |
---|
| 109 | Download ``flex_extract`` |
---|
| 110 | ========================= |
---|
| 111 | |
---|
[30f7911] | 112 | There are two options to download ``flex_extract``: |
---|
[41c9dbc] | 113 | |
---|
| 114 | tar ball |
---|
[30f7911] | 115 | You can download a tar ball with the latest release from the `flex_extract page <https://www.flexpart.eu/wiki/FpInputMetEcmwf>`_ from our ``FLEXPART`` community website and then untar the file. |
---|
| 116 | |
---|
[41c9dbc] | 117 | .. code-block:: bash |
---|
| 118 | |
---|
| 119 | tar -xvf <flex_extract_vX.X.tar> |
---|
| 120 | |
---|
[30f7911] | 121 | git repo |
---|
| 122 | Alternatively, if you have ``git`` installed on your machine, and if you are interested to keep the code in a version control system, you may clone the latest version from our git repository master branch. |
---|
[41c9dbc] | 123 | |
---|
| 124 | .. code-block:: bash |
---|
| 125 | |
---|
[b1674ed] | 126 | $ git clone --single-branch --branch master https://www.flexpart.eu/gitmob/flex_extract |
---|
[41c9dbc] | 127 | |
---|
| 128 | |
---|
| 129 | |
---|
| 130 | |
---|
| 131 | |
---|
| 132 | .. _ref-requirements: |
---|
| 133 | |
---|
[30f7911] | 134 | Dependencies |
---|
| 135 | ============ |
---|
[41c9dbc] | 136 | |
---|
[30f7911] | 137 | The software required for running ``flex_extract`` depends on the :doc:`Documentation/Overview/app_modes` and therefore is described in the respective specific installation sections. |
---|
[41c9dbc] | 138 | |
---|
[30f7911] | 139 | Generally speaking, ``flex_extract`` requires `Python 3`_ and Fortran together with certain modules / libraries. |
---|
| 140 | We tested ``flex_extract`` with the python3 package from the the GNU/Linux distribution and Anaconda Python. The required python3 modules should prefarably be installed as distribution packages, or alternatively using Python's own package manager ``pip`` (this may mess up some aspects of your python installation, especially if you use ``pip`` as root. Think about using virtual environments.). |
---|
| 141 | |
---|
| 142 | Before installing the system packages check the availability with ``dpkg -s <package-name> | grep Status`` or ``rpm -q <package_name>``, depending on your system. For example: |
---|
| 143 | |
---|
| 144 | .. code-block:: sh |
---|
| 145 | |
---|
| 146 | $ dpkg -s libeccodes-dev | grep Status |
---|
| 147 | # or |
---|
| 148 | $ rpm -q libeccodes-dev |
---|
[41c9dbc] | 149 | |
---|
| 150 | |
---|
| 151 | |
---|
| 152 | |
---|
| 153 | |
---|
| 154 | |
---|
| 155 | .. _ref-install-fe: |
---|
| 156 | |
---|
| 157 | Installation of ``flex_extract`` |
---|
| 158 | ================================ |
---|
| 159 | |
---|
[30f7911] | 160 | The actual installation of ``flex_extract`` is done by executing a shell script called ``setup.sh``. |
---|
| 161 | It defines some parameters and calls a Python script passing the parameters as command line arguments. For details, see :doc:`Documentation/Input/setup`. |
---|
[41c9dbc] | 162 | |
---|
| 163 | For each application mode installation section we describe the requirements for the explicit |
---|
| 164 | environment and how it is installed, test if it works and how the actual ``flex_extract`` |
---|
| 165 | installation has to be done. At the users local side not all software has to be present for ``flex_extract``. |
---|
| 166 | |
---|
| 167 | |
---|
[30f7911] | 168 | Select one of the following modes to install: |
---|
[41c9dbc] | 169 | |
---|
[30f7911] | 170 | :doc:`Installation/remote` |
---|
[41c9dbc] | 171 | |
---|
[30f7911] | 172 | :doc:`Installation/gateway` |
---|
[41c9dbc] | 173 | |
---|
[30f7911] | 174 | :doc:`Installation/local` |
---|
[41c9dbc] | 175 | |
---|
| 176 | |
---|
[30f7911] | 177 | .. toctree:: |
---|
| 178 | :hidden: |
---|
| 179 | :maxdepth: 2 |
---|
[0b00607] | 180 | |
---|
[30f7911] | 181 | Installation/remote |
---|
| 182 | Installation/gateway |
---|
| 183 | Installation/local |
---|
[41c9dbc] | 184 | |
---|
| 185 | |
---|
| 186 | |
---|
| 187 | |
---|
| 188 | |
---|
| 189 | |
---|
| 190 | |
---|
| 191 | |
---|
| 192 | |
---|
| 193 | |
---|
| 194 | |
---|
| 195 | |
---|
| 196 | |
---|
| 197 | |
---|
| 198 | |
---|
| 199 | |
---|
| 200 | |
---|
| 201 | |
---|
| 202 | |
---|
| 203 | |
---|
| 204 | ``Flex_extract`` in combination with ``FLEXPART`` |
---|
| 205 | ================================================= |
---|
| 206 | |
---|
[30f7911] | 207 | Some users might wish to incorporate ``flex_extract`` directly into the ``FLEXPART`` distribution. Then the installation path has to be changed by setting the parameter `installdir` in the ``setup.sh`` file to the ``script`` directory in the ``FLEXPART`` root directoy. |
---|
[41c9dbc] | 208 | |
---|
| 209 | |
---|
| 210 | |
---|
| 211 | |
---|
| 212 | |
---|
| 213 | |
---|
| 214 | |
---|
| 215 | |
---|
| 216 | |
---|
| 217 | .. _ref-testinstallfe: |
---|
| 218 | |
---|
| 219 | Test installation |
---|
| 220 | ================= |
---|
| 221 | |
---|
| 222 | Fortran program test |
---|
| 223 | -------------------- |
---|
| 224 | |
---|
[b1674ed] | 225 | To check whether the Fortran program ``calc_etadot`` has been compiled and runs properly, it can be applied to a prepared minimal dataset. |
---|
[41c9dbc] | 226 | |
---|
[30f7911] | 227 | For this, go from the ``flex_extract`` root directory to the ``Testing/Installation/Convert/`` directory and execute the Fortran program. |
---|
[41c9dbc] | 228 | |
---|
| 229 | .. note:: |
---|
| 230 | Remember that you might have to log in at the ECMWF server if you used the installation for the **remote** or **gateway** mode. There you find the ``flex_extract`` root directory in your ``$HOME`` directory. |
---|
| 231 | |
---|
| 232 | .. code-block:: bash |
---|
| 233 | |
---|
[30f7911] | 234 | cd Testing/Installation/Convert |
---|
| 235 | # execute the Fortran progam without arguments |
---|
[b1674ed] | 236 | ../../../Source/Fortran/calc_etadot |
---|
[41c9dbc] | 237 | |
---|
[30f7911] | 238 | The installation was successfull if you obtain on standard output: |
---|
[41c9dbc] | 239 | |
---|
| 240 | .. code-block:: bash |
---|
[b1674ed] | 241 | |
---|
| 242 | STATISTICS: 98842.4598 98709.7359 5120.5385 |
---|
| 243 | STOP SUCCESSFULLY FINISHED calc_etadot: CONGRATULATIONS |
---|
[41c9dbc] | 244 | |
---|
| 245 | |
---|
[30f7911] | 246 | Now go back to the root directory: |
---|
[41c9dbc] | 247 | |
---|
| 248 | .. code-block:: bash |
---|
| 249 | |
---|
| 250 | $ cd ../../../ |
---|
| 251 | |
---|
| 252 | |
---|
| 253 | |
---|
| 254 | Full test |
---|
| 255 | --------- |
---|
| 256 | |
---|
[0b00607] | 257 | see :doc:`quick_start` |
---|
[41c9dbc] | 258 | |
---|
| 259 | |
---|