Changeset 0b00607 in flex_extract.git for for_developers/Sphinx/source/installation.rst

Timestamp:

Jul 29, 2019, 8:23:57 AM (5 years ago)

Author:

Anne Philipp <anne.philipp@…>

Branches:

master, ctbto, dev

Children:

bc27d19

Parents:

41c9dbc

Message:

Documentation status version 0

File:

• for_developers/Sphinx/source/installation.rst (modified) (26 diffs)

for_developers/Sphinx/source/installation.rst

@@ -67 +67 @@
 
 **Public user**: 
-    To be able to download all intended public datasets with ``flex_extract`` such as **ERA-Interim**, **CERA-20C** and **ERA5**, the public user has to create 2 accounts. 
-    At first, registration at the ECMWF website by filling out this `registration form`_ is needed to be able to retrieve *ERA-Interim* and *CERA-20C* data. For the retrievement of *ERA5* a registration at the `Copernicus Climate Data Store <https://cds.climate.copernicus.eu/user/register>`_ is needed in addition.
+    To be able to download public datasets with ``flex_extract`` such as **ERA-Interim** and **CERA-20C** (**ERA5** is not supported via ECMWF Web API anymore), the public user has to create an account at ECMWF. 
+    Use the registration at the ECMWF website by filling out this `registration form`. 
+    
+    .. note::
+
+        In the future retrievement of *ERA5* will be possible via the CDS API for public users also. Then a registration at the `Copernicus Climate Data Store <https://cds.climate.copernicus.eu/user/register>`_ is needed in addition.
     
     
@@ -77 +81 @@
 .. _ref-licence:
     
-Agree on Licences for Public Datasets
+Agree on licences for public datasets
 =====================================
 
@@ -123 +127 @@
 .. _ref-requirements: 
  
-Environment Requirements
+Environment requirements
 ========================
 
@@ -173 +177 @@
 .. _ref-remote-mode: 
 
-Remote Mode
+Remote mode
 -----------
 
 .. _ref-req-remote: 
  
-Remote Environment Requirements
+Remote environment requirements
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -196 +200 @@
 .. _ref-prep-remote:
 
-Prepare Remote Environment
+Prepare remote environment
 ^^^^^^^^^^^^^^^^^^^^^^^^^^
  
@@ -206 +210 @@
 .. _ref-install-remote:
 
-Remote Installation
+Remote installation
 ^^^^^^^^^^^^^^^^^^^
 
@@ -266 +270 @@
 ``flex_extract`` root directory for installation. 
 Before executing it, it is necessary to adapt some parameters from ``setup.sh``
-described in :ref:`ref-setup`. 
+described in :doc:`Documentation/Input/setup`. 
 
 Open ``setup.sh`` with your editor and adapt the values:  
@@ -275 +279 @@
 | .. code-block:: bash                         | .. code-block:: bash                         | 
 |                                              |                                              | 
-|   ...                                        |   ...                                        |     
+|   ...                                        |   ...                                        |   
 |   # -----------------------------------------|   # -----------------------------------------|
 |   # AVAILABLE COMMANDLINE ARGUMENTS TO SET   |   # AVAILABLE COMMANDLINE ARGUMENTS TO SET   |
@@ -290 +294 @@
 |   JOB_TEMPLATE='job.template'                |   JOB_TEMPLATE='job.template'                |
 |   CONTROLFILE='CONTROL_EA5'                  |   CONTROLFILE='CONTROL_EA5'                  | 
-|   ...                                        |   ...                                        |    
+|   ...                                        |   ...                                        |   
 +----------------------------------------------+----------------------------------------------+
 
@@ -359 +363 @@
 .. _ref-gateway-mode:
 
-Gateway Mode
+Gateway mode
 ------------
 
@@ -365 +369 @@
 .. _ref-req-gateway: 
  
-Gateway Environment Requirements
+Gateway environment requirements
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -379 +383 @@
 .. _ref-prep-gateway:
 
-Prepare Gateway Environment
+Prepare gateway environment
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 The easiest way to install all required packages is to use the package management system of your Linux distribution. To do so, it is necessary to use a user with admin rights.
-The installation was tested on a *Debian GNU/Linux buster/sid* and an *Ubuntu 18.04 Bionic Beaver* system.
+The installation was tested on a *GNU/Linux Debian buster* and an *Ubuntu 18.04 Bionic Beaver* system.
 
 .. code-block:: sh
@@ -389 +393 @@
    # On a Linux Debian or Ubuntu system do
    # (if not already available):
-   sudo apt -PV install python3
-   sudo apt -PV install pip
-   pip install genshi
-   pip install numpy
+   apt-get install python3
+   apt-get install pip
+   apt-get install genshi
+   apt-get install numpy
 
 
 .. _ref-test-gateway:
 
-Test Gateway Environment
+Test gateway environment
 ^^^^^^^^^^^^^^^^^^^^^^^^
  
@@ -414 +418 @@
 .. _ref-install-gateway:
 
-Gateway Installation
+Gateway installation
 ^^^^^^^^^^^^^^^^^^^^
 
@@ -431 +435 @@
    Your passcode: ***
    
-``Flex_extract`` will be run on an ECMWF server which makes the setup the same as for the **remote mode**. In the ``setup.sh`` script :ref:`[ref] <ref-setup>`, select the ``Makefile.gfortran`` for the ``CONVERT2`` Fortran program and the ECMWF server (*target*) you would like to use. 
+``Flex_extract`` will be run on an ECMWF server which makes the setup the same as for the **remote mode**. In the ``setup.sh`` script `[ref] <Documentation/Input/setup.html>`_, select the ``Makefile.gfortran`` for the ``CONVERT2`` Fortran program and the ECMWF server (*target*) you would like to use. 
 The job script, send to the job queue via the ECaccess software, selects again automatically the correct libraries from the module system. For enableing the file transfer you have to set the *ECUID*, *ECGID*, *GATEWAY* and *DESTINATION* parameter values.
  
@@ -506 +510 @@
 .. _ref-local-mode:
 
-Local Mode
+Local mode
 ----------
 
@@ -513 +517 @@
 .. _ref-req-local: 
  
-Local Environment Requirements
+Local environment requirements
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -536 +540 @@
 .. _ref-prep-local:
 
-Prepare Local Environment
+Prepare local environment
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -546 +550 @@
    # On a Linux Debian or Ubuntu system do
    # (if not already available):
-   sudo apt -PV install python3
-   sudo apt -PV install pip
-   sudo apt -PV install gfortran
-   sudo apt -PV install fftw3-dev 
-   sudo apt -PV install libeccodes-dev
-   sudo apt -PV install libemos-dev 
-   sudo apt -PV install python3-eccodes
-   pip install genshi
-   pip install numpy
+   apt-get install python3 (usually available on normal Linux systems)
+   apt-get install pip
+   apt-get install gfortran
+   apt-get install fftw3-dev 
+   apt-get install libeccodes-dev
+   apt-get install libemos-dev 
+   apt-get install python3-eccodes
+   apt-get install genshi
+   apt-get install numpy
    pip install cdsapi 
    pip install ecmwf-api-client 
@@ -571 +575 @@
 The CDS API (cdsapi) and the ECMWF Web API (ecmwf-api-client) have both to be installed since ERA5 can only be retrieved with the ``CDS API`` and all other datasets with the ``ECMWF Web API``.     
        
+.. note:: 
+
+    Since **public users** currently don't have access to the full *ERA5* dataset they can skip the installation of the ``CDS API``. 
+    
 Both user groups have to provide key's with their credentials for the Web API's in their home directory. Therefore, follow these instructions:
    
@@ -582 +590 @@
 .. _ref-test-local:
    
-Test Local Environment
+Test local environment
 ^^^^^^^^^^^^^^^^^^^^^^
 
@@ -615 +623 @@
 """""""""""""
 
-**Member user**
-    Please use this piece of python code:
-
-    .. code-block:: python
-
-        from ecmwfapi import ECMWFService
+
++----------------------------------------------------------+----------------------------------------------------------+
+|Please use this piece of python code for **Member user**: |Please use this piece of python code for **Public user**: |
++----------------------------------------------------------+----------------------------------------------------------+
+|.. code-block:: python                                    |.. code-block:: python                                    |
+|                                                          |                                                          |
+|    from ecmwfapi import ECMWFService                     |    from ecmwfapi import ECMWFDataServer                  |
+|                                                          |                                                          |
+|    server = ECMWFService('mars')                         |    server = ECMWFDataServer()                            |
+|                                                          |                                                          |
+|    server.retrieve({                                     |    server.retrieve({                                     |
+|        'stream'    : "oper",                             |        'stream'    : "enda",                             |
+|        'levtype'   : "sfc",                              |        'levtype'   : "sfc",                              |
+|        'param'     : "165.128/166.128/167.128",          |        'param'     : "165.128/166.128/167.128",          |
+|        'dataset'   : "interim",                          |        'dataset'   : "cera20c",                          |
+|        'step'      : "0",                                |        'step'      : "0",                                |
+|        'grid'      : "0.75/0.75",                        |        'grid'      : "1./1.",                            |
+|        'time'      : "00/06/12/18",                      |        'time'      : "00/06/12/18",                      |
+|        'date'      : "2014-07-01/to/2014-07-31",         |        'date'      : "2000-07-01/to/2000-07-31",         |
+|        'type'      : "an",                               |        'type'      : "an",                               |
+|        'class'     : "ei",                               |        'class'     : "ep",                               |
+|        'target'    : "download_erainterim_ecmwfapi.grib" |        'target'    : "download_cera20c_ecmwfapi.grib"    |
+|    })                                                    |    })                                                    |
++----------------------------------------------------------+----------------------------------------------------------+
+
             
-        server = ECMWFService('mars')
-            
-        server.retrieve({
-            'stream'    : "oper",
-            'levtype'   : "sfc",
-            'param'     : "165.128/166.128/167.128",
-            'dataset'   : "interim",
-            'step'      : "0",
-            'grid'      : "0.75/0.75",
-            'time'      : "00/06/12/18",
-            'date'      : "2014-07-01/to/2014-07-31",
-            'type'      : "an",
-            'class'     : "ei",
-            'target'    : "download_erainterim_ecmwfapi.grib"
-        })
-
-
-**Public user**
-    Please use this piece of python code: 
-
-    .. code-block:: python
-
-        from ecmwfapi import ECMWFDataServer
-            
-        server = ECMWFDataServer()
-            
-        server.retrieve({
-            'stream'    : "enda",
-            'levtype'   : "sfc",
-            'param'     : "165.128/166.128/167.128",
-            'dataset'   : "cera20c",
-            'step'      : "0",
-            'grid'      : "1./1.",
-            'time'      : "00/06/12/18",
-            'date'      : "2000-07-01/to/2000-07-31",
-            'type'      : "an",
-            'class'     : "ep",
-            'target'    : "download_cera20c_ecmwfapi.grib"
-        })
     
 CDS API 
@@ -667 +655 @@
 Since ERA5 extraction with CDS API might take some time due to the very high number of requests, you can start by retrieving some online stored pressure levels (not from MARS). This is usually much faster and gives a quick result to find out if the web API works:
 
-**All users**
-    Please use this piece of python code:
-    
-    .. code-block:: python
-
-        import cdsapi
-        
-        c = cdsapi.Client()
-        
-        c.retrieve("reanalysis-era5-pressure-levels",
-        {
-        "variable": "temperature",
-        "pressure_level": "1000",
-        "product_type": "reanalysis",
-        "year": "2008",
-        "month": "01",
-        "day": "01",
-        "time": "12:00",
-        "format": "grib"
-        },
-        "download_cdsapi.grib")
+Please use this piece of python code to retrieve a small sample of *ERA5* pressure levels:
+
+.. code-block:: python
+
+    import cdsapi
+    
+    c = cdsapi.Client()
+    
+    c.retrieve("reanalysis-era5-pressure-levels",
+    {
+    "variable": "temperature",
+    "pressure_level": "1000",
+    "product_type": "reanalysis",
+    "year": "2008",
+    "month": "01",
+    "day": "01",
+    "time": "12:00",
+    "format": "grib"
+    },
+    "download_cdsapi.grib")
 
 
 Afterwards, when you know that the CDS API generally works, you can try to extract some
-data from the MARS archive. From the latest experience this can take a while.    
-
-**Member user**
-    Please use this piece of python code:
-    
-    .. code-block:: python
-
-       import cdsapi
-       
-       c = cdsapi.Client()
-       
-       c.retrieve('reanalysis-era5-complete',
-       {
-           'class'   : 'ea',
-           'expver'  : '1',
-           'stream'  : 'oper',
-           'type'    : 'fc',
-           'step'    : '3/to/12/by/3',
-           'param'   : '130.128',
-           'levtype' : 'ml',
-           'levelist': '135/to/137',
-           'date'    : '2013-01-01',
-           'time'    : '06/18',
-           'area'    : '50/-5/40/5',
-           'grid'    : '1.0/1.0', 
-           'format'  : 'grib',
-       }, 'download_era5_cdsapi.grib')
-
-
-**Public user**
+data from the MARS archive. From the latest experience we know that this can take a while.    
+
+.. **Member user**
+
+Please use this piece of python code to retrieve a small *ERA5* data sample as a **member user**! The **public user** doesn't have access to the full *ERA5* dataset!
+
+.. code-block:: python
+
+   import cdsapi
+   
+   c = cdsapi.Client()
+   
+   c.retrieve('reanalysis-era5-complete',
+   {
+       'class'   : 'ea',
+       'expver'  : '1',
+       'stream'  : 'oper',
+       'type'    : 'fc',
+       'step'    : '3/to/12/by/3',
+       'param'   : '130.128',
+       'levtype' : 'ml',
+       'levelist': '135/to/137',
+       'date'    : '2013-01-01',
+       'time'    : '06/18',
+       'area'    : '50/-5/40/5',
+       'grid'    : '1.0/1.0', 
+       'format'  : 'grib',
+   }, 'download_era5_cdsapi.grib')
+
+
+..  ********************** COMMENTED OUT FOR FUTURE 
+    ********************** PUBLIC RETRIEVAL IS CURRENTLY NOT ACCESSIBLE 
+   
+    **Public user**
     Please use this piece of python code: 
 
@@ -754 +745 @@
 .. _ref-install-local:
 
-Local Installation
+Local installation
 ^^^^^^^^^^^^^^^^^^
 
@@ -763 +754 @@
 the ``ifort`` compiler:
 
- *Makefile.local.gfortran
- *Makefile.local.ifort
+ * Makefile.local.gfortran
+ * Makefile.local.ifort
 
 They can be found in the path ``flex_extract_vX.X/source/fortran``, where
@@ -928 +919 @@
    
 
-Mars request test
------------------
-
-    IN PREPARATION ...
 
 Full test
 ---------
 
-    IN PREPARATION ...
-
-    
+    see :doc:`quick_start`
+
+