Changeset a9d33f6 in flex_extract.git


Ignore:
Timestamp:
Jun 1, 2020, 7:58:31 PM (4 years ago)
Author:
Petra Seibert <petra.seibert [at) univie.ac.at>
Branches:
master, ctbto, dev
Children:
5bee29a
Parents:
5920b7f
Message:

changes related to makefile, structure and names of top-level sections

Location:
For_developers/Sphinx/source
Files:
10 edited

Legend:

Unmodified
Added
Removed
  • For_developers/Sphinx/source/Documentation/Input/fortran_makefile.rst

    r5920b7f ra9d33f6  
    1616| Files to be used as they are!
    1717   
    18     | **makefile_ecgate**: For  use on ECMWF's server **ecgate**.
    19     | **makefile_cray**:   For  use on ECMWF's server **cca/ccb**.
     18    | **makefile_ecgate**: For  use on the ECMWF server **ecgate**.
     19    | **makefile_cray**:   For  use on the ECMWF servers **cca/ccb**.
    2020   
    2121| **Local mode**
     
    2424    | **makefile_fast**:  For use with the gfortran compiler and optimisation mode.
    2525    | **makefile_debug**: For use with the gfortran compiler and debugging mode. Primarily for developers.
     26    | **makefile_local_gfortran**: Linked to makefile_fast. Default value.
     27
     28They can be found at ``flex_extract_vX.X/Source/Fortran/``, where ``vX.X`` should be substituted by the current flex_extract version number.
     29
     30.. caution::   
     31   It is necessary to adapt **ECCODES_INCLUDE_DIR** and **ECCODES_LIB** in these
     32   ``makefiles`` if other than standard paths are used.
     33
     34Thus, go to the ``Fortran`` source directory and open the ``makefile`` of your
     35choice, and check / modify with an editor of your choice:
     36
     37.. code-block:: bash
     38
     39   cd flex_extract_vX.X/Source/Fortran
     40   nedit makefile_fast
     41 
     42Set the paths to the ``eccodes`` library on your local machine, if necessary.
     43
     44.. caution::
     45   This can vary from system to system.
     46   It is suggested to use a command like
     47
     48   .. code-block:: bash
     49
     50      # for the ECCODES_INCLUDE_DIR path do:
     51      $ dpkg -L libeccodes-dev | grep eccodes.mod
     52      # for the ECCODES_LIB path do:
     53      $ dpkg -L libeccodes-dev | grep libeccodes.so
     54     
     55   to find out the path to the ``eccodes`` library.
     56   
     57Assign these paths to the parameters **ECCODES_INCLUDE_DIR**
     58and **ECCODES_LIB** in the makefile, and save it.
     59
     60.. code-block:: bash
     61
     62   # these are the paths on Debian Buster:
     63   ECCODES_INCLUDE_DIR=/usr/lib/x86_64-linux-gnu/fortran/gfortran-mod-15/
     64   ECCODES_LIB= -L/usr/lib -leccodes_f90 -leccodes -lm 
     65   
    2666
    2767If you want to use another compiler than gfortran locally, you can still take ``makefile_fast``,
    2868and adapt everything that is compiler-specific in this file.
    29 
    30 For instructions on how to adapt the ``makefile`` (local application mode only),
    31 please see :ref:`ref-install-local`.
    3269
    3370   
  • For_developers/Sphinx/source/Documentation/Input/jobscript.rst

    r5920b7f ra9d33f6  
    139139    mailfail ${USER}
    140140    mailops ${USER}
    141     makefile Makefile.gfortran
    142141    marsclass EP
    143142    maxstep 0
  • For_developers/Sphinx/source/Documentation/Input/setup.rst

    r550435b ra9d33f6  
    174174                            Valid targets: local | ecgate | cca , the latter two
    175175                            are at ECMWF (default: None)
    176       --makefile MAKEFILE   Name of Makefile to use for compiling the Fortran
    177                             program (default: None)
     176      --makefile MAKEFILE   Name of makefile to compile the Fortran
     177                            code. (default depends on `target`: local - makefile_local_gfortran,
     178                            ecgate - makefile_ecgate, cca - makefile_cray)
    178179      --ecuid ECUID         The user id at ECMWF. (default: None)
    179180      --ecgid ECGID         The group id at ECMWF. (default: None)
  • For_developers/Sphinx/source/Documentation/disagg.rst

    rf20342a ra9d33f6  
    1717    :widths: 5,15,5,10
    1818   
    19     LSP,  "large-scale precipitation",          ":math:`m`",          "modified linear interpolation"
    20     CP,   "convective precipitation",           ":math:`m`",          "modified linear interpolation"
    21     SSHF, "surface sensible heat flux",         ":math:`J m^{-2}`",   "bicubic interpolation"
    22     EWSS, "eastward turbulent surface stress",  ":math:`N m^{-2} s`", "bicubic interpolation"
    23     NSSS, "northward turbulent surface stress", ":math:`N m^{-2} s`", "bicubic interpolation"
    24     SSR,  "surface net solar radiation",        ":math:`J m^{-2}`",   "bicubic interpolation"
     19    LSP,  "large-scale precipitation",          "m",          "modified linear interpolation"
     20    CP,   "convective precipitation",           "m",          "modified linear interpolation"
     21    SSHF, "surface sensible heat flux",         "J m:math:`^{-2}`",   "bicubic interpolation"
     22    EWSS, "eastward turbulent surface stress",  "N m:math:`^{-2}` s", "bicubic interpolation"
     23    NSSS, "northward turbulent surface stress", "N m:math:`^{-2}` s", "bicubic interpolation"
     24    SSR,  "surface net solar radiation",        "J m:math:`^{-2}`",   "bicubic interpolation"
    2525   
    2626
     
    6262
    6363
    64 Disaggregation is done for four adjacent timespans (:math:`a_0, a_1, a_2, a_3`) which generates a new, disaggregated value which is output at the central point of the four adjacent timespans.
     64Disaggregation is done for four adjacent time intervals (:math:`a_0, a_1, a_2, a_3`) which generates a new, disaggregated value which is output at the central point of the four adjacent time intervals.
    6565
    6666.. math::
    6767
    68        p_{ac} &= 0.5 * a_1\\
     68       p_{ac} &= 0.5 a_1\\
    6969            m &= a_0 + a_2 > 0.\\
    70     p_{ac}(m) &= a_1(m) * a_2(m) / (a_0(m) + a_2(m))\\
     70    p_{ac}(m) &= a_1(m) a_2(m) / (a_0(m) + a_2(m))\\
    7171       p_{bd} &= 0.5 * a_2\\
    7272            m &= a_1 + a_3 > 0.\\
    73     p_{bd}(m) &= a_1(m) * a_2(m) / (a_1(m) + a_3(m))\\
     73    p_{bd}(m) &= a_1(m) a_2(m) / (a_1(m) + a_3(m))\\
    7474
    7575
    76 This new point :math:`p` is used for linear interpolation of the complete timeseries afterwards. If one of the four original timespans has a value below 0, it is set to 0 prior to the calculation.
     76This new point :math:`p` is used for linear interpolation of the complete timeseries afterwards. If one of the four original time intervals has a value below 0, it is set to 0 prior to the calculation.
    7777   
    7878.. math::
     
    162162The accumulated values for the other variables are first divided by the number of hours and
    163163then interpolated to the exact times using a bicubic interpolation which conserves the integrals of the fluxes within each timespan.
    164 Disaggregation is done for four adjacent timespans (:math:`p_a, p_b, p_c, p_d`) which produces a new, disaggregated value that is the output at the central point of the four adjacent timespans.
     164Disaggregation is done for four adjacent time intervals (:math:`p_a, p_b, p_c, p_d`) which produces a new, disaggregated value that is the output at the central point of the four adjacent time intervals.
    165165
    166166.. math::
  • For_developers/Sphinx/source/Installation/local.rst

    rf20342a ra9d33f6  
    1111         
    1212   
    13 .. _Python 3: https://docs.python.org/3/
    14 .. _Python3: https://www.python.org/downloads/
    15 .. _Anaconda Python3: https://www.anaconda.com/distribution/#download-section
    16 
     13.. _Python3: https://www.python.org/
     14.. _Anaconda Python3: https://www.anaconda.com/distribution/
    1715.. _numpy: http://www.numpy.org/
    1816.. _ecmwf-api-client: https://confluence.ecmwf.int/display/WEBAPI/ECMWF+Web+API+Home
    1917.. _cdsapi: https://cds.climate.copernicus.eu/api-how-to
    2018.. _genshi: https://genshi.edgewall.org/
    21 .. _eccodes for python: https://packages.debian.org/sid/python3-eccodes
     19.. _eccodes for python: https://pypi.org/project/eccodes-python/
    2220.. _eccodes for conda: https://anaconda.org/conda-forge/eccodes
    2321.. _gfortran: https://gcc.gnu.org/wiki/GFortran
     
    3028.. _ECMWF ectrans site: https://confluence.ecmwf.int/display/ECAC/Unattended+file+transfer+-+ectrans
    3129.. _ECaccess Presentation: https://confluence.ecmwf.int/download/attachments/45759146/ECaccess.pdf
    32 .. _ECMWF's instructions on gateway server: https://confluence.ecmwf.int/display/ECAC/ECaccess+Home
     30.. _ECMWF instructions on gateway servers: https://confluence.ecmwf.int/display/ECAC/ECaccess+Home
    3331.. _Computing Representative: https://www.ecmwf.int/en/about/contact-us/computing-representatives
    3432.. _MARS access: https://confluence.ecmwf.int//display/WEBAPI/Access+MARS
    35 
    3633.. _download section: https://www.flexpart.eu/downloads
    3734
    38  
    39    
    40    
    41    
     35     
    4236.. _ref-local-mode:
    43 
    44 
    4537
    4638.. _ref-req-local:
     
    5446and libraries, since the preparation of the extraction and the post-processing is done on the local machine:
    5547
    56 +-------------------------------------------------+-----------------+
    57 |  Python part                                    | Fortran part    |
    58 +-------------------------------------------------+-----------------+
    59 | 1. `Python3`_                                   | 1. `gfortran`_  |
    60 | 2. `numpy`_                                     | 2. `fftw3`_     |
    61 | 3. `genshi`_                                    | 3. `eccodes`_   |
    62 | 4. `eccodes for python`_                        | 4. `emoslib`_   |
    63 | 5. `ecmwf-api-client`_ (everything except ERA5) |                 |
    64 | 6. `cdsapi`_ (just for ERA5 and member user)    |                 |
    65 +-------------------------------------------------+-----------------+
     48+------------------------------------------------+----------------+
     49|  Python code                                   | Fortran code   |
     50+------------------------------------------------+----------------+
     51| * `Python3`_                                   | * `gfortran`_  |
     52| * `numpy`_                                     | * `fftw3`_     |
     53| * `genshi`_                                    | * `eccodes`_   |
     54| * `eccodes for python`_                        | * `emoslib`_   |
     55| * `ecmwf-api-client`_ (everything except ERA5) |                |
     56| * `cdsapi`_ (just for ERA5 and member user)    |                |
     57+------------------------------------------------+----------------+
    6658
    6759
     
    265257
    266258
    267 
    268259.. _ref-install-local:
    269260
     
    271262==================
    272263
    273 First, adapt the Fortran ``makefile`` for your environment (if necessary) and insert it into ``setup.sh`` script (see :ref:`Fortran Makefile <ref-convert>` for more information).
    274 They can be found at ``flex_extract_vX.X/Source/Fortran/``, where ``vX.X`` should be substituted by the current flex_extract version number.
    275 
    276 .. caution::   
    277    It is necessary to adapt **ECCODES_INCLUDE_DIR** and **ECCODES_LIB** in these
    278    ``makefiles`` if other than standard paths are used.
    279 
    280 Thus, go to the ``Fortran`` source directory and open the ``makefile`` of your
    281 choice, and check / modify with an editor of your choice:
    282 
    283 .. code-block:: bash
    284 
    285    cd flex_extract_vX.X/Source/Fortran
    286    nedit makefile_fast
     264   
     265The Fortran program called ``calc_etadot`` will be compiled during the
     266installation process. A suitable makefile (``makefile_local_gfortran``) for the compilation is set by default.
     267This may be overwritten by the ``MAKEFILE`` parameter in ``setup.sh``.
     268
     269However, you may have to adapt the makefile for your environment (the current default makefile works on Debian stretch and similar GNU/Linux distributions). If you use a new name for it, you will have to insert it into ``setup.sh``
     270For details on the makefile and how to adapt them, see :ref:`Fortran Makefile <ref-convert>`.
     271
    287272 
    288 Set the paths to the ``eccodes`` library on your local machine, if necessary.
    289 
    290 .. caution::
    291    This can vary from system to system.
    292    It is suggested to use a command like
    293 
    294    .. code-block:: bash
    295 
    296       # for the ECCODES_INCLUDE_DIR path do:
    297       $ dpkg -L libeccodes-dev | grep eccodes.mod
    298       # for the ECCODES_LIB path do:
    299       $ dpkg -L libeccodes-dev | grep libeccodes.so
    300      
    301    to find out the path to the ``eccodes`` library.
    302    
    303 Assign these paths to the parameters **ECCODES_INCLUDE_DIR**
    304 and **ECCODES_LIB** in the makefile, and save it.
    305 
    306 .. code-block:: bash
    307 
    308    # these are the paths on Debian Buster:
    309    ECCODES_INCLUDE_DIR=/usr/lib/x86_64-linux-gnu/fortran/gfortran-mod-15/
    310    ECCODES_LIB= -L/usr/lib -leccodes_f90 -leccodes -lm 
    311    
    312    
    313 The Fortran program called ``calc_etadot`` will be compiled during the
    314 installation process. Therefore, the name of the ``makefile`` to be used needs to be given in  ``setup.sh``.
    315 
    316273In the root directory of ``flex_extract``, open the ``setup.sh`` script
    317274with an editor and adapt the installation parameters in the section labelled with
     
    330287   #
    331288   TARGET='local'
    332    MAKEFILE='makefile_fast'
    333289   ECUID=None
    334290   ECGID=None
     
    357313        Install flex_extract_v7.1 software at local in directory <path-to-flex_extract>/flex_extract_v7.1
    358314
    359         Using makefile: makefile_fast
     315        Using makefile: makefile_local_gfortran
    360316        gfortran   -O3 -march=native -Bstatic -leccodes_f90 -leccodes -Bdynamic -lm -ljasper -lemosR64 -I. -I/usr/local/include -fdefault-real-8 -fopenmp -fconvert=big-endian   -c     ./rwgrib2.f90
    361317        gfortran   -O3 -march=native -Bstatic -leccodes_f90 -leccodes -Bdynamic -lm -ljasper -lemosR64 -I. -I/usr/local/include -fdefault-real-8 -fopenmp -fconvert=big-endian   -c     ./phgrreal.f90
  • For_developers/Sphinx/source/Installation/remote.rst

    rb1674ed ra9d33f6  
    4848   
    4949+---------------------------+-----------------+
    50 |  Python part              | Fortran part    |
     50|  Python code              | Fortran code    |
    5151+---------------------------+-----------------+
    5252| * `Python3`_              | * `gfortran`_   |
  • For_developers/Sphinx/source/documentation.rst

    rf20342a ra9d33f6  
    1 *************
    2 Documentation
    3 *************
     1************************
     2Code-Level Documentation
     3************************
    44       
    55    Overview (under construction)
    66     
    7     Control & input data
     7    Scripts, control & input data
    88   
    99    Output data (under construction)
     
    1111    Disaggregation of flux data (under construction)
    1212   
    13     Vertical coordinate (under construction)
     13    Vertical wind (under construction)
    1414      - methods (GAUSS, ETA, OMEGA)
    1515      - calc_etadot
  • For_developers/Sphinx/source/index.rst

    r5920b7f ra9d33f6  
    44   contain the root `toctree` directive.
    55
    6 =================================================
    7 Welcome to ``flex_extract``'s user documentation!
    8 =================================================
     6===================================================
     7Welcome to the ``flex_extract`` Documentation!
     8===================================================
    99
    1010``Flex_extract`` is an open-source software to retrieve meteorological fields from the MARS archive of the European Centre for Medium-Range Weather Forecasts (ECMWF) to serve as input for the FLEXTRA/FLEXPART atmospheric transport modelling system.
     
    1414    <table style="width:100%;">
    1515      <tr>
    16         <td style="width:30%;height:150px;">
    17           <div style="width:100%;height:60px;margin-left:20%;position:relative;">
    18             <img style="position:absolute;height:90%;vertical-align:bottom;" src="_static/install_icon.png">
    19           </div>
    20           <h2><a href="installation.html">Installation</a></h2>
    21           <ul>
    22             <li><a href="installation.html#ref-requirements">Environment requirements</a></li>
    23             <li><a href="installation.html#ref-download">Download flex_extract</a></li>
    24             <li><a href="installation.html#ref-install-fe">Install flex_extract</a></li>
    25           </ul>
    26         </td>         
    2716        <td style="width:30%;height:150px;">
    2817          <div style="width:100%;height:60px;margin-left:20%;position:relative;">
     
    3827        <td style="width:30%;height:150px;">
    3928          <div style="width:100%;height:60px;margin-left:20%;position:relative;">
    40             <img style="position:absolute;height:90%;vertical-align:bottom;" src="_static/prog_icon.png">
     29            <img style="position:absolute;height:90%;vertical-align:bottom;" src="_static/install_icon.png">
    4130          </div>
    42           <h2><a href="documentation.html">Documentation</a></h2>
     31          <h2><a href="installation.html">Installation</a></h2>
    4332          <ul>
    44             <li><a href="Documentation/overview.html">Overview</a></li>
    45             <li><a href="Documentation/input.html">Control & input data</a></li>
    46             <li><a href="Documentation/output.html">Output data</a></li>
     33            <li><a href="installation.html#ref-requirements">Dependencies</a></li>
     34            <li><a href="installation.html#ref-download">Download flex_extract</a></li>
     35            <li><a href="installation.html#ref-install-fe">Install flex_extract</a></li>
    4736          </ul>
    48         </td>
    49       </tr>
    50       <tr>   
     37        </td>         
    5138        <td style="width:30%;height:150px;">
    5239          <div style="width:100%;height:60px;margin-left:20%;position:relative;">
    5340            <img style="position:absolute;height:90%;vertical-align:bottom;" src="_static/guide_icon.png">
    5441          </div>
    55           <h2><a href="quick_start.html">Quick Start</a></h2>
     42          <h2><a href="quick_start.html">Usage</a></h2>
    5643          <ul>
    5744            <li><a href="quick_start.html#job-preparation">Job preparation</a></li>
    58             <li><a href="quick_start.html#selection-and-adjustment-of-control-files">Handling CONTROL files</a></li>
     45            <li><a href="quick_start.html#selection-and-adjustment-of-control-files">CONTROL files</a></li>
    5946            <li><a href="quick_start.html#hints-for-definition-of-some-parameter-combinations">Hints on parameter combinations</a></li>
    6047          </ul>
    61         </td> 
     48        </td>        </tr>
     49      <tr>   
     50        <td style="width:30%;height:150px;">
     51          <div style="width:100%;height:60px;margin-left:20%;position:relative;">
     52            <img style="position:absolute;height:90%;vertical-align:bottom;" src="_static/prog_icon.png">
     53          </div>
     54          <h2><a href="documentation.html">Code-level documentation</a></h2>
     55          <ul>
     56            <li><a href="Documentation/overview.html">Overview</a></li>
     57            <li><a href="Documentation/input.html">Scripts, control & input data</a></li>
     58            <li><a href="Documentation/output.html">Output data</a></li>
     59          </ul>
     60        </td>
    6261        <td style="width:30%;height:150px;">
    6362          <div style="width:100%;height:60px;margin-left:20%;position:relative;">
     
    7574            <img style="position:absolute;height:90%;vertical-align:bottom;" src="_static/dev_icon.png">
    7675          </div>
    77           <h2><a href="dev_guide.html">Developer</a></h2>
     76          <h2><a href="dev_guide.html">Developer guide</a></h2>
    7877          <ul>
    79             <li><a href="Developers/gen_docu.html">Updateing the documentation</a></li>
     78            <li><a href="Developers/gen_docu.html">Updating the documentation</a></li>
    8079            <li><a href=""></a><Placeholder></li>
    8180            <li><a href=""></a><Placeholder></li>
     
    132131   :caption: Table of Contents:
    133132
     133   ecmwf_data
    134134   installation
    135135   quick_start
    136    ecmwf_data
    137136   documentation
    138137   evaluation
  • For_developers/Sphinx/source/installation.rst

    rb936fd3 ra9d33f6  
    135135============
    136136
    137 The software required for running ``flex_extract`` depends on the :doc:`Documentation/Overview/app_modes` and therefore is described in the respective specific installation sections.
     137The software required to run ``flex_extract`` depends on the :doc:`Documentation/Overview/app_modes` and therefore is described in the respective specific installation sections.
    138138   
    139139Generally speaking, ``flex_extract`` requires `Python 3`_ and Fortran together with certain modules / libraries.
    140 We tested ``flex_extract`` with the python3 package from the the GNU/Linux distribution and Anaconda Python. The required python3 modules should prefarably be installed as distribution packages, or alternatively using Python's own package manager ``pip`` (this may mess up some aspects of your python installation, especially if you use ``pip`` as root. Think about using virtual environments.).
    141 
    142 Before installing the system packages check the availability with ``dpkg -s <package-name> |  grep Status`` or ``rpm -q <package_name>``, depending on your system. For example:
     140We tested ``flex_extract`` with the Python3 package from the the GNU/Linux distribution and Anaconda Python. The required Python3 modules should prefarably be installed as distribution packages, or alternatively using Python's own package manager ``pip`` (this may mess up some aspects of your python installation, especially if you use ``pip`` as root. Think about using virtual environments.).
     141
     142Before installing the system packages check the availability with (Debian-based system) ``dpkg -s <package-name> |  grep Status`` or (Redhat-based system)``rpm -q <package_name>``. For example:
    143143
    144144.. code-block:: sh
     
    150150
    151151
    152 
    153 
    154 
    155152.. _ref-install-fe:
    156153
     
    183180    Installation/local
    184181
    185 
    186 
    187 
    188 
    189 
    190 
    191 
    192 
    193 
    194 
    195 
    196 
    197 
    198 
    199 
    200    
    201    
    202182   
    203183
     
    206186
    207187Some users might wish to incorporate ``flex_extract`` directly into the ``FLEXPART`` distribution. Then the installation path has to be changed by setting the parameter `installdir` in the ``setup.sh`` file to the ``script`` directory in the ``FLEXPART`` root directoy.
    208 
    209 
    210 
    211 
    212 
    213 
    214 
    215188
    216189
  • For_developers/Sphinx/source/quick_start.rst

    r5920b7f ra9d33f6  
    1 ***********
    2 Quick Start
    3 ***********
     1*****
     2Usage
     3*****
    44
    55``flex_extract`` is a command-line tool. In the first versions, it was started via a korn shell script and since version 6, the entry point was a python script. From version 7.1, a bash shell script was implemented to call ``flex_extract`` with the command-line parameters.
Note: See TracChangeset for help on using the changeset viewer.
hosted by ZAMG