Changeset 0b00607 in flex_extract.git for for_developers/Sphinx/source/Documentation/Input/setup.rst

Timestamp:

Jul 29, 2019, 8:23:57 AM (5 years ago)

Author:

Anne Philipp <anne.philipp@…>

Branches:

master, ctbto, dev

Children:

bc27d19

Parents:

41c9dbc

Message:

Documentation status version 0

File:

• for_developers/Sphinx/source/Documentation/Input/setup.rst (modified) (1 diff)

for_developers/Sphinx/source/Documentation/Input/setup.rst

@@ -3 +3 @@
 **************************************
 
-.. _ref-setup:
-
-
-The installation script ``setup.sh`` prepares the call of the Python 
-installation script (``install.py``) with its necessary command line arguments. 
-The ``setup.sh`` script is located in the root directory of ``flex_extract`` and has a 
-section labelled with "AVAILABLE COMMANDLINE ARGUMENTS TO SET" which
-contains the available command line parameters for ``install.py``. The user has to adapt these
-parameters for his personal use. The parameters are listed and described in the following table.
-The script also does some checks to guarantee necessary parameters were set.
-   
-.. csv-table:: 
-   :file: ../../_files/InstallationParameters.csv
-   :header-rows: 1
-   :widths: auto   
-
-After the installation some tests can be conducted. 
-They are described in section :ref:`ref-testinstallfe`.
+
+The installation of ``flex_extract`` is done by the Shell script ``setup.sh`` which is located in the root directory of ``flex_extract``.
+It calls the top-level Python script ``install.py`` which does all necessary operations to prepare the selected application environment. This includes:
+
+- preparing the file ``ECMWF_ENV`` with the user credentials for member state access to ECMWF servers (in **remote** and **gateway** mode)
+- preparation of a compilation Korn-shell script (in **remote** and **gateway** mode)
+- preparation of a job template with user credentials (in **remote** and **gateway** mode)
+- create a tar-ball of all necessary files
+- copying tar-ball to target location (depending on application mode and installation path)
+- submit compilation script to batch queue at ECMWF servers (in **remote** and **gateway** mode) or just untar tar-ball at target location (**local mode**)
+- compilation of the FORTRAN90 program ``CONVERT2``
+
+
+The Python installation script ``install.py`` has a couple of command line arguments which are defined in ``setup.sh`` in the section labelled with "*AVAILABLE COMMANDLINE ARGUMENTS TO SET*". The user has to adapt these parameters for his personal use. The parameters are listed and described in :ref:`ref-instparams`. The script also does some checks to guarantee necessary parameters were set.
+   
+After the installation process, some tests can be conducted. They are described in section :ref:`ref-testinstallfe`.
+
+The following diagram sketches the involved files and scripts in the installation process:
+
+.. _ref-install-blockdiag:
+
+.. blockdiag::
+
+   blockdiag {
+   
+     default_fontsize = 24; 
+     
+     // Set node metrix
+     node_width = 300; // default is 128
+   
+     // Set span metrix
+     span_width = 80;  // default value is 64
+     
+     ECMWF_ENV.template [shape = flowchart.input];
+     compilejob.template [shape = flowchart.input];
+     job.template [shape = flowchart.input];
+     
+     compilejob.ksh [shape = roundedbox];
+    // tarball
+    // ECMWF_ENV
+    // job.temp
+     
+     "CONTROL file" [shape = flowchart.input];
+     
+     setup.sh [shape = roundedbox];
+     install.py [shape = roundedbox];
+     
+     "ECMWF server"  [shape = flowchart.terminator];
+     
+     //beginpoint [shape = beginpoint];
+     
+     orientation = landscape;
+     
+     //beginpoint -> setup.sh;
+     setup.sh -> install.py;
+     
+     install.py <- "CONTROL file";
+                 
+     install.py -> ECMWF_ENV, job.temp, compilejob.ksh, tarball;   
+
+     ECMWF_ENV.template, job.template, compilejob.template  -> install.py; 
+     
+     tarball, compilejob.ksh -> "ECMWF server";
+     
+          
+     group exec {
+       // set backgound color
+       color = "#FF6633";
+       
+       orientation = portrait;
+       setup.sh;
+       install.py;
+     }
+     
+     group out {
+       color = "#FFFFFF";
+       group output {
+         color = "#99FF99";
+         ECMWF_ENV;
+         job.temp; 
+         compilejob.ksh; 
+       }
+        
+       group ECMWF {
+         color = "#006600";
+         tarball;
+         
+       }
+     }
+ 
+     group input {
+       
+       color = "#FFFFFF";
+       
+       group temps {
+         color = "#66CCFF";
+                  
+         ECMWF_ENV.template;
+         job.template;
+         compilejob.template;
+       }
+       
+       group in {
+         color = "#3366FF";
+         "CONTROL file";
+       }
+     }
+     
+   }
+
+
+.. blockdiag::
+   :caption: Diagram of data flow during the installation process. The trapezoids are input files with the light blue area being the template files. The edge-rounded, orange boxes are the executable files which start the installation process and reads the input files. The rectangular, green boxes are the output files. The light green files are files which are only needed in the remota and gateway mode.
+
+   blockdiag {
+   
+     group{
+       orientation = portrait;
+       label = "Legend";
+       fontsize = 28;
+       color = "#FFFFFF";
+       'executable scripts' [shape = roundedbox];
+       'input files' [shape = flowchart.input];
+       'output files';
+        server [shape = flowchart.terminator];
+     }
+   }
+
+.. _ref-instparams:
+
+Installation Parameter
+----------------------
+   
+.. exceltable:: Parameter for Installation
+    :file:  ../../_files/InstallationParameter.xls
+    :header: 1  
+   
+
+
+Content of ``setup.sh``
+-----------------------
+  
+.. literalinclude:: ../../../../../setup.sh 
+   :language: bash
+   :caption: setup.sh
+       
+       
+.. _ref-install-script:
+       
+Usage of ``install.py`` (optional)
+----------------------------------
+
+It is also possible to start the installation process of ``flex_extract`` directly from command line by using the ``install.py`` script instead of the wrapping Shell script ``setup.sh``.  This top-level script is located in 
+``flex_extract_vX.X/source/python`` and is executable. With the ``help`` parameter we see again all possible 
+command line parameter. 
+
+.. code-block:: bash
+ 
+    install.py --help
+    
+    usage: install.py [-h] [--target INSTALL_TARGET] [--makefile MAKEFILE]
+                  [--ecuid ECUID] [--ecgid ECGID] [--gateway GATEWAY]
+                  [--destination DESTINATION] [--installdir INSTALLDIR]
+                  [--job_template JOB_TEMPLATE] [--controlfile CONTROLFILE]
+
+    Install flex_extract software locally or on ECMWF machines
+
+    optional arguments:
+      -h, --help            show this help message and exit
+      --target INSTALL_TARGET
+                            Valid targets: local | ecgate | cca , the latter two
+                            are at ECMWF (default: None)
+      --makefile MAKEFILE   Name of Makefile to use for compiling the Fortran
+                            program (default: None)
+      --ecuid ECUID         The user id at ECMWF. (default: None)
+      --ecgid ECGID         The group id at ECMWF. (default: None)
+      --gateway GATEWAY     The name of the local gateway server. (default: None)
+      --destination DESTINATION
+                            The ecaccess association, e.g. myUser@genericSftp
+                            (default: None)
+      --installdir INSTALLDIR
+                            Root directory where flex_extract will be installed
+                            to. (default: None)
+      --job_template JOB_TEMPLATE
+                            The rudimentary template file to create a batch job
+                            template for submission to ECMWF servers. (default:
+                            job.template)
+      --controlfile CONTROLFILE
+                            The file with all CONTROL parameters. (default:
+                            CONTROL_EA5)
+
+
+