The Installation Script - setup.sh
¶
The installation of flex_extract
is done by the Shell script setup.sh
which is located in the root directory of flex_extract
.
It calls the top-level Python script install.py
which does all necessary operations to prepare the selected application environment. This includes:
- preparing the file
ECMWF_ENV
with the user credentials for member state access to ECMWF servers (in remote and gateway mode) - preparation of a compilation Korn-shell script (in remote and gateway mode)
- preparation of a job template with user credentials (in remote and gateway mode)
- create a tar-ball of all necessary files
- copying tar-ball to target location (depending on application mode and installation path)
- submit compilation script to batch queue at ECMWF servers (in remote and gateway mode) or just untar tar-ball at target location (local mode)
- compilation of the FORTRAN90 program
CONVERT2
The Python installation script install.py
has a couple of command line arguments which are defined in setup.sh
in the section labelled with “AVAILABLE COMMANDLINE ARGUMENTS TO SET”. The user has to adapt these parameters for his personal use. The parameters are listed and described in Installation Parameter. The script also does some checks to guarantee necessary parameters were set.
After the installation process, some tests can be conducted. They are described in section Test installation.
The following diagram sketches the involved files and scripts in the installation process:
Installation Parameter¶
Parameter | Format | Possible value range | Default | Description |
---|---|---|---|---|
TARGET |
String |
local, ecgate, cca |
None |
Defines which flex_extract application mode will be used. Local = local mode; egate = Remote or Gateway mode, cca = Remote or Gateway mode. Wether the local mode is for public or member state users doesn’t matter during the installation process. |
MAKEFILE |
String |
any Makefile present, according to environment |
Makefile.gfortran |
Name of the makefile in source/fortran directory to be used for compiling the Fortran program. “Makefile.gfortran” has the configuration for the ecgate environment. For local server versions there are templates “Makefile.local.gfortran” and “Makefile.local.ifort” which have to be adapted (pathes to the eccodes library) by the user. |
ECUID |
String |
e.g. “ecname” |
None |
The user id at ECMWF. Necessary for access to ECMWF server. (Remote and Gateway mode) |
ECGID |
String |
e.g. “groupname” |
None |
The group id at ECMWF. Necessary for access to ECMWF server. (Remote and Gateway mode) |
GATEWAY |
String |
any server adress |
None |
The name of the local gateway server. Necessary for transfering files from the ECMWF servers to the local member state gateway server. (Gateway mode) |
DESTINATION |
String |
any destination which is stored at local gateway server |
None |
The ecaccess association, e.g. myUser@genericSftp. Used for transfering files from the ECMWF servers to the local member state gateway server. (Gateway mode) |
INSTALLDIR |
String |
a full path |
$HOME on ECMWF server and current flex_extract root path on local server |
Root path where flex_extract should be installed. If it is not set it will be set to $HOME on ECMWF server and set to the current flex_extract root path on local servers. |
JOB_TEMPLATE |
String |
job.template |
job.template |
The rudimentary template file to create a batch job template for submission to ECMWF servers. Should not be changed since it is optimized for ECMWF server. (Remote and Gateway mode) |
CONTROLFILE |
String |
any CONTROL file |
CONTROL_EA5 |
The file with all CONTROL parameters. It does not matter which CONTROL file is used. Due to interval process it is just necessary that there is a CONTROL file in place which can be opend. |
Content of setup.sh
¶
#!/bin/bash
#
# @Author: Anne Philipp
#
# @Date: September, 10 2018
#
# @Description:
# This file defines the flex_extract's available installation
# parameters and puts them together for the call of the actual
# python installation script.
# It also does some checks to guarantees necessary parameters
# were set.
#
# -----------------------------------------------------------------
# AVAILABLE COMMANDLINE ARGUMENTS TO SET
#
# THE USER HAS TO SPECIFY THESE PARAMETERS
#
TARGET='ecgate'
MAKEFILE='Makefile.gfortran'
ECUID='km4a'
ECGID='at'
GATEWAY='srvx8.img.univie.ac.at'
DESTINATION='annep@genericSftp'
INSTALLDIR=None
JOB_TEMPLATE=''
CONTROLFILE='CONTROL_EA5'
# -----------------------------------------------------------------
#
# AFTER THIS LINE THE USER DOES NOT HAVE TO CHANGE ANYTHING !!!
#
# -----------------------------------------------------------------
# PATH TO INSTALLATION SCRIPT
script="Source/Python/install.py"
# INITIALIZE EMPTY PARAMETERLIST
parameterlist=""
# DEFAULT PARAMETERLIST
if [ -n "$TARGET" ]; then
parameterlist=" --target=$TARGET"
else
echo "ERROR: No installation target specified."
echo "EXIT WITH ERROR"
exit
fi
# CHECK FOR MORE PARAMETER
if [ "$TARGET" == "ecgate" ] || [ "$TARGET" == "cca" ]; then
# check if necessary Parameters are set
if [ -z "$ECUID" ] || [ -z "$ECGID" ] || [ -z "$GATEWAY" ] || [ -z "$DESTINATION" ]; then
echo "ERROR: At least one of the following parameters are not set: ECUID, ECGID, GATEWAY, DESTINATION!"
echo "EXIT WITH ERROR"
exit
else
parameterlist+=" --ecuid=$ECUID --ecgid=$ECGID --gateway=$GATEWAY --destination=$DESTINATION"
fi
fi
if [ -n "$MAKEFILE" ]; then
parameterlist+=" --makefile=$MAKEFILE"
fi
if [ -n "$FLEXPARTDIR" ]; then # not empty
parameterlist+=" --flexpartdir=$FLEXPARTDIR"
fi
if [ -n "$JOB_TEMPLATE" ]; then
parameterlist+=" --job_template=$JOB_TEMPLATE"
fi
if [ -n "$CONTROLFILE" ]; then
parameterlist+=" --controlfile=$CONTROLFILE"
fi
# -----------------------------------------------------------------
# CALL INSTALLATION SCRIPT WITH DETERMINED COMMANDLINE ARGUMENTS
$script $parameterlist
Usage of install.py
(optional)¶
It is also possible to start the installation process of flex_extract
directly from command line by using the install.py
script instead of the wrapping Shell script setup.sh
. This top-level script is located in
flex_extract_vX.X/source/python
and is executable. With the help
parameter we see again all possible
command line parameter.
install.py --help
usage: install.py [-h] [--target INSTALL_TARGET] [--makefile MAKEFILE]
[--ecuid ECUID] [--ecgid ECGID] [--gateway GATEWAY]
[--destination DESTINATION] [--installdir INSTALLDIR]
[--job_template JOB_TEMPLATE] [--controlfile CONTROLFILE]
Install flex_extract software locally or on ECMWF machines
optional arguments:
-h, --help show this help message and exit
--target INSTALL_TARGET
Valid targets: local | ecgate | cca , the latter two
are at ECMWF (default: None)
--makefile MAKEFILE Name of Makefile to use for compiling the Fortran
program (default: None)
--ecuid ECUID The user id at ECMWF. (default: None)
--ecgid ECGID The group id at ECMWF. (default: None)
--gateway GATEWAY The name of the local gateway server. (default: None)
--destination DESTINATION
The ecaccess association, e.g. myUser@genericSftp
(default: None)
--installdir INSTALLDIR
Root directory where flex_extract will be installed
to. (default: None)
--job_template JOB_TEMPLATE
The rudimentary template file to create a batch job
template for submission to ECMWF servers. (default:
job.template)
--controlfile CONTROLFILE
The file with all CONTROL parameters. (default:
CONTROL_EA5)