Remote mode installation

Remote environment requirements

The environment for flex_extract on ECMWF servers has to provide these software packages and libraries:

Python part Fortran part

Prepare remote environment

The environment on the ECMWF server (such as ecgate or cca) is already prepared. ECMWF server provide all libraries via a module system which is going to be selected by flex_extract automatically.

Remote installation

First, log in on one of the ECMWF Linux server, such as ecgate or cca/ccb. Substitute <ecuid> with your ECMWF user name:

ssh -X <ecuid>@ecaccess.ecmwf.int

This will lead to the following output on the command line, asking for your password:

Authorized access only.

***************************************************************
   For further information, read the ECaccess documentation at:

   https://software.ecmwf.int/wiki/display/ECAC/ECaccess+Home

   You can also use ECaccess to load & download files from your
   EChome, ECscratch or ECfs directories using the ECaccess FTP
   server:

   ftp://uid@ecaccess.ecmwf.int/

   Please note you must use your UID and ActivID code to login!
***************************************************************

<ecuid>@131.130.157.5's password: ***
Select hostname (ecgate, cca, ccb) [ecgate]: ecgate

[<ecuid>@ecgb11 ~]$

Then, copy the flex_extract tar ball (from section Download flex_extract) to the $HOME directory of the ECMWF Linux server via scp. Substitute the <localuser> and <server.edu> placeholders with your credentials. Untar the file and change into the flex_extract root directory.

scp <localuser>@<server.edu>:/path/to/tarfile/flex_extract_vX.X.tar.gz  $HOME/
cd $HOME
tar xvf flex_extract_vX.X.tar.gz
cd flex_extract_vX.X

On these ECMWF servers, it is not necessary to prepare the environment or the Makefile for the Fortran program (CONVERT2) as described above. All third party libraries are available from a module system. The Makefile is optimized for ECMWF servers and the compilation script compilejob.ksh, which will be submitted by flex_extract to the batch job queue at ECMWF, does load all relevant modules from the ECMWF’s module system.

So there is just the need to execute the setup.sh script from the flex_extract root directory for installation. Before executing it, it is necessary to adapt some parameters from setup.sh described in Documentation/Input/setup.

Open setup.sh with your editor and adapt the values:

Take this for target = ectrans Take this for target = cca 
...
# -----------------------------------------
# AVAILABLE COMMANDLINE ARGUMENTS TO SET
#
# THE USER HAS TO SPECIFY THESE PARAMETER
#
TARGET='ecgate'
MAKEFILE='Makefile.gfortran'
ECUID='uid'
ECGID='gid'
GATEWAY=None
DESTINATION=None
INSTALLDIR=None
JOB_TEMPLATE='job.template'
CONTROLFILE='CONTROL_EA5'
...

...
# -----------------------------------------
# AVAILABLE COMMANDLINE ARGUMENTS TO SET
#
# THE USER HAS TO SPECIFY THESE PARAMETER
#
TARGET='cca'
MAKEFILE='Makefile.CRAY'
ECUID='uid'
ECGID='gid'
GATEWAY=None
DESTINATION=None
INSTALLDIR=''
JOB_TEMPLATE='job.template'
CONTROLFILE='CONTROL_EA5'
...

Please substitute the values of ECUID and ECGID with your own credentials and settings.

Note

If a local gateway server is available the transfer of files could be done via the ECaccess commands. Therefore a valid GATEWAY and DESTINATION have to be present and should be set in the setup.sh file.

Afterwards, type:

module load python3
./setup.sh

to start the installation. You should see the following output at the command line.

# Output of setup.sh
Create tarball ...
Job compilation script has been submitted to ecgate for installation in ${HOME}/flex_extract_vX.X
You should get an email with subject "flexcompile" within the next few minutes!

The email content should look like this with a “SUCCESS” statement in the last line:

gfortran   -m64 -fdefault-real-8 -fcray-pointer -fno-second-underscore  -ffixed-line-length-132 -fopenmp  -fconvert=big-endian  -c -g -O3 -fopenmp phgrreal.f
gfortran   -m64 -fdefault-real-8 -fcray-pointer -fno-second-underscore  -ffixed-line-length-132 -fopenmp  -fconvert=big-endian  -c -g -O3 -fopenmp grphreal.f
gfortran   -m64 -fdefault-real-8 -fcray-pointer -fno-second-underscore  -ffixed-line-length-132 -fopenmp  -fconvert=big-endian  -c -g -O3 -fopenmp ftrafo.f
gfortran   -m64 -fdefault-real-8 -fcray-pointer -fno-second-underscore  -ffixed-line-length-132 -fopenmp  -fconvert=big-endian  -c -O3 -I. -I/usr/local/apps/eccodes/2.12.0/GNU/6.3.0/include -g rwGRIB2.f90
gfortran   -m64 -fdefault-real-8 -fcray-pointer -fno-second-underscore  -ffixed-line-length-132 -fopenmp  -fconvert=big-endian  -c -O3 -I. -I/usr/local/apps/eccodes/2.12.0/GNU/6.3.0/include -g posnam.f
gfortran   -m64 -fdefault-real-8 -fcray-pointer -fno-second-underscore  -ffixed-line-length-132 -fopenmp  -fconvert=big-endian  -c -O3 -I. -I/usr/local/apps/eccodes/2.12.0/GNU/6.3.0/include -g preconvert.f90
gfortran   -m64 -fdefault-real-8 -fcray-pointer -fno-second-underscore  -ffixed-line-length-132 -fopenmp  -fconvert=big-endian  -g -O3 -o ./CONVERT2 ftrafo.o phgrreal.o grphreal.o rwGRIB2.o posnam.o preconvert.o -L/usr/local/apps/eccodes/2.12.0/GNU/6.3.0/lib -Wl,-rpath,/usr/local/apps/eccodes/2.12.0/GNU/6.3.0/lib -leccodes_f90 -leccodes -ljasper -lpthread -L/usr/local/apps/jasper/1.900.1/LP64/lib -ljasper -lm -L/usr/local/apps/libemos/000455/GNU/6.3.0/lib  -Wl,-rpath,/usr/local/apps/libemos/000455/GNU/6.3.0/lib  -lemos.R64.D64.I32 -L/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -Wl,-rpath,/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -lfftw3
-rwxr-x---. 1 USER at 353134 May 23 12:27 CONVERT2
SUCCESS!