source: flex_extract.git/Documentation/html/_sources/Installation/remote.rst.txt @ 6931f61

ctbtodev
Last change on this file since 6931f61 was 6931f61, checked in by anphi <anne.philipp@…>, 4 years ago

Update Onlinedocumentation after review of language editing
  • Property mode set to 100644
File size: 11.3 KB
Line 
1************************
2Remote mode installation
3************************
4
5.. role:: underline
6    :class: underline
7   
8.. toctree::
9    :hidden:
10    :maxdepth: 2   
11   
12     
13.. _Python 3: https://docs.python.org/3/
14.. _Python3: https://www.python.org/downloads/
15.. _Anaconda Python3: https://www.anaconda.com/distribution/#download-section
16
17.. _numpy: http://www.numpy.org/
18.. _ecmwf-api-client: https://confluence.ecmwf.int/display/WEBAPI/ECMWF+Web+API+Home
19.. _cdsapi: https://cds.climate.copernicus.eu/api-how-to
20.. _genshi: https://genshi.edgewall.org/
21.. _eccodes for python: https://packages.debian.org/sid/python3-eccodes
22.. _eccodes for conda: https://anaconda.org/conda-forge/eccodes
23.. _gfortran: https://gcc.gnu.org/wiki/GFortran
24.. _fftw3: http://www.fftw.org
25.. _eccodes: https://software.ecmwf.int/wiki/display/ECC
26.. _emoslib: https://software.ecmwf.int/wiki/display/EMOS/Emoslib
27.. _member state: https://www.ecmwf.int/en/about/who-we-are/member-states
28.. _registration form: https://apps.ecmwf.int/registration/
29.. _CDS API registration: https://cds.climate.copernicus.eu/user/register
30.. _ECMWF ectrans site: https://confluence.ecmwf.int/display/ECAC/Unattended+file+transfer+-+ectrans
31.. _ECaccess Presentation: https://confluence.ecmwf.int/download/attachments/45759146/ECaccess.pdf
32.. _ECMWF's instructions on gateway server: https://confluence.ecmwf.int/display/ECAC/ECaccess+Home
33.. _Computing Representative: https://www.ecmwf.int/en/about/contact-us/computing-representatives
34.. _MARS access: https://confluence.ecmwf.int//display/WEBAPI/Access+MARS
35
36.. _download section: https://www.flexpart.eu/downloads
37
38
39.. _ref-remote-mode:
40
41
42.. _ref-req-remote:
43 
44Remote mode - dependencies
45==========================
46
47The following software is required, and already available at the ECMWF servers:
48   
49+---------------------------+-----------------+
50|  Python code              | Fortran code    |
51+---------------------------+-----------------+
52| * `Python3`_              | * `gfortran`_   |
53| * `numpy`_                | * `fftw3`_      |
54| * `genshi`_               | * `eccodes`_    |
55| * `eccodes for python`_   | * `emoslib`_    |
56+---------------------------+-----------------+
57
58
59.. _ref-prep-remote:
60
61Prepare remote environment
62==========================
63 
64ECMWF servers provide all libraries via a module system. Loading the required modules is already built into ``flex_extract`` and no user action is needed.
65
66
67.. _ref-install-remote:
68
69Remote installation
70===================
71
72First, log in on one of the ECMWF servers, such as *ecgate* or *cca/ccb*.
73Substitute *<ecuid>* with your ECMWF user name:
74
75.. code-block:: bash
76   
77   ssh -X <ecuid>@ecaccess.ecmwf.int
78
79This will lead to the following output on the command line, asking for your password:
80   
81.. code-block:: bash
82
83   Authorized access only.
84
85   ***************************************************************
86      For further information, read the ECaccess documentation at:
87
88      https://software.ecmwf.int/wiki/display/ECAC/ECaccess+Home
89
90      You can also use ECaccess to load & download files from your
91      EChome, ECscratch or ECfs directories using the ECaccess FTP
92      server:
93
94      ftp://uid@ecaccess.ecmwf.int/
95
96      Please note you must use your UID and ActivID code to login!
97   ***************************************************************
98
99   <ecuid>@<ipname/address>'s password: ***
100   Select hostname (ecgate, cca, ccb) [ecgate]: ecgate
101
102   [<ecuid>@ecgb11 ~]$
103   
104Substitute the *<localuser>* and *<localmachine.tld>* placeholders with your local user name and the IP name or address of your local machine.
105Untar the file and change into the ``flex_extract`` root directory.
106   
107.. code-block:: bash
108
109   scp <localuser>@<localmachine.tld>:</path/to/tarfile/>flex_extract_vX.X.tar.gz  $HOME/
110   cd $HOME
111   tar xvf flex_extract_vX.X.tar.gz
112   cd flex_extract_vX.X
113   
114
115   
116Execute the ``setup.sh`` script from the ``flex_extract``'s root directory.
117Before executing it, it is necessary to adapt some parameters from ``setup.sh``
118described in :doc:`../Documentation/Input/setup`.
119
120Open ``setup.sh`` with your preferred editor (e.g., nano) and adapt the values: 
121   
122+----------------------------------------------+----------------------------------------------+   
123|   Use this for target = **ectrans**          |   Use this for target = **cca** or **ccb**   |
124+----------------------------------------------+----------------------------------------------+
125| .. code-block:: bash                         | .. code-block:: bash                         |
126|                                              |                                              |
127|   ...                                        |   ...                                        |   
128|   # -----------------------------------------|   # -----------------------------------------|
129|   # AVAILABLE COMMANDLINE ARGUMENTS TO SET   |   # AVAILABLE COMMANDLINE ARGUMENTS TO SET   |
130|   #                                          |   #                                          | 
131|   # THE USER HAS TO SPECIFY THESE PARAMETER  |   # THE USER HAS TO SPECIFY THESE PARAMETER  |
132|   #                                          |   #                                          |
133|   TARGET='ecgate'                            |   TARGET='cca'                               |
134|   MAKEFILE='makefile_ecgate'                 |   MAKEFILE='makefile_cray'                   | 
135|   ECUID='<username>'                         |   ECUID='<username>'                         | 
136|   ECGID='<groupID>'                          |   ECGID='<groupID>'                          |
137|   GATEWAY='<gatewayname>'                    |   GATEWAY='<gatewayname>'                    |
138|   DESTINATION='<username>@genericSftp'       |   DESTINATION='<username>@genericSftp'       |
139|   INSTALLDIR=None                            |   INSTALLDIR=''                              |
140|   JOB_TEMPLATE='installscript.template'      |   JOB_TEMPLATE='installscript.template'      |
141|   CONTROLFILE='CONTROL_EA5'                  |   CONTROLFILE='CONTROL_EA5'                  |
142|   ...                                        |   ...                                        |   
143+----------------------------------------------+----------------------------------------------+
144
145:underline:`Please substitute the values of ECUID and ECGID
146with your own ones (look at any of your files with ``ls -l'' to see uid and gid).`
147
148.. note::
149
150   If a local gateway server is available, files can be transferred with ``ECaccess`` commands. In that case, a valid *GATEWAY* and *DESTINATION* have to be present in the ``setup.sh`` file (even if not used, the lines must not be deleted).
151
152Afterwards, type:
153
154.. code-block:: bash
155 
156   ./setup.sh
157   
158to start the installation. You should see the following on standard output.
159   
160   
161.. code-block:: bash
162
163   # Output of setup.sh
164   Create tarball ...
165   Job compilation script has been submitted to ecgate for installation in ${HOME}/flex_extract_vX.X
166   You should get an email with subject "flexcompile" within the next few minutes!
167
168   
169``Flex_extract`` automatically uses the email address connected to the user account on ECMWF servers. The email content should look like this with a "SUCCESS" statement in the last line:
170
171.. code-block:: bash
172
173    gfortran    -O3 -march=native -L/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -Wl,-rpath,/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -leccodes_f90 -leccodes -ljasper -lpthread -L/usr/local/apps/jasper/1.900.1/LP64/lib -ljasper -lm -L/usr/local/apps/libemos/000455/GNU/6.3.0/lib  -Wl,-rpath,/usr/local/apps/libemos/000455/GNU/6.3.0/lib  -lemos.R64.D64.I32 -L/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -Wl,-rpath,/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -lfftw3    -I. -I/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/include -fdefault-real-8 -fopenmp -fconvert=big-endian   -c     ./rwgrib2.f90
174    gfortran    -O3 -march=native -L/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -Wl,-rpath,/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -leccodes_f90 -leccodes -ljasper -lpthread -L/usr/local/apps/jasper/1.900.1/LP64/lib -ljasper -lm -L/usr/local/apps/libemos/000455/GNU/6.3.0/lib  -Wl,-rpath,/usr/local/apps/libemos/000455/GNU/6.3.0/lib  -lemos.R64.D64.I32 -L/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -Wl,-rpath,/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -lfftw3    -I. -I/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/include -fdefault-real-8 -fopenmp -fconvert=big-endian   -c     ./phgrreal.f90
175    gfortran    -O3 -march=native -L/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -Wl,-rpath,/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -leccodes_f90 -leccodes -ljasper -lpthread -L/usr/local/apps/jasper/1.900.1/LP64/lib -ljasper -lm -L/usr/local/apps/libemos/000455/GNU/6.3.0/lib  -Wl,-rpath,/usr/local/apps/libemos/000455/GNU/6.3.0/lib  -lemos.R64.D64.I32 -L/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -Wl,-rpath,/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -lfftw3    -I. -I/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/include -fdefault-real-8 -fopenmp -fconvert=big-endian   -c     ./grphreal.f90
176    gfortran    -O3 -march=native -L/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -Wl,-rpath,/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -leccodes_f90 -leccodes -ljasper -lpthread -L/usr/local/apps/jasper/1.900.1/LP64/lib -ljasper -lm -L/usr/local/apps/libemos/000455/GNU/6.3.0/lib  -Wl,-rpath,/usr/local/apps/libemos/000455/GNU/6.3.0/lib  -lemos.R64.D64.I32 -L/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -Wl,-rpath,/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -lfftw3    -I. -I/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/include -fdefault-real-8 -fopenmp -fconvert=big-endian   -c     ./ftrafo.f90
177    gfortran    -O3 -march=native -L/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -Wl,-rpath,/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -leccodes_f90 -leccodes -ljasper -lpthread -L/usr/local/apps/jasper/1.900.1/LP64/lib -ljasper -lm -L/usr/local/apps/libemos/000455/GNU/6.3.0/lib  -Wl,-rpath,/usr/local/apps/libemos/000455/GNU/6.3.0/lib  -lemos.R64.D64.I32 -L/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -Wl,-rpath,/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -lfftw3    -I. -I/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/include -fdefault-real-8 -fopenmp -fconvert=big-endian   -c     ./calc_etadot.f90
178    gfortran    -O3 -march=native -L/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -Wl,-rpath,/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -leccodes_f90 -leccodes -ljasper -lpthread -L/usr/local/apps/jasper/1.900.1/LP64/lib -ljasper -lm -L/usr/local/apps/libemos/000455/GNU/6.3.0/lib  -Wl,-rpath,/usr/local/apps/libemos/000455/GNU/6.3.0/lib  -lemos.R64.D64.I32 -L/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -Wl,-rpath,/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -lfftw3    -I. -I/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/include -fdefault-real-8 -fopenmp -fconvert=big-endian   -c     ./posnam.f90
179    gfortran   rwgrib2.o calc_etadot.o ftrafo.o grphreal.o posnam.o phgrreal.o -o calc_etadot_fast.out  -O3 -march=native -L/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -Wl,-rpath,/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -leccodes_f90 -leccodes -ljasper -lpthread -L/usr/local/apps/jasper/1.900.1/LP64/lib -ljasper -lm -L/usr/local/apps/libemos/000455/GNU/6.3.0/lib  -Wl,-rpath,/usr/local/apps/libemos/000455/GNU/6.3.0/lib  -lemos.R64.D64.I32 -L/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -Wl,-rpath,/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -lfftw3    -fopenmp
180    ln -sf calc_etadot_fast.out calc_etadot
181    lrwxrwxrwx. 1 <username> at 20 Mar  8 14:11 calc_etadot -> calc_etadot_fast.out
182    SUCCESS!
183   
Note: See TracBrowser for help on using the repository browser.
hosted by ZAMG