[30f7911] | 1 | ************************ |
---|
| 2 | Remote mode installation |
---|
| 3 | ************************ |
---|
| 4 | |
---|
| 5 | .. role:: underline |
---|
| 6 | :class: underline |
---|
| 7 | |
---|
| 8 | .. toctree:: |
---|
| 9 | :hidden: |
---|
| 10 | :maxdepth: 2 |
---|
| 11 | |
---|
| 12 | |
---|
| 13 | .. _Python 3: https://docs.python.org/3/ |
---|
| 14 | .. _Python3: https://www.python.org/downloads/ |
---|
| 15 | .. _Anaconda Python3: https://www.anaconda.com/distribution/#download-section |
---|
| 16 | |
---|
| 17 | .. _numpy: http://www.numpy.org/ |
---|
| 18 | .. _ecmwf-api-client: https://confluence.ecmwf.int/display/WEBAPI/ECMWF+Web+API+Home |
---|
| 19 | .. _cdsapi: https://cds.climate.copernicus.eu/api-how-to |
---|
| 20 | .. _genshi: https://genshi.edgewall.org/ |
---|
| 21 | .. _eccodes for python: https://packages.debian.org/sid/python3-eccodes |
---|
| 22 | .. _eccodes for conda: https://anaconda.org/conda-forge/eccodes |
---|
| 23 | .. _gfortran: https://gcc.gnu.org/wiki/GFortran |
---|
| 24 | .. _fftw3: http://www.fftw.org |
---|
| 25 | .. _eccodes: https://software.ecmwf.int/wiki/display/ECC |
---|
| 26 | .. _emoslib: https://software.ecmwf.int/wiki/display/EMOS/Emoslib |
---|
| 27 | .. _member state: https://www.ecmwf.int/en/about/who-we-are/member-states |
---|
| 28 | .. _registration form: https://apps.ecmwf.int/registration/ |
---|
| 29 | .. _CDS API registration: https://cds.climate.copernicus.eu/user/register |
---|
| 30 | .. _ECMWF ectrans site: https://confluence.ecmwf.int/display/ECAC/Unattended+file+transfer+-+ectrans |
---|
| 31 | .. _ECaccess Presentation: https://confluence.ecmwf.int/download/attachments/45759146/ECaccess.pdf |
---|
| 32 | .. _ECMWF's instructions on gateway server: https://confluence.ecmwf.int/display/ECAC/ECaccess+Home |
---|
| 33 | .. _Computing Representative: https://www.ecmwf.int/en/about/contact-us/computing-representatives |
---|
| 34 | .. _MARS access: https://confluence.ecmwf.int//display/WEBAPI/Access+MARS |
---|
| 35 | |
---|
| 36 | .. _download section: https://www.flexpart.eu/downloads |
---|
| 37 | |
---|
| 38 | |
---|
| 39 | .. _ref-remote-mode: |
---|
| 40 | |
---|
| 41 | |
---|
| 42 | .. _ref-req-remote: |
---|
| 43 | |
---|
| 44 | Remote mode - dependencies |
---|
| 45 | ========================== |
---|
| 46 | |
---|
| 47 | The following software is required, and already available at the ECMWF servers: |
---|
| 48 | |
---|
| 49 | +---------------------------+-----------------+ |
---|
| 50 | | Python part | Fortran part | |
---|
| 51 | +---------------------------+-----------------+ |
---|
| 52 | | * `Python3`_ | * `gfortran`_ | |
---|
| 53 | | * `numpy`_ | * `fftw3`_ | |
---|
| 54 | | * `genshi`_ | * `eccodes`_ | |
---|
| 55 | | * `eccodes for python`_ | * `emoslib`_ | |
---|
| 56 | +---------------------------+-----------------+ |
---|
| 57 | |
---|
| 58 | |
---|
| 59 | .. _ref-prep-remote: |
---|
| 60 | |
---|
| 61 | Prepare remote environment |
---|
| 62 | ========================== |
---|
| 63 | |
---|
| 64 | ECMWF servers provide all libraries via a module system. Loading the required modules is already built into ``flex_extract`` and no user action is needed. |
---|
| 65 | |
---|
| 66 | |
---|
| 67 | .. _ref-install-remote: |
---|
| 68 | |
---|
| 69 | Remote installation |
---|
| 70 | =================== |
---|
| 71 | |
---|
| 72 | First, log in on one of the ECMWF servers, such as *ecgate* or *cca/ccb*. |
---|
| 73 | Substitute *<ecuid>* with your ECMWF user name: |
---|
| 74 | |
---|
| 75 | .. code-block:: bash |
---|
| 76 | |
---|
| 77 | ssh -X <ecuid>@ecaccess.ecmwf.int |
---|
| 78 | |
---|
[b1674ed] | 79 | This will lead to the following output on the command line, asking for your password: |
---|
[30f7911] | 80 | |
---|
| 81 | .. code-block:: bash |
---|
| 82 | |
---|
| 83 | Authorized access only. |
---|
| 84 | |
---|
| 85 | *************************************************************** |
---|
| 86 | For further information, read the ECaccess documentation at: |
---|
| 87 | |
---|
| 88 | https://software.ecmwf.int/wiki/display/ECAC/ECaccess+Home |
---|
| 89 | |
---|
| 90 | You can also use ECaccess to load & download files from your |
---|
| 91 | EChome, ECscratch or ECfs directories using the ECaccess FTP |
---|
| 92 | server: |
---|
| 93 | |
---|
| 94 | ftp://uid@ecaccess.ecmwf.int/ |
---|
| 95 | |
---|
| 96 | Please note you must use your UID and ActivID code to login! |
---|
| 97 | *************************************************************** |
---|
| 98 | |
---|
| 99 | <ecuid>@<ipname/address>'s password: *** |
---|
| 100 | Select hostname (ecgate, cca, ccb) [ecgate]: ecgate |
---|
| 101 | |
---|
| 102 | [<ecuid>@ecgb11 ~]$ |
---|
| 103 | |
---|
| 104 | Substitute the *<localuser>* and *<localmachine.tld>* placeholders with your local user name and the IP name or address of your local machine. |
---|
| 105 | Untar the file and change into the ``flex_extract`` root directory. |
---|
| 106 | |
---|
| 107 | .. code-block:: bash |
---|
| 108 | |
---|
| 109 | scp <localuser>@<localmachine.tld>:</path/to/tarfile/>flex_extract_vX.X.tar.gz $HOME/ |
---|
| 110 | cd $HOME |
---|
| 111 | tar xvf flex_extract_vX.X.tar.gz |
---|
| 112 | cd flex_extract_vX.X |
---|
| 113 | |
---|
| 114 | |
---|
| 115 | |
---|
| 116 | Execute the ``setup.sh`` script from the ``flex_extract``'s root directory. |
---|
| 117 | Before executing it, it is necessary to adapt some parameters from ``setup.sh`` |
---|
[b1674ed] | 118 | described in :doc:`../Documentation/Input/setup`. |
---|
[30f7911] | 119 | |
---|
[b1674ed] | 120 | Open ``setup.sh`` with your preferred editor (e.g., nano) and adapt the values: |
---|
[30f7911] | 121 | |
---|
| 122 | +----------------------------------------------+----------------------------------------------+ |
---|
[b1674ed] | 123 | | Use this for target = **ectrans** | Use this for target = **cca** or **ccb** | |
---|
[30f7911] | 124 | +----------------------------------------------+----------------------------------------------+ |
---|
| 125 | | .. code-block:: bash | .. code-block:: bash | |
---|
| 126 | | | | |
---|
| 127 | | ... | ... | |
---|
| 128 | | # -----------------------------------------| # -----------------------------------------| |
---|
| 129 | | # AVAILABLE COMMANDLINE ARGUMENTS TO SET | # AVAILABLE COMMANDLINE ARGUMENTS TO SET | |
---|
| 130 | | # | # | |
---|
| 131 | | # THE USER HAS TO SPECIFY THESE PARAMETER | # THE USER HAS TO SPECIFY THESE PARAMETER | |
---|
| 132 | | # | # | |
---|
| 133 | | TARGET='ecgate' | TARGET='cca' | |
---|
[b1674ed] | 134 | | MAKEFILE='makefile_ecgate' | MAKEFILE='makefile_cray' | |
---|
| 135 | | ECUID='<username>' | ECUID='<username>' | |
---|
| 136 | | ECGID='<groupID>' | ECGID='<groupID>' | |
---|
| 137 | | GATEWAY='<gatewayname>' | GATEWAY='<gatewayname>' | |
---|
| 138 | | DESTINATION='<username>@genericSftp' | DESTINATION='<username>@genericSftp' | |
---|
[30f7911] | 139 | | INSTALLDIR=None | INSTALLDIR='' | |
---|
| 140 | | JOB_TEMPLATE='job.template' | JOB_TEMPLATE='job.template' | |
---|
| 141 | | CONTROLFILE='CONTROL_EA5' | CONTROLFILE='CONTROL_EA5' | |
---|
| 142 | | ... | ... | |
---|
| 143 | +----------------------------------------------+----------------------------------------------+ |
---|
| 144 | |
---|
| 145 | :underline:`Please substitute the values of ECUID and ECGID |
---|
| 146 | with your own ones (look at any of your files with ``ls -l'' to see uid and gid).` |
---|
| 147 | |
---|
| 148 | .. note:: |
---|
| 149 | |
---|
| 150 | If a local gateway server is available, files can be transferred with ``ECaccess`` commands. In that case, a valid *GATEWAY* and *DESTINATION* have to be present in the ``setup.sh`` file (even if not used, the lines must not be deleted). |
---|
| 151 | |
---|
| 152 | Afterwards, type: |
---|
| 153 | |
---|
| 154 | .. code-block:: bash |
---|
| 155 | |
---|
| 156 | module load python3 |
---|
| 157 | ./setup.sh |
---|
| 158 | |
---|
| 159 | to start the installation. You should see the following on standard output. |
---|
| 160 | |
---|
| 161 | |
---|
| 162 | .. code-block:: bash |
---|
| 163 | |
---|
| 164 | # Output of setup.sh |
---|
| 165 | Create tarball ... |
---|
| 166 | Job compilation script has been submitted to ecgate for installation in ${HOME}/flex_extract_vX.X |
---|
| 167 | You should get an email with subject "flexcompile" within the next few minutes! |
---|
| 168 | |
---|
| 169 | |
---|
[b1674ed] | 170 | ``Flex_extract`` automatically uses the email address connected to the user account on ECMWF servers. The email content should look like this with a "SUCCESS" statement in the last line: |
---|
[30f7911] | 171 | |
---|
| 172 | .. code-block:: bash |
---|
| 173 | |
---|
[b1674ed] | 174 | gfortran -O3 -march=native -L/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -Wl,-rpath,/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -leccodes_f90 -leccodes -ljasper -lpthread -L/usr/local/apps/jasper/1.900.1/LP64/lib -ljasper -lm -L/usr/local/apps/libemos/000455/GNU/6.3.0/lib -Wl,-rpath,/usr/local/apps/libemos/000455/GNU/6.3.0/lib -lemos.R64.D64.I32 -L/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -Wl,-rpath,/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -lfftw3 -I. -I/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/include -fdefault-real-8 -fopenmp -fconvert=big-endian -c ./rwgrib2.f90 |
---|
| 175 | gfortran -O3 -march=native -L/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -Wl,-rpath,/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -leccodes_f90 -leccodes -ljasper -lpthread -L/usr/local/apps/jasper/1.900.1/LP64/lib -ljasper -lm -L/usr/local/apps/libemos/000455/GNU/6.3.0/lib -Wl,-rpath,/usr/local/apps/libemos/000455/GNU/6.3.0/lib -lemos.R64.D64.I32 -L/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -Wl,-rpath,/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -lfftw3 -I. -I/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/include -fdefault-real-8 -fopenmp -fconvert=big-endian -c ./phgrreal.f90 |
---|
| 176 | gfortran -O3 -march=native -L/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -Wl,-rpath,/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -leccodes_f90 -leccodes -ljasper -lpthread -L/usr/local/apps/jasper/1.900.1/LP64/lib -ljasper -lm -L/usr/local/apps/libemos/000455/GNU/6.3.0/lib -Wl,-rpath,/usr/local/apps/libemos/000455/GNU/6.3.0/lib -lemos.R64.D64.I32 -L/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -Wl,-rpath,/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -lfftw3 -I. -I/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/include -fdefault-real-8 -fopenmp -fconvert=big-endian -c ./grphreal.f90 |
---|
| 177 | gfortran -O3 -march=native -L/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -Wl,-rpath,/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -leccodes_f90 -leccodes -ljasper -lpthread -L/usr/local/apps/jasper/1.900.1/LP64/lib -ljasper -lm -L/usr/local/apps/libemos/000455/GNU/6.3.0/lib -Wl,-rpath,/usr/local/apps/libemos/000455/GNU/6.3.0/lib -lemos.R64.D64.I32 -L/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -Wl,-rpath,/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -lfftw3 -I. -I/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/include -fdefault-real-8 -fopenmp -fconvert=big-endian -c ./ftrafo.f90 |
---|
| 178 | gfortran -O3 -march=native -L/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -Wl,-rpath,/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -leccodes_f90 -leccodes -ljasper -lpthread -L/usr/local/apps/jasper/1.900.1/LP64/lib -ljasper -lm -L/usr/local/apps/libemos/000455/GNU/6.3.0/lib -Wl,-rpath,/usr/local/apps/libemos/000455/GNU/6.3.0/lib -lemos.R64.D64.I32 -L/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -Wl,-rpath,/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -lfftw3 -I. -I/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/include -fdefault-real-8 -fopenmp -fconvert=big-endian -c ./calc_etadot.f90 |
---|
| 179 | gfortran -O3 -march=native -L/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -Wl,-rpath,/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -leccodes_f90 -leccodes -ljasper -lpthread -L/usr/local/apps/jasper/1.900.1/LP64/lib -ljasper -lm -L/usr/local/apps/libemos/000455/GNU/6.3.0/lib -Wl,-rpath,/usr/local/apps/libemos/000455/GNU/6.3.0/lib -lemos.R64.D64.I32 -L/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -Wl,-rpath,/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -lfftw3 -I. -I/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/include -fdefault-real-8 -fopenmp -fconvert=big-endian -c ./posnam.f90 |
---|
| 180 | gfortran rwgrib2.o calc_etadot.o ftrafo.o grphreal.o posnam.o phgrreal.o -o calc_etadot_fast.out -O3 -march=native -L/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -Wl,-rpath,/usr/local/apps/eccodes/2.13.0/GNU/7.3.0/lib -leccodes_f90 -leccodes -ljasper -lpthread -L/usr/local/apps/jasper/1.900.1/LP64/lib -ljasper -lm -L/usr/local/apps/libemos/000455/GNU/6.3.0/lib -Wl,-rpath,/usr/local/apps/libemos/000455/GNU/6.3.0/lib -lemos.R64.D64.I32 -L/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -Wl,-rpath,/usr/local/apps/fftw/3.3.4/GNU/6.3.0/lib -lfftw3 -fopenmp |
---|
| 181 | ln -sf calc_etadot_fast.out calc_etadot |
---|
| 182 | lrwxrwxrwx. 1 <username> at 20 Mar 8 14:11 calc_etadot -> calc_etadot_fast.out |
---|
| 183 | SUCCESS! |
---|
[30f7911] | 184 | |
---|