Changeset b1674ed in flex_extract.git for Documentation/html/Documentation/Input/setup.html
- Timestamp:
- Apr 1, 2020, 5:03:13 PM (4 years ago)
- Branches:
- master, ctbto, dev
- Children:
- d78851d, e0005c9
- Parents:
- 50f9ca6
- File:
-
- 1 edited
-
Documentation/html/Documentation/Input/setup.html (modified) (8 diffs)
Legend:
- Unmodified
- Added
- Removed
-
Documentation/html/Documentation/Input/setup.html
reca358d rb1674ed 22 22 23 23 <script type="text/javascript" id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script> 24 <script type="text/javascript"src="../../_static/jquery.js"></script>25 <script type="text/javascript"src="../../_static/underscore.js"></script>26 <script type="text/javascript"src="../../_static/doctools.js"></script>27 <script type="text/javascript"src="../../_static/language_data.js"></script>28 <script async="async" type="text/javascript"src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>24 <script src="../../_static/jquery.js"></script> 25 <script src="../../_static/underscore.js"></script> 26 <script src="../../_static/doctools.js"></script> 27 <script src="../../_static/language_data.js"></script> 28 <script async="async" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script> 29 29 30 30 <script type="text/javascript" src="../../_static/js/theme.js"></script> … … 103 103 </li> 104 104 <li class="toctree-l3"><a class="reference internal" href="compilejob.html">The Compilation Jobscript <code class="docutils literal notranslate"><span class="pre">compilejob.ksh</span></code></a></li> 105 <li class="toctree-l3"><a class="reference internal" href="fortran_makefile.html">The Fortran Makefile - <code class="docutils literal notranslate"><span class="pre"> CONVERT2</span></code></a></li>105 <li class="toctree-l3"><a class="reference internal" href="fortran_makefile.html">The Fortran Makefile - <code class="docutils literal notranslate"><span class="pre">calc_etadot</span></code></a></li> 106 106 <li class="toctree-l3"><a class="reference internal" href="run.html">The executable Script - <code class="docutils literal notranslate"><span class="pre">run.sh</span></code></a></li> 107 107 <li class="toctree-l3"><a class="reference internal" href="jobscript.html">The Jobscript <code class="docutils literal notranslate"><span class="pre">job.ksh</span></code></a></li> … … 199 199 It calls the top-level Python script <code class="docutils literal notranslate"><span class="pre">install.py</span></code> which does all necessary operations to prepare the selected application environment. This includes:</p> 200 200 <ul class="simple"> 201 <li> preparing the file <code class="docutils literal notranslate"><span class="pre">ECMWF_ENV</span></code> with the user credentials for member state access to ECMWF servers (in <strong>remote</strong> and <strong>gateway</strong> mode)</li>202 <li> preparation of a compilation Korn-shell script (in <strong>remote</strong> and <strong>gateway</strong> mode)</li>203 <li> preparation of a job template with user credentials (in <strong>remote</strong> and <strong>gateway</strong> mode)</li>204 <li> create a tar-ball of all necessary files</li>205 <li> copying tar-ball to target location (depending on application mode and installation path)</li>206 <li> submit compilation script to batch queue at ECMWF servers (in <strong>remote</strong> and <strong>gateway</strong> mode) or just untar tar-ball at target location (<strong>local mode</strong>)</li>207 <li> compilation of the FORTRAN90 program <code class="docutils literal notranslate"><span class="pre">CONVERT2</span></code></li>201 <li><p>preparing the file <code class="docutils literal notranslate"><span class="pre">ECMWF_ENV</span></code> with the user credentials for member state access to ECMWF servers (in <strong>remote</strong> and <strong>gateway</strong> mode)</p></li> 202 <li><p>preparation of a compilation Korn-shell script (in <strong>remote</strong> and <strong>gateway</strong> mode)</p></li> 203 <li><p>preparation of a job template with user credentials (in <strong>remote</strong> and <strong>gateway</strong> mode)</p></li> 204 <li><p>create a tar-ball of all necessary files</p></li> 205 <li><p>copying tar-ball to target location (depending on application mode and installation path)</p></li> 206 <li><p>submit compilation script to batch queue at ECMWF servers (in <strong>remote</strong> and <strong>gateway</strong> mode) or just untar tar-ball at target location (<strong>local mode</strong>)</p></li> 207 <li><p>compilation of the FORTRAN90 program <code class="docutils literal notranslate"><span class="pre">calc_etadot</span></code></p></li> 208 208 </ul> 209 209 <p>The Python installation script <code class="docutils literal notranslate"><span class="pre">install.py</span></code> has a couple of command line arguments which are defined in <code class="docutils literal notranslate"><span class="pre">setup.sh</span></code> in the section labelled with “<em>AVAILABLE COMMANDLINE ARGUMENTS TO SET</em>”. The user has to adapt these parameters for his personal use. The parameters are listed and described in <a class="reference internal" href="#ref-instparams"><span class="std std-ref">Installation Parameter</span></a>. The script also does some checks to guarantee necessary parameters were set.</p> 210 210 <p>After the installation process, some tests can be conducted. They are described in section <a class="reference internal" href="../../installation.html#ref-testinstallfe"><span class="std std-ref">Test installation</span></a>.</p> 211 211 <p>The following diagram sketches the involved files and scripts in the installation process:</p> 212 <div><img height="440" id="ref-install-blockdiag" src="../../_images/blockdiag-f0a58657deb31f0c134ffed39d9be96f0d278617.png" width="1600" /></div><div class="figure " id="id1">212 <div><img height="440" id="ref-install-blockdiag" src="../../_images/blockdiag-f0a58657deb31f0c134ffed39d9be96f0d278617.png" width="1600" /></div><div class="figure align-default" id="id1"> 213 213 <div><img height="120" src="../../_images/blockdiag-c5183a0da960384d172e76fcbb0cc6765f0b1945.png" width="832" /></div><p class="caption"><span class="caption-text">Diagram of data flow during the installation process. The trapezoids are input files with the light blue area being the template files. The edge-rounded, orange boxes are the executable files which start the installation process and reads the input files. The rectangular, green boxes are the output files. The light green files are files which are only needed in the remota and gateway mode.</span><a class="headerlink" href="#id1" title="Permalink to this image">¶</a></p> 214 214 </div> 215 215 <div class="section" id="installation-parameter"> 216 216 <span id="ref-instparams"></span><h2>Installation Parameter<a class="headerlink" href="#installation-parameter" title="Permalink to this headline">¶</a></h2> 217 <table border="1" class="docutils" id="id2">217 <table class="docutils align-default" id="id2"> 218 218 <caption><span class="caption-text">Parameter for Installation</span><a class="headerlink" href="#id2" title="Permalink to this table">¶</a></caption> 219 219 <colgroup> 220 <col width="10%" />221 <col width="5%" />222 <col width="15%" />223 <col width="11%" />224 <col width="58%" />220 <col style="width: 10%" /> 221 <col style="width: 5%" /> 222 <col style="width: 15%" /> 223 <col style="width: 11%" /> 224 <col style="width: 58%" /> 225 225 </colgroup> 226 <thead valign="bottom">227 <tr class="row-odd"><th class="head"> Parameter</th>228 <th class="head"> Format</th>229 <th class="head"> Possible value range</th>230 <th class="head"> Default</th>231 <th class="head"> Description</th>226 <thead> 227 <tr class="row-odd"><th class="head"><p>Parameter</p></th> 228 <th class="head"><p>Format</p></th> 229 <th class="head"><p>Possible value range</p></th> 230 <th class="head"><p>Default</p></th> 231 <th class="head"><p>Description</p></th> 232 232 </tr> 233 233 </thead> 234 <tbody valign="top"> 235 <tr class="row-even"><td><p class="first last">TARGET</p> 236 </td> 237 <td><p class="first last">String</p> 238 </td> 239 <td><p class="first last">local, ecgate, cca</p> 240 </td> 241 <td><p class="first last">None</p> 242 </td> 243 <td><p class="first last">Defines which flex_extract application mode will be used. Local = local mode; egate = Remote or Gateway mode, cca = Remote or Gateway mode. Wether the local mode is for public or member state users doesn’t matter during the installation process.</p> 244 </td> 245 </tr> 246 <tr class="row-odd"><td><p class="first last">MAKEFILE</p> 247 </td> 248 <td><p class="first last">String</p> 249 </td> 250 <td><p class="first last">any Makefile present, according to environment</p> 251 </td> 252 <td><p class="first last">Makefile.gfortran</p> 253 </td> 254 <td><p class="first last">Name of the makefile in source/fortran directory to be used for compiling the Fortran program. “Makefile.gfortran” has the configuration for the ecgate environment. For local server versions there are templates “Makefile.local.gfortran” and “Makefile.local.ifort” which have to be adapted (pathes to the eccodes library) by the user.</p> 255 </td> 256 </tr> 257 <tr class="row-even"><td><p class="first last">ECUID</p> 258 </td> 259 <td><p class="first last">String</p> 260 </td> 261 <td><p class="first last">e.g. “ecname”</p> 262 </td> 263 <td><p class="first last">None</p> 264 </td> 265 <td><p class="first last">The user id at ECMWF. Necessary for access to ECMWF server. (Remote and Gateway mode) </p> 266 </td> 267 </tr> 268 <tr class="row-odd"><td><p class="first last">ECGID</p> 269 </td> 270 <td><p class="first last">String</p> 271 </td> 272 <td><p class="first last">e.g. “groupname”</p> 273 </td> 274 <td><p class="first last">None</p> 275 </td> 276 <td><p class="first last">The group id at ECMWF. Necessary for access to ECMWF server. (Remote and Gateway mode)</p> 277 </td> 278 </tr> 279 <tr class="row-even"><td><p class="first last">GATEWAY</p> 280 </td> 281 <td><p class="first last">String</p> 282 </td> 283 <td><p class="first last">any server adress</p> 284 </td> 285 <td><p class="first last">None</p> 286 </td> 287 <td><p class="first last">The name of the local gateway server. Necessary for transfering files from the ECMWF servers to the local member state gateway server. (Gateway mode)</p> 288 </td> 289 </tr> 290 <tr class="row-odd"><td><p class="first last">DESTINATION</p> 291 </td> 292 <td><p class="first last">String</p> 293 </td> 294 <td><p class="first last">any destination which is stored at local gateway server</p> 295 </td> 296 <td><p class="first last">None</p> 297 </td> 298 <td><p class="first last">The ecaccess association, e.g. myUser@genericSftp. Used for transfering files from the ECMWF servers to the local member state gateway server. (Gateway mode)</p> 299 </td> 300 </tr> 301 <tr class="row-even"><td><p class="first last">INSTALLDIR</p> 302 </td> 303 <td><p class="first last">String</p> 304 </td> 305 <td><p class="first last">a full path </p> 306 </td> 307 <td><p class="first last">$HOME on ECMWF server and current flex_extract root path on local server</p> 308 </td> 309 <td><p class="first last">Root path where flex_extract should be installed. If it is not set it will be set to $HOME on ECMWF server and set to the current flex_extract root path on local servers. </p> 310 </td> 311 </tr> 312 <tr class="row-odd"><td><p class="first last">JOB_TEMPLATE</p> 313 </td> 314 <td><p class="first last">String</p> 315 </td> 316 <td><p class="first last">job.template</p> 317 </td> 318 <td><p class="first last">job.template</p> 319 </td> 320 <td><p class="first last">The rudimentary template file to create a batch job template for submission to ECMWF servers. Should not be changed since it is optimized for ECMWF server. (Remote and Gateway mode)</p> 321 </td> 322 </tr> 323 <tr class="row-even"><td><p class="first last">CONTROLFILE</p> 324 </td> 325 <td><p class="first last">String</p> 326 </td> 327 <td><p class="first last">any CONTROL file</p> 328 </td> 329 <td><p class="first last">CONTROL_EA5</p> 330 </td> 331 <td><p class="first last">The file with all CONTROL parameters. It does not matter which CONTROL file is used. Due to interval process it is just necessary that there is a CONTROL file in place which can be opend. </p> 332 </td> 234 <tbody> 235 <tr class="row-even"><td><p>TARGET</p></td> 236 <td><p>String</p></td> 237 <td><p>local, ecgate, cca</p></td> 238 <td><p>None</p></td> 239 <td><p>Defines which flex_extract application mode will be used. Local = local mode; egate = Remote or Gateway mode, cca = Remote or Gateway mode. Wether the local mode is for public or member state users doesn’t matter during the installation process.</p></td> 240 </tr> 241 <tr class="row-odd"><td><p>MAKEFILE</p></td> 242 <td><p>String</p></td> 243 <td><p>any Makefile present, according to environment</p></td> 244 <td><p>Makefile.gfortran</p></td> 245 <td><p>Name of the makefile in source/fortran directory to be used for compiling the Fortran program. “Makefile.gfortran” has the configuration for the ecgate environment. For local server versions there are templates “Makefile.local.gfortran” and “Makefile.local.ifort” which have to be adapted (pathes to the eccodes library) by the user.</p></td> 246 </tr> 247 <tr class="row-even"><td><p>ECUID</p></td> 248 <td><p>String</p></td> 249 <td><p>e.g. “ecname”</p></td> 250 <td><p>None</p></td> 251 <td><p>The user id at ECMWF. Necessary for access to ECMWF server. (Remote and Gateway mode) </p></td> 252 </tr> 253 <tr class="row-odd"><td><p>ECGID</p></td> 254 <td><p>String</p></td> 255 <td><p>e.g. “groupname”</p></td> 256 <td><p>None</p></td> 257 <td><p>The group id at ECMWF. Necessary for access to ECMWF server. (Remote and Gateway mode)</p></td> 258 </tr> 259 <tr class="row-even"><td><p>GATEWAY</p></td> 260 <td><p>String</p></td> 261 <td><p>any server adress</p></td> 262 <td><p>None</p></td> 263 <td><p>The name of the local gateway server. Necessary for transfering files from the ECMWF servers to the local member state gateway server. (Gateway mode)</p></td> 264 </tr> 265 <tr class="row-odd"><td><p>DESTINATION</p></td> 266 <td><p>String</p></td> 267 <td><p>any destination which is stored at local gateway server</p></td> 268 <td><p>None</p></td> 269 <td><p>The ecaccess association, e.g. myUser@genericSftp. Used for transfering files from the ECMWF servers to the local member state gateway server. (Gateway mode)</p></td> 270 </tr> 271 <tr class="row-even"><td><p>INSTALLDIR</p></td> 272 <td><p>String</p></td> 273 <td><p>a full path </p></td> 274 <td><p>$HOME on ECMWF server and current flex_extract root path on local server</p></td> 275 <td><p>Root path where flex_extract should be installed. If it is not set it will be set to $HOME on ECMWF server and set to the current flex_extract root path on local servers. </p></td> 276 </tr> 277 <tr class="row-odd"><td><p>JOB_TEMPLATE</p></td> 278 <td><p>String</p></td> 279 <td><p>job.template</p></td> 280 <td><p>job.template</p></td> 281 <td><p>The rudimentary template file to create a batch job template for submission to ECMWF servers. Should not be changed since it is optimized for ECMWF server. (Remote and Gateway mode)</p></td> 282 </tr> 283 <tr class="row-even"><td><p>CONTROLFILE</p></td> 284 <td><p>String</p></td> 285 <td><p>any CONTROL file</p></td> 286 <td><p>CONTROL_EA5</p></td> 287 <td><p>The file with all CONTROL parameters. It does not matter which CONTROL file is used. Due to interval process it is just necessary that there is a CONTROL file in place which can be opend. </p></td> 333 288 </tr> 334 289 </tbody> … … 368 323 <span class="nv">TARGET</span><span class="o">=</span><span class="s1">'ecgate'</span> 369 324 <span class="nv">MAKEFILE</span><span class="o">=</span><span class="s1">'makefile_ecgate'</span> 370 <span class="nv">ECUID</span><span class="o">=</span><span class="s1">' km4a'</span>371 <span class="nv">ECGID</span><span class="o">=</span><span class="s1">' at'</span>372 <span class="nv">GATEWAY</span><span class="o">=</span><span class="s1">' srvx8.img.univie.ac.at'</span>373 <span class="nv">DESTINATION</span><span class="o">=</span><span class="s1">' annep@genericSftp'</span>325 <span class="nv">ECUID</span><span class="o">=</span><span class="s1">'<username>'</span> 326 <span class="nv">ECGID</span><span class="o">=</span><span class="s1">'<groupID>'</span> 327 <span class="nv">GATEWAY</span><span class="o">=</span><span class="s1">'<gatewayname>'</span> 328 <span class="nv">DESTINATION</span><span class="o">=</span><span class="s1">'<username>@genericSftp'</span> 374 329 <span class="nv">INSTALLDIR</span><span class="o">=</span>None 375 330 <span class="nv">JOB_TEMPLATE</span><span class="o">=</span><span class="s1">''</span> … … 399 354 <span class="k">if</span> <span class="o">[</span> <span class="s2">"</span><span class="nv">$TARGET</span><span class="s2">"</span> <span class="o">==</span> <span class="s2">"ecgate"</span> <span class="o">]</span> <span class="o">||</span> <span class="o">[</span> <span class="s2">"</span><span class="nv">$TARGET</span><span class="s2">"</span> <span class="o">==</span> <span class="s2">"cca"</span> <span class="o">]</span><span class="p">;</span> <span class="k">then</span> 400 355 <span class="c1"># check if necessary Parameters are set</span> 401 <span class="k">if</span> <span class="o">[</span> -z <span class="s2">"</span><span class="nv">$ECUID</span><span class="s2">"</span> <span class="o">]</span> <span class="o">||</span> <span class="o">[</span> -z <span class="s2">"</span><span class="nv">$ECGID</span><span class="s2">"</span> <span class="o">]</span> <span class="o">||</span> <span class="o">[</span> -z <span class="s2">"</span><span class="nv">$GATEWAY</span><span class="s2">"</span> <span class="o">]</span> <span class="o">||</span> <span class="o">[</span> -z <span class="s2">"</span><span class="nv">$DESTINATION</span><span class="s2">"</span> <span class="o">]</span><span class="p">;</span> <span class="k">then</span>402 <span class="nb">echo</span> <span class="s2">"ERROR: At least one of the following parameters are not set: ECUID, ECGID, GATEWAY, DESTINATION!"</span>356 <span class="k">if</span> <span class="o">[</span> -z <span class="s2">"</span><span class="nv">$ECUID</span><span class="s2">"</span> <span class="o">]</span> <span class="o">||</span> <span class="o">[</span> -z <span class="s2">"</span><span class="nv">$ECGID</span><span class="s2">"</span> <span class="o">]</span> <span class="o">||</span> <span class="o">[</span> <span class="s2">"</span><span class="nv">$ECUID</span><span class="s2">"</span> <span class="o">==</span> <span class="s2">"<username>"</span> <span class="o">]</span> <span class="o">||</span> <span class="o">[</span> <span class="s2">"</span><span class="nv">$ECGID</span><span class="s2">"</span> <span class="o">==</span> <span class="s2">"<groupID>"</span> <span class="o">]</span> <span class="p">;</span> <span class="k">then</span> 357 <span class="nb">echo</span> <span class="s2">"ERROR: At least one of the following parameters are not properly set: ECUID or ECGID!"</span> 403 358 <span class="nb">echo</span> <span class="s2">"EXIT WITH ERROR"</span> 404 359 <span class="nb">exit</span> … … 406 361 <span class="nv">parameterlist</span><span class="o">+=</span><span class="s2">" --ecuid=</span><span class="nv">$ECUID</span><span class="s2"> --ecgid=</span><span class="nv">$ECGID</span><span class="s2"> --gateway=</span><span class="nv">$GATEWAY</span><span class="s2"> --destination=</span><span class="nv">$DESTINATION</span><span class="s2">"</span> 407 362 <span class="k">fi</span> 363 <span class="k">if</span> <span class="o">[</span> -z <span class="s2">"</span><span class="nv">$GATEWAY</span><span class="s2">"</span> <span class="o">]</span> <span class="o">||</span> <span class="o">[</span> -z <span class="s2">"</span><span class="nv">$DESTINATION</span><span class="s2">"</span> <span class="o">]</span> <span class="o">||</span> <span class="o">[</span> <span class="s2">"</span><span class="nv">$GATEWAY</span><span class="s2">"</span> <span class="o">==</span> <span class="s2">"<gatewayname>"</span> <span class="o">]</span> <span class="o">||</span> <span class="o">[</span> <span class="s2">"</span><span class="nv">$DESTINATION</span><span class="s2">"</span> <span class="o">==</span> <span class="s2">"<username>@genericSftp"</span> <span class="o">]</span> <span class="p">;</span> <span class="k">then</span> 364 <span class="nb">echo</span> <span class="s2">"WARNING: Not setting parameters GATEWAY and DESTINATION means there will be no file transfer to local gateway server."</span> 365 <span class="k">fi</span> 408 366 <span class="k">fi</span> 409 367 <span class="k">if</span> <span class="o">[</span> -n <span class="s2">"</span><span class="nv">$MAKEFILE</span><span class="s2">"</span> <span class="o">]</span><span class="p">;</span> <span class="k">then</span> … … 431 389 <span id="ref-install-script"></span><h2>Usage of <code class="docutils literal notranslate"><span class="pre">install.py</span></code> (optional)<a class="headerlink" href="#usage-of-install-py-optional" title="Permalink to this headline">¶</a></h2> 432 390 <p>It is also possible to start the installation process of <code class="docutils literal notranslate"><span class="pre">flex_extract</span></code> directly from command line by using the <code class="docutils literal notranslate"><span class="pre">install.py</span></code> script instead of the wrapping Shell script <code class="docutils literal notranslate"><span class="pre">setup.sh</span></code>. This top-level script is located in 433 <code class="docutils literal notranslate"><span class="pre">flex_extract_vX.X/ source/python</span></code> and is executable. With the <code class="docutils literal notranslate"><span class="pre">help</span></code> parameter we see again all possible391 <code class="docutils literal notranslate"><span class="pre">flex_extract_vX.X/Source/Python</span></code> and is executable. With the <code class="docutils literal notranslate"><span class="pre">help</span></code> parameter we see again all possible 434 392 command line parameter.</p> 435 393 <div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>install.py --help … … 492 450 <div role="contentinfo"> 493 451 <p> 494 © Copyright 20 19, Anne Philipp and Leopold Haimberger452 © Copyright 2020, Anne Philipp and Leopold Haimberger 495 453 496 454 </p>
Note: See TracChangeset
for help on using the changeset viewer.
