1 | ************************** |
---|
2 | The job script ``job.ksh`` |
---|
3 | ************************** |
---|
4 | |
---|
5 | The job script is a Korn-shell script which will be created at runtime for each ``flex_extract`` execution in the application modes **remote** and **gateway**. |
---|
6 | |
---|
7 | It is based on the ``job.temp`` template file stored in the ``Templates`` directory. |
---|
8 | This template is generated in the installation process from a ``job.template`` template file. |
---|
9 | |
---|
10 | ``Flex_extract`` uses the Python package `genshi <https://genshi.edgewall.org/>`_ to generate |
---|
11 | the Korn-shell script from the template files by substituting the individual parameters. |
---|
12 | These individual parameters are marked by ``$$`` in ``job.temp``. |
---|
13 | |
---|
14 | The job script has a number of settings for the batch system which are fixed, and differentiates between the *ecgate* and the *cca/ccb* |
---|
15 | server system to load the necessary modules for the environment when submitted to the batch queue. |
---|
16 | |
---|
17 | The submission is done by the ``ECaccess`` tool from within ``flex_extract`` with the command ``ecaccess-job-submit``. |
---|
18 | |
---|
19 | |
---|
20 | |
---|
21 | What does the job script do? |
---|
22 | ---------------------------- |
---|
23 | |
---|
24 | #. It sets necessary batch system parameters. |
---|
25 | #. It prepares the job environment at the ECMWF servers by loading the necessary library modules. |
---|
26 | #. It sets some environment variables for the single session. |
---|
27 | #. It creates the directory structure in the user's ``$SCRATCH`` file system. |
---|
28 | #. It creates a CONTROL file on the ECMWF servers whith the parameters set before creating the ``jobscript.ksh``. ``Flex_extract`` has a set of parameters which are passed to the job script with their default or the user-defined values. It also sets ``CONTROL`` as an environment variable. |
---|
29 | #. ``Flex_extract`` is started from within the ``work`` directory of the new directory structure by calling the ``submit.py`` script. It sets new paths for input and output directories and the recently generated ``CONTROL`` file. |
---|
30 | #. At the end, it checks whether the script has returned an error or not, and emails the log file to the user. |
---|
31 | |
---|
32 | |
---|
33 | |
---|
34 | |
---|
35 | Example ``jobscript.ksh`` |
---|
36 | ------------------------- |
---|
37 | |
---|
38 | .. code-block:: bash |
---|
39 | |
---|
40 | #!/bin/ksh |
---|
41 | |
---|
42 | # ON ECGB: |
---|
43 | # start with ecaccess-job-submit -queueName ecgb NAME_OF_THIS_FILE on gateway server |
---|
44 | # start with sbatch NAME_OF_THIS_FILE directly on machine |
---|
45 | |
---|
46 | #SBATCH --workdir=/scratch/ms/at/km4a |
---|
47 | #SBATCH --qos=normal |
---|
48 | #SBATCH --job-name=flex_ecmwf |
---|
49 | #SBATCH --output=flex_ecmwf.%j.out |
---|
50 | #SBATCH --error=flex_ecmwf.%j.out |
---|
51 | #SBATCH --mail-type=FAIL |
---|
52 | #SBATCH --time=12:00:00 |
---|
53 | |
---|
54 | ## CRAY specific batch requests |
---|
55 | ##PBS -N flex_ecmwf |
---|
56 | ##PBS -q np |
---|
57 | ##PBS -S /usr/bin/ksh |
---|
58 | ## -o /scratch/ms/at/km4a/flex_ecmwf.${PBS_JOBID}.out |
---|
59 | ## job output is in .ecaccess_DO_NOT_REMOVE |
---|
60 | ##PBS -j oe |
---|
61 | ##PBS -V |
---|
62 | ##PBS -l EC_threads_per_task=24 |
---|
63 | ##PBS -l EC_memory_per_task=32000MB |
---|
64 | |
---|
65 | set -x |
---|
66 | export VERSION=7.1 |
---|
67 | case ${HOST} in |
---|
68 | *ecg*) |
---|
69 | module unload grib_api |
---|
70 | module unload eccodes |
---|
71 | module unload python |
---|
72 | module unload emos |
---|
73 | module load python3 |
---|
74 | module load eccodes/2.12.0 |
---|
75 | module load emos/455-r64 |
---|
76 | export PATH=${PATH}:${HOME}/flex_extract_v7.1/Source/Python |
---|
77 | ;; |
---|
78 | *cca*) |
---|
79 | module unload python |
---|
80 | module switch PrgEnv-cray PrgEnv-intel |
---|
81 | module load python3 |
---|
82 | module load eccodes/2.12.0 |
---|
83 | module load emos |
---|
84 | export SCRATCH=${TMPDIR} |
---|
85 | export PATH=${PATH}:${HOME}/flex_extract_v7.1/Source/Python |
---|
86 | ;; |
---|
87 | esac |
---|
88 | |
---|
89 | cd ${SCRATCH} |
---|
90 | mkdir -p python$$ |
---|
91 | cd python$$ |
---|
92 | |
---|
93 | export CONTROL=CONTROL |
---|
94 | |
---|
95 | cat >${CONTROL}<<EOF |
---|
96 | accmaxstep 24 |
---|
97 | acctime 18 |
---|
98 | acctype FC |
---|
99 | accuracy 24 |
---|
100 | addpar None |
---|
101 | area 74.0/-24.0/10.0/60.0 |
---|
102 | basetime None |
---|
103 | cds_api None |
---|
104 | controlfile CONTROL_CERA |
---|
105 | cwc 1 |
---|
106 | dataset None |
---|
107 | date_chunk 3 |
---|
108 | debug 1 |
---|
109 | destination annep@genericSftp |
---|
110 | doubleelda 0 |
---|
111 | dpdeta 1 |
---|
112 | dtime 3 |
---|
113 | ec_api None |
---|
114 | ecfsdir ectmp:/${USER}/econdemand/ |
---|
115 | ecgid at |
---|
116 | ecstorage 0 |
---|
117 | ectrans 1 |
---|
118 | ecuid km4a |
---|
119 | end_date 20000809 |
---|
120 | eta 1 |
---|
121 | etadiff 0 |
---|
122 | etapar 77 |
---|
123 | expver 1 |
---|
124 | format GRIB1 |
---|
125 | gateway srvx8.img.univie.ac.at |
---|
126 | gauss 0 |
---|
127 | gaussian |
---|
128 | grib2flexpart 0 |
---|
129 | grid 1.0/1.0 |
---|
130 | inputdir /raid60/nas/tmc/Anne/Interpolation/flexextract/flex_extract_v7.1/run/workspace |
---|
131 | install_target None |
---|
132 | job_chunk 1 |
---|
133 | job_template job.temp |
---|
134 | left -24. |
---|
135 | level 91 |
---|
136 | levelist 1/to/91 |
---|
137 | logicals gauss omega omegadiff eta etadiff dpdeta cwc wrf grib2flexpart ecstorage ectrans debug oper request public purefc rrint doubleelda |
---|
138 | lower 10. |
---|
139 | mailfail ${USER} |
---|
140 | mailops ${USER} |
---|
141 | marsclass EP |
---|
142 | maxstep 0 |
---|
143 | number 000 |
---|
144 | omega 0 |
---|
145 | omegadiff 0 |
---|
146 | oper 0 |
---|
147 | outputdir /raid60/nas/tmc/Anne/Interpolation/flexextract/flex_extract_v7.1/run/workspace |
---|
148 | prefix CE |
---|
149 | public 0 |
---|
150 | purefc 0 |
---|
151 | queue ecgate |
---|
152 | request 2 |
---|
153 | resol 159 |
---|
154 | right 60. |
---|
155 | rrint 0 |
---|
156 | smooth 0 |
---|
157 | start_date 20000809 |
---|
158 | step 00 00 00 00 00 00 00 00 |
---|
159 | stream ENDA |
---|
160 | time 00 03 06 09 12 15 18 21 |
---|
161 | type AN AN AN AN AN AN AN AN |
---|
162 | upper 74. |
---|
163 | wrf 0 |
---|
164 | |
---|
165 | EOF |
---|
166 | |
---|
167 | |
---|
168 | submit.py --controlfile=${CONTROL} --inputdir=./work --outputdir=./work 1> prot 2>&1 |
---|
169 | |
---|
170 | if [ $? -eq 0 ] ; then |
---|
171 | l=0 |
---|
172 | for muser in `grep -i MAILOPS ${CONTROL}`; do |
---|
173 | if [ ${l} -gt 0 ] ; then |
---|
174 | mail -s flex.${HOST}.$$ ${muser} <prot |
---|
175 | fi |
---|
176 | l=$((${l}+1)) |
---|
177 | done |
---|
178 | else |
---|
179 | l=0 |
---|
180 | for muser in `grep -i MAILFAIL ${CONTROL}`; do |
---|
181 | if [ ${l} -gt 0 ] ; then |
---|
182 | mail -s "ERROR! flex.${HOST}.$$" ${muser} <prot |
---|
183 | fi |
---|
184 | l=$((${l}+1)) |
---|
185 | done |
---|
186 | fi |
---|
187 | |
---|
188 | |
---|
189 | |
---|
190 | |
---|
191 | .. toctree:: |
---|
192 | :hidden: |
---|
193 | :maxdepth: 2 |
---|