**************************
The job script ``job.ksh``
**************************
The job script is a Korn-shell script which will be created at runtime for each ``flex_extract`` execution in the application modes **remote** and **gateway**.
It is based on the ``job.temp`` template file stored in the ``Templates`` directory.
This template is generated in the installation process from a ``job.template`` template file.
``Flex_extract`` uses the Python package `genshi `_ to generate
the Korn-shell script from the template files by substituting the individual parameters.
These individual parameters are marked by ``$$`` in ``job.temp``.
The job script has a number of settings for the batch system which are fixed, and differentiates between the *ecgate* and the *cca/ccb*
server system to load the necessary modules for the environment when submitted to the batch queue.
The submission is done by the ``ECaccess`` tool from within ``flex_extract`` with the command ``ecaccess-job-submit``.
What does the job script do?
----------------------------
#. It sets necessary batch system parameters.
#. It prepares the job environment at the ECMWF servers by loading the necessary library modules.
#. It sets some environment variables for the single session.
#. It creates the directory structure in the user's ``$SCRATCH`` file system.
#. It creates a CONTROL file on the ECMWF servers whith the parameters set before creating the ``jobscript.ksh``. ``Flex_extract`` has a set of parameters which are passed to the job script with their default or the user-defined values. It also sets ``CONTROL`` as an environment variable.
#. ``Flex_extract`` is started from within the ``work`` directory of the new directory structure by calling the ``submit.py`` script. It sets new paths for input and output directories and the recently generated ``CONTROL`` file.
#. At the end, it checks whether the script has returned an error or not, and emails the log file to the user.
Example ``jobscript.ksh``
-------------------------
.. code-block:: bash
#!/bin/ksh
# ON ECGB:
# start with ecaccess-job-submit -queueName ecgb NAME_OF_THIS_FILE on gateway server
# start with sbatch NAME_OF_THIS_FILE directly on machine
#SBATCH --workdir=/scratch/ms/at/km4a
#SBATCH --qos=normal
#SBATCH --job-name=flex_ecmwf
#SBATCH --output=flex_ecmwf.%j.out
#SBATCH --error=flex_ecmwf.%j.out
#SBATCH --mail-type=FAIL
#SBATCH --time=12:00:00
## CRAY specific batch requests
##PBS -N flex_ecmwf
##PBS -q np
##PBS -S /usr/bin/ksh
## -o /scratch/ms/at/km4a/flex_ecmwf.${PBS_JOBID}.out
## job output is in .ecaccess_DO_NOT_REMOVE
##PBS -j oe
##PBS -V
##PBS -l EC_threads_per_task=24
##PBS -l EC_memory_per_task=32000MB
set -x
export VERSION=7.1
case ${HOST} in
*ecg*)
module unload grib_api
module unload eccodes
module unload python
module unload emos
module load python3
module load eccodes/2.12.0
module load emos/455-r64
export PATH=${PATH}:${HOME}/flex_extract_v7.1/Source/Python
;;
*cca*)
module unload python
module switch PrgEnv-cray PrgEnv-intel
module load python3
module load eccodes/2.12.0
module load emos
export SCRATCH=${TMPDIR}
export PATH=${PATH}:${HOME}/flex_extract_v7.1/Source/Python
;;
esac
cd ${SCRATCH}
mkdir -p python$$
cd python$$
export CONTROL=CONTROL
cat >${CONTROL}< prot 2>&1
if [ $? -eq 0 ] ; then
l=0
for muser in `grep -i MAILOPS ${CONTROL}`; do
if [ ${l} -gt 0 ] ; then
mail -s flex.${HOST}.$$ ${muser}