Auto-generated documentation for the Python scripts¶

Programs
- install
- submit
Classes
Modules

Programs ¶

install ¶

This script installs the flex_extract program.

Depending on the selected installation environment (locally or on the ECMWF server ecgate or cca) the program extracts the command line arguments and the CONTROL file parameter and prepares the corresponding environment. The necessary files are collected in a tar ball and placed at the target location. There, is is untared, the environment variables are set, and the Fortran code is compiled. If the ECMWF environment is selected, a job script is prepared and submitted for the remaining configurations after putting the tar ball on the target ECMWF server.

Type: install.py –help to get information about command line parameters. Read the documentation for usage instructions.

install.check_install_conditions(c)[source]¶

Checks necessary attributes and conditions for the installation, e.g. whether they exist and contain values. Otherwise set default values.

Parameters: c (ControlFile) – Contains all the parameters of CONTROL file and command line.

install.del_fortran_build(src_path)[source]¶

Clean up the Fortran source directory and remove all build files (e.g. *.o, *.mod and FORTRAN EXECUTABLE)

Parameters: src_path (str) – Path to the fortran source directory.

install.get_install_cmdline_args()[source]¶

Decomposes the command line arguments and assigns them to variables. Apply default values for arguments not present.

Returns: args – Contains the commandline arguments from script/program call.
Return type: Namespace

install.install_local(c)[source]¶

Perform the actual installation on a local machine.

Parameters: c (ControlFile) – Contains all the parameters of CONTROL file and command line.

install.install_via_gateway(c)[source]¶

Prepare data transfer to remote gateway and submit a job script which will install everything on the remote gateway.

Parameters: c (ControlFile) – Contains all the parameters of CONTROL file and command line.

install.main()[source]¶: Controls the installation process. Calls the installation function if target is specified.

install.mk_compilejob(makefile, ecuid, ecgid, fp_root)[source]¶

Modifies the original job template file so that it is specified for the user and the environment were it will be applied. Result is stored in a new file “job.temp” in the python directory.

Parameters

makefile (str) – Name of the makefile which should be used to compile the Fortran program.
ecuid (str) – The user id on ECMWF server.
ecgid (str) – The group id on ECMWF server.
fp_root (str) – Path to the root directory of FLEXPART environment or flex_extract environment.

install.mk_env_vars(ecuid, ecgid, gateway, destination)[source]¶

Creates a file named ECMWF_ENV which contains the necessary environmental variables at ECMWF servers. It is based on the template ECMWF_ENV.template.

Parameters

ecuid (str) – The user id on ECMWF server.
ecgid (str) – The group id on ECMWF server.
gateway (str) – The gateway server the user is using.
destination (str) – The remote destination which is used to transfer files from ECMWF server to local gateway server.

install.mk_fortran_build(src_path, makefile)[source]¶

Compiles the Fortran code and generates the executable.

Parameters

src_path (str) – Path to the fortran source directory.
makefile (str) – The name of the makefile which should be used.

install.mk_job_template(ecuid, ecgid, fp_root)[source]¶

Modifies the original job template file so that it is specified for the user and the environment were it will be applied. Result is stored in a new file.

Parameters

ecuid (str) – The user id on ECMWF server.
ecgid (str) – The group id on ECMWF server.
fp_root (str) – Path to the root directory of FLEXPART environment or flex_extract environment.

install.mk_tarball(tarball_path, target)[source]¶

Creates a tarball with all necessary files which need to be sent to the installation directory. It does not matter whether this is local or remote. Collects all Python files, the Fortran source and makefiles, the ECMWF_ENV file, the CONTROL files as well as the template files.

Parameters

tarball_path (str) – The complete path to the tar file which will contain all relevant data for flex_extract.
target (str) – The queue where the job is submitted to.

install.un_tarball(tarball_path)[source]¶

Extracts the given tarball into current directory.

Parameters: tarball_path (str) – The complete path to the tar file which will contain all relevant data for flex_extract.

submit ¶

This script allows the user to extract meteorological fields from the ECMWF.

It prepares the settings for retrieving the data from ECMWF servers and process the resulting files to prepare them for the use with FLEXPART or FLEXTRA.

If it is supposed to work locally then it works through the necessary functions get_mars_data and prepare_flexpart. Otherwise it prepares a job script (korn shell) which will do the necessary work on the ECMWF server. This script will de submitted via the ecaccess command ecaccess-job-submit.

This file can also be imported as a module which then contains the following functions:

main - the main function of the script

submit - calls mk_jobscript depending on operation mode and submits its

mk_jobscript - creates the job script from a template

Type: submit.py –help to get information about command line parameters. Read the documentation for usage instructions.

submit.main()[source]¶: Get the arguments from script call and from CONTROL file. Decides from the argument “queue” if the local version is done “queue=None” or the gateway version with “queue=ecgate” or “queue=cca”.

submit.mk_jobscript(jtemplate, job_file, clist)[source]¶

Creates the job script from template.

Parameters

jtemplate (str) – Job template file from sub-directory “Templates” for submission to ECMWF. It contains all necessary module and variable settings for the ECMWF environment as well as the job call and mail report instructions. Default is _config.TEMPFILE_JOB.
job_file (str) – Path to the job script file.
clist (list of str) – Contains all necessary parameters for ECMWF CONTROL file.

submit.submit(jtemplate, c, queue)[source]¶

Prepares the job script and submits it to the specified queue.

Parameters

jtemplate (str) – Job template file from sub-directory “_templates” for submission to ECMWF. It contains all necessary module and variable settings for the ECMWF environment as well as the job call and mail report instructions. Default is _config.TEMPFILE_JOB.
c (ControlFile) – Contains all the parameters of CONTROL file and command line.
queue (str) – Name of queue for submission to ECMWF (e.g. ecgate or cca )

Classes ¶

ControlFile ¶

class ControlFile.ControlFile(filename)[source]¶

Contains the information which are stored in the CONTROL files.

The CONTROL file is the steering part of the FLEXPART extraction software. All necessary parameters needed to retrieve the data fields from the MARS archive for driving FLEXPART are set in a CONTROL file. Some specific parameters like the start and end dates can be overwritten by the command line parameters, but in generall all parameters needed for a complete set of fields for FLEXPART can be set in the CONTROL file.

controlfile¶

The name of the control file to be processed. Default value is the filename passed to the init function when initialised.

Type: str

start_date¶

The first day of the retrieval period. Default value is None.

Type: str

end_date¶

The last day of the retrieval period. Default value is None.

Type: str

date_chunk¶

Length of period for a single mars retrieval. Default value is 3.

Type: int

dtime¶

The time step in hours. Default value is None.

Type: str

basetime¶

The time for a half day retrieval. The 12 hours upfront are to be retrieved. Default value is None.

Type: int

maxstep¶

The maximum forecast step for non flux data. Default value is None.

Type: int

type¶

List of field type per retrieving hour. Default value is None.

Type: list of str

time¶

List of retrieving times in hours. Default valuer is None.

Type: list of str

step¶

List of forecast time steps in hours for non flux data. Default value is None.

Type: list of str or str

acctype¶

The field type for the accumulated forecast fields. Default value is None.

Type: str

acctime¶

The starting time of the accumulated forecasts. Default value is None.

Type: str

accmaxstep¶

The maximum forecast step for the accumulated forecast fields (flux data). Default value is None.

Type: int

marsclass¶

Characterisation of dataset. Default value is None.

Type: str

dataset¶

For public datasets there is the specific naming and parameter dataset which has to be used to characterize the type of data. Default value is None.

Type: str

stream¶

Identifies the forecasting system used to generate the data. Default value is None.

Type: str

number¶

Selects the member in ensemble forecast run. Default value is ‘OFF’.

Type: str

expver¶

The version number of the dataset. Default value is ‘1’.

Type: str

gaussian¶

This parameter is deprecated and should no longer be used. Specifies the desired type of Gaussian grid for the output. Default value is an empty string ‘’.

Type: str

grid¶

Specifies the output grid which can be either a Gaussian grid or a Latitude/Longitude grid. Default value is None.

Type: str

area¶

Specifies the desired sub-area of data to be extracted. Default value is None.

Type: str

left¶

The western most longitude of the area to be extracted. Default value is None.

Type: str

lower¶

The southern most latitude of the area to be extracted. Default value is None.

Type: str

upper¶

The northern most latitued of the area to be extracted. Default value is None.

Type: str

right¶

The eastern most longitude of the area to be extracted. Default value is None.

Type: str

level¶

Specifies the maximum level. Default value is None.

Type: str

levelist¶

Specifies the required level list. Default value is None.

Type: str

resol¶

Specifies the desired triangular truncation of retrieved data, before carrying out any other selected post-processing. Default value is None.

Type: str

gauss¶

Switch to select gaussian fields (1) or regular lat/lon (0). Default value is 0.

Type: int

accuracy¶

Specifies the number of bits per value to be used in the generated GRIB coded fields. Default value is 24.

Type: int

omega¶

Switch to select omega retrieval (1) or not (0). Default value is 0.

Type: int

omegadiff¶

Switch to decide to calculate Omega and Dps/Dt from continuity equation for diagnostic purposes (1) or not (0). Default value is 0.

Type: int

eta¶

Switch to select direct retrieval of etadot from MARS (1) or wether it has to be calculated (0). Then Default value is 0.

Type: int

etadiff¶

Switch to select calculation of etadot and Dps/Dt from continuity equation for diagnostic purposes (1) or not (0). Default value is 0.

Type: int

etapar¶

GRIB parameter Id for etadot fields. Default value is 77.

Type: int

dpdeta¶

Switch to select multiplication of etadot with dpdeta. Default value is 1.

Type: int

smooth¶

Spectral truncation of ETADOT after calculation on Gaussian grid. Default value is 0.

Type: int

format¶

The format of the GRIB data. Default value is ‘GRIB1’.

Type: str

addpar¶

List of additional surface level ECMWF parameter to be retrieved. Default value is None.

Type: str

prefix¶

Prefix string for the final FLEXPART/FLEXTRA ready input files. Default value is ‘EN’.

Type: str

cwc¶

Switch to select wether the sum of cloud liquid water content and cloud ice water content should be retrieved. Default value is 0.

Type: int

wrf¶

Switch to select further parameters for retrievment to support WRF simulations. Default value is 0.

Type: int

ecfsdir¶

Path to the ECMWF storage ‘ectmp:/${USER}/econdemand/’

Type: str

mailfail¶

Email list for sending error log files from ECMWF servers. The email addresses should be seperated by a comma. Default value is [‘${USER}’].

Type: list of str

mailops¶

Email list for sending operational log files from ECMWF servers. The email addresses should be seperated by a comma. Default value is [‘${USER}’].

Type: list of str

ecstorage¶

Switch to select storage of FLEXPART ready output files in the ECFS file system. Default value is 0.

Type: int

ectrans¶

Switch to select the transfer of FLEXPART ready output files to the gateway server. Default value is 0.

Type: int

inputdir¶

Path to the temporary directory for the retrieval grib files and other processing files. Default value is _config.PATH_INPUT_DIR.

Type: str

outputdir¶

Path to the final directory where the final FLEXPART ready input files are stored. Default value is None.

Type: str

flexextractdir¶

Path to the flex_extract root directory. Default value is _config.PATH_FLEXEXTRACT_DIR.

Type: str

exedir¶

Path to the FORTRAN executable file. Default value is _config.PATH_FORTRAN_SRC.

Type: str

installdir¶

Path to a FLEXPART root directory. Default value is None.

Type: str

makefile¶

Name of the makefile to be used for the Fortran program. Default value is None.

Type: str

destination¶

The remote destination which is used to transfer files from ECMWF server to local gateway server. Default value is None.

Type: str

gateway¶

The gateway server the user is using. Default value is None.

Type: str

ecuid¶

The user id on ECMWF server. Default value is None.

Type: str

ecgid¶

The group id on ECMWF server. Default value is None.

Type: str

install_target¶

Defines the location where the installation is to be done. Default value is None.

Type: str

debug¶

Switch to keep temporary files at the end of postprocessing (1) or to delete all temporary files except the final output files (0). Default value is 0.

Type: int

oper¶

Switch to prepare the operational job script. Start date, end date and basetime will be prepared with environment variables. Default value is 0.

Type: int

request¶

Switch to select between just retrieving the data (0), writing the mars parameter values to a csv file (1) or doing both (2). Default value is 0.

Type: int

public¶

Switch to select kind of ECMWF Web Api access and the possible data sets. Public data sets (1) and Memberstate data sets (0). Default value is 0.

Type: int

ec_api¶

Tells wether the ECMWF Web API was able to load or not. Default value is None.

Type: boolean

cds_api¶

Tells wether the CDS API was able to load or not. Default value is None.

Type: boolean

purefc¶

Switch to decide wether the job is a pure forecast retrieval or coupled with analysis data. Default value is 0.

Type: int

rrint¶

Switch to select between old precipitation disaggregation method (0) or the new IA3 disaggegration method (1). Default value is 0.

Type: int

doubleelda¶

Switch to select the calculation of extra ensemble members for the ELDA stream. It doubles the amount of retrieved ensemble members.

Type: int

logicals¶

List of the names of logical switches which controls the flow of the program. Default list is [‘gauss’, ‘omega’, ‘omegadiff’, ‘eta’, ‘etadiff’, ‘dpdeta’, ‘cwc’, ‘wrf’, ‘ecstorage’, ‘ectrans’, ‘debug’, ‘request’, ‘public’, ‘purefc’, ‘rrint’, ‘doubleelda’]

Type: list of str

assign_args_to_control(args)[source]¶

Overwrites the existing ControlFile instance attributes with the command line arguments.

Parameters: args (Namespace) – Contains the commandline arguments from script/program call.

assign_envs_to_control(envs)[source]¶

Assigns the ECMWF environment parameter.

Parameters: envs (dict of str) – Contains the ECMWF environment parameternames “ECUID”, “ECGID”, “DESTINATION” and “GATEWAY” with its corresponding values. They were read from the file “ECMWF_ENV”.

check_conditions(queue)[source]¶

Checks a couple of necessary attributes and conditions, such as if they exist and contain values. Otherwise set default values.

Parameters: queue (str) – Name of the queue if submitted to the ECMWF servers. Used to check if ecuid, ecgid, gateway and destination are set correctly and are not empty.

to_list()[source]¶

Just generates a list of strings containing the attributes and assigned values except the attributes “_expanded”, “exedir”, “flexextractdir” and “installdir”.

Returns: l – A sorted list of the all ControlFile class attributes with their values except the attributes “_expanded”, “exedir”, “flexextractdir” and “installdir”.
Return type: list of *

EcFlexpart ¶

class EcFlexpart.EcFlexpart(c, fluxes=False)[source]¶

Class to represent FLEXPART specific ECMWF data.

FLEXPART needs grib files in a specifc format. All necessary data fields for one time step are stored in a single file. The class represents an instance with all the parameter and settings necessary for retrieving MARS data and modifing them so they are fitting FLEXPART needs. The class is able to disaggregate the fluxes and convert grid types to the one needed by FLEXPART, therefore using the FORTRAN program.

mreq_count¶

Counter for the number of generated mars requests.

Type: int

inputdir¶

Path to the directory where the retrieved data is stored.

Type: str

dataset¶

For public datasets there is the specific naming and parameter dataset which has to be used to characterize the type of data.

Type: str

basetime¶

The time for a half day retrieval. The 12 hours upfront are to be retrieved.

Type: int

dtime¶

Time step in hours.

Type: str

acctype¶

The field type for the accumulated forecast fields.

Type: str

acctime¶

The starting time from the accumulated forecasts.

Type: str

accmaxstep¶

The maximum forecast step for the accumulated forecast fields.

Type: str

marsclass¶

Characterisation of dataset.

Type: str

stream¶

Identifies the forecasting system used to generate the data.

Type: str

number¶

Selects the member in ensemble forecast run.

Type: str

resol¶

Specifies the desired triangular truncation of retrieved data, before carrying out any other selected post-processing.

Type: str

accuracy¶

Specifies the number of bits per value to be used in the generated GRIB coded fields.

Type: str

addpar¶

List of additional parameters to be retrieved.

Type: str

level¶

Specifies the maximum level.

Type: str

expver¶

The version of the dataset.

Type: str

levelist¶

Specifies the required levels.

Type: str

glevelist¶

Specifies the required levels for gaussian grids.

Type: str

gaussian¶

This parameter is deprecated and should no longer be used. Specifies the desired type of Gaussian grid for the output.

Type: str

grid¶

Specifies the output grid which can be either a Gaussian grid or a Latitude/Longitude grid.

Type: str

area¶

Specifies the desired sub-area of data to be extracted.

Type: str

purefc¶

Switch for definition of pure forecast mode or not.

Type: int

outputfilelist¶

The final list of FLEXPART ready input files.

Type: list of str

types¶

Determines the combination of type of fields, time and forecast step to be retrieved.

Type: dictionary

params¶

Collection of grid types and their corresponding parameters, levels, level types and the grid definition.

Type: dictionary

server¶

This is the connection to the ECMWF data servers.

Type: ECMWFService or ECMWFDataServer

public¶

Decides which Web API Server version is used.

Type: int

dates¶

Contains start and end date of the retrieval in the format “YYYYMMDD/to/YYYYMMDD”

Type: str

calc_extra_elda(path, prefix)[source]¶

Calculates extra ensemble members for ELDA - Stream.

This is a specific feature which doubles the number of ensemble members for the ELDA Stream.

Parameters

path (str) – Path to the output files.
prefix (str) – The prefix of the output filenames as defined in Control file.

create(inputfiles, c)[source]¶

An index file will be created which depends on the combination of “date”, “time” and “stepRange” values. This is used to iterate over all messages in each grib file which were passed through the parameter “inputfiles” to seperate specific parameters into fort.* files. Afterwards the FORTRAN program is called to convert the data fields all to the same grid and put them in one file per unique time step (combination of “date”, “time” and “stepRange”).

Note

This method is based on the ECMWF example index.py https://software.ecmwf.int/wiki/display/GRIB/index.py

Parameters

inputfiles (UioFiles) – Contains a list of files.
c (ControlFile) – Contains all the parameters of CONTROL file and command line.

deacc_fluxes(inputfiles, c)[source]¶

De-accumulate and disaggregate flux data.

Goes through all flux fields in ordered time and de-accumulate the fields. Afterwards the fields are disaggregated in time. Different versions of disaggregation is provided for rainfall data (darain, modified linear) and the surface fluxes and stress data (dapoly, cubic polynomial).

Parameters

inputfiles (UioFiles) – Contains the list of files that contain flux data.
c (ControlFile) – Contains all the parameters of CONTROL file and command line.

process_output(c)[source]¶

Postprocessing of FLEXPART input files.

The grib files are postprocessed depending on the selection in CONTROL file. The resulting files are moved to the output directory if its not equal to the input directory. The following modifications might be done if properly switched in CONTROL file: GRIB2 - Conversion to GRIB2 ECTRANS - Transfer of files to gateway server ECSTORAGE - Storage at ECMWF server

Parameters: c (ControlFile) – Contains all the parameters of CONTROL file and command line.

retrieve(server, dates, public, request, inputdir='.')[source]¶

Finalizing the retrieval information by setting final details depending on grid type. Prepares MARS retrievals per grid type and submits them.

Parameters

server (ECMWFService or ECMWFDataServer) – The connection to the ECMWF server. This is different for member state users which have full access and non member state users which have only access to the public data sets. The decision is made from command line argument “public”; for public access its True (ECMWFDataServer) for member state users its False (ECMWFService)
dates (str) – Contains start and end date of the retrieval in the format “YYYYMMDD/to/YYYYMMDD”
request (int) – Selects the mode of retrieval. 0: Retrieves the data from ECMWF. 1: Prints the mars requests to an output file. 2: Retrieves the data and prints the mars request.
inputdir (str, optional) – Path to the directory where the retrieved data is about to be stored. The default is the current directory (‘.’).

write_namelist(c)[source]¶

Creates a namelist file in the temporary directory and writes the following values to it: maxl, maxb, mlevel, mlevelist, mnauf, metapar, rlo0, rlo1, rla0, rla1, momega, momegadiff, mgauss, msmooth, meta, metadiff, mdpdeta

Parameters

c (ControlFile) – Contains all the parameters of CONTROL file and command line.
filename (str) – Name of the namelist file.

GribUtil ¶

class GribUtil.GribUtil(filenames)[source]¶

Class for GRIB utilities (new methods) based on GRIB API

The GRIB API provides all necessary tools to work directly with the grib files. Nevertheless, the GRIB API tools are very basic and are in direct connection with the grib files. This class provides some higher functions which apply a set of GRIB API tools together in the respective context. So, the class initially contains a list of grib files (their names) and the using program then applies the methods directly on the class objects without having to think about how the actual GRIB API tools have to be arranged.

copy_dummy_msg(filename_in, keynames, keyvalues, selectwhere=True, filemode='wb')[source]¶

Add the content of another input grib file to the objects file but only messages corresponding to keys/values passed to the function. The selectWhere switch decides if to copy the keys equal to (True) or different to (False) the keynames/keyvalues list passed to the function.

Parameters

filename_in (string) – Filename of the input file to read the grib messages from.
selectwhere (boolean, optional) – Decides if to copy the keynames and values equal to (True) or different to (False) the keynames/keyvalues list passed to the function. Default is True.
keynames (list of string) – List of keynames.
keyvalues (list of string) – List of keyvalues.
filemode (string, optional) – Sets the mode for the output file. Default is “wb”.

get_keys(keynames, wherekeynames, wherekeyvalues)[source]¶

Get keyvalues for a given list of keynames a where statement can be given (list of key and list of values)

Parameters

keynames (list of string) – List of keynames.
wherekeynames (list of string) – List of key names for indexing grib message parameter.
wherekeyvalues (list of string) – List of key values corresponding the key names.

Returns

return_list – List of keyvalues for given keynames.

Return type

list of string

index(index_keys, index_file='my.idx')[source]¶

Create index file from a list of files if it does not exist or read an index file.

Parameters

index_keys (list of string) – Contains the list of key parameter names from which the index is to be created.
index_file (string, optional) – Filename where the indices are stored. Default is “my.idx”.

Returns

iid – Grib index id.

Return type

integer

set_keys(fromfile, filemode='wb', keynames=[], keyvalues=[], wherekeynames=[], wherekeyvalues=[])[source]¶

Opens the file to read the grib messages and then write the selected messages (with wherekeys) to a new output file. Also, the keyvalues of the passed list of keynames are set.

Parameters

fromfile (string) – Filename of the input file to read the grib messages from.
keynames (list of string) – List of keynames to set in the selected messages. Default is an empty list.
keyvalues (list of string) – List of keyvalues to set in the selected messages. Default is an empty list.
wherekeynames (list of string) – List of keynames to select correct message.
wherekeyvalues (list of string) – List of keyvalues for keynames to select correct message.
filemode (string, optional) – Sets the mode for the output file. Default is “wb”.

MarsRetrieval ¶

class MarsRetrieval.MarsRetrieval(server, public, marsclass='EA', dataset='', type='', levtype='', levelist='', repres='', date='', resol='', stream='', area='', time='', step='', expver='1', number='', accuracy='', grid='', gaussian='', target='', param='')[source]¶

Specific syntax and content for submission of MARS retrievals.

A MARS revtrieval has a specific syntax with a selection of keywords and their corresponding values. This class provides the necessary functions by displaying the selected parameters and their values and the actual retrievement of the data through a mars request or a Python web api interface. The initialization already expects all the keyword values.

A description of MARS keywords/arguments and examples of their values can be found here: https://software.ecmwf.int/wiki/display/UDOC/ Identification+keywords#Identificationkeywords-class

server¶

This is the connection to the ECMWF data servers.

Type: ECMWFService or ECMWFDataServer

public¶

Decides which Web API Server version is used.

Type: int

marsclass¶

Characterisation of dataset.

Type: str, optional

dataset¶

For public datasets there is the specific naming and parameter dataset which has to be used to characterize the type of data.

Type: str, optional

type¶

Determines the type of fields to be retrieved.

Type: str, optional

levtype¶

Denotes type of level.

Type: str, optional

levelist¶

Specifies the required levels.

Type: str, optional

repres¶

Selects the representation of the archived data.

Type: str, optional

date¶

Specifies the Analysis date, the Forecast base date or Observations date.

Type: str, optional

resol¶

Specifies the desired triangular truncation of retrieved data, before carrying out any other selected post-processing.

Type: str, optional

stream¶

Identifies the forecasting system used to generate the data.

Type: str, optional

area¶

Specifies the desired sub-area of data to be extracted.

Type: str, optional

time¶

Specifies the time of the data in hours and minutes.

Type: str, optional

step¶

Specifies the forecast time step from forecast base time.

Type: str, optional

expver¶

The version of the dataset.

Type: str, optional

number¶

Selects the member in ensemble forecast run.

Type: str, optional

accuracy¶

Specifies the number of bits per value to be used in the generated GRIB coded fields.

Type: str, optional

grid¶

Specifies the output grid which can be either a Gaussian grid or a Latitude/Longitude grid.

Type: str, optional

gaussian¶

This parameter is deprecated and should no longer be used. Specifies the desired type of Gaussian grid for the output.

Type: str, optional

target¶

Specifies a file into which data is to be written after retrieval or manipulation.

Type: str, optional

param¶

Specifies the meteorological parameter.

Type: str, optional

data_retrieve()[source]¶: Submits a MARS retrieval. Depending on the existence of ECMWF Web-API or CDS API it is submitted via Python or a subprocess in the Shell. The parameter for the mars retrieval are taken from the defined class attributes.

display_info()[source]¶: Prints all class attributes and their values to the standard output.

print_infodata_csv(inputdir, request_number)[source]¶

Write all request parameter in alpabetical order into a “csv” file.

Parameters

inputdir (str) – The path where all data from the retrievals are stored.
request_number (int) – Number of mars requests for flux and non-flux data.

UioFiles ¶

class UioFiles.UioFiles(path, pattern)[source]¶

Collection of files matching a specific pattern.

The pattern can contain regular expressions for the files. The files are listed and can be transformed to a single string or they can be deleted.

path¶

Directory where to list the files.

Type: str

pattern¶

Regular expression pattern. For example: ‘*.grb’

Type: str

files¶

List of files matching the pattern in the path.

Type: list of str

delete_files()[source]¶: Deletes the files.

Modules ¶

get_mars_data ¶

This script extracts MARS data from ECMWF.

At first, the necessary parameters from command line and CONTROL files are extracted. They define the data set to be extracted from MARS.

This file can also be imported as a module and contains the following functions:

main - the main function of the script

get_mars_data - overall control of ECMWF data retrievment

write_reqheader - writes the header into the mars_request file

mk_server - creates the server connection to ECMWF servers

mk_dates - defines the start and end date

remove_old - deletes old retrieved grib files

do_retrieval - creates individual retrievals

Type get_mars_data.py –help to get information about command line parameters. Read the documentation for usage instructions.

get_mars_data.check_dates_for_nonflux_fc_times(types, times)[source]¶

Checks if the time 18UTC corresponds to forecast field.

Parameters

types (list of str) – List of field types.
times (list of str or str) – The time in hours of the field.

Returns

Return type

True or False

get_mars_data.do_retrievement(c, server, start, end, delta_t, fluxes=False)[source]¶

Divides the total retrieval period into smaller chunks and retrieves the data from MARS.

Parameters

c (ControlFile) – Contains all the parameters of CONTROL file and command line.
server (ECMWFService or ECMWFDataServer) – The server connection to ECMWF.
start (datetime) – The start date of the retrieval.
end (datetime) – The end date of the retrieval.
delta_t (datetime) – Delta_t + 1 is the maximum time period of a single retrieval.
fluxes (boolean, optional) – Decides if the flux parameters are to be retrieved or the rest of the parameter list. Default value is False.

get_mars_data.get_mars_data(c)[source]¶

Retrieves the ECMWF data required for a FLEXPART simulation.

Start and end dates for retrieval period are set. Retrievals are divided into shorter periods if necessary and if datechunk parameter is set.

Parameters: c (ControlFile) – Contains all the parameters of CONTROL file and command line.

get_mars_data.main()[source]¶

Controls the program to retrieve data from MARS.

This is done if called directly from command line. Then, arguments and control file are taken as input.

get_mars_data.mk_dates(c, fluxes)[source]¶

Prepares start and end date depending on flux or non-flux type of data.

If forecasts for a maximum of one day (24 h) are to be retrieved, then collect accumulation data (flux data) with additional days in the beginning and at the end (needed for complete disaggregation of original period)

If forecast data for more than +24 h are to be retrieved, then collect accumulation data (flux data) with the exact start and end date (disaggregation will be done for the exact time period with boundary conditions)

Since for basetime the extraction contains the 12 hours upfront, if basetime is 0, the starting date has to be the day before

Parameters

c (ControlFile) – Contains all the parameters of CONTROL file and command line.
fluxes (boolean, optional) – Decides if the flux parameter settings are stored or the rest of the parameter list. Default value is False.

Returns

start (datetime) – The start date of the retrieving data set.
end (datetime) – The end date of the retrieving data set.
chunk (datetime) – Time period in days for one single mars retrieval.

get_mars_data.mk_server(c)[source]¶

Creates a server connection with available Python API.

The API selected depends on availability and the data set to be retrieved. The CDS API is used for ERA5 data, no matter whether the user is a member-state or a public user. ECMWF WebAPI is used for all other available datasets.

Parameters: c (ControlFile) – Contains all the parameters of CONTROL file and command line.
Returns: server – Connection to ECMWF server via python interface ECMWF WebAPI or CDS API.
Return type: ECMWFDataServer, ECMWFService or Client

get_mars_data.remove_old(pattern, inputdir)[source]¶

Deletes old retrieval files from current input directory matching the pattern.

Parameters

pattern (str) – The substring pattern which identifies the files to be deleted.
inputdir (str, optional) – Path to the directory where the retrieved data are stored.

get_mars_data.write_reqheader(marsfile)[source]¶

Writes header with column names into MARS request file.

Parameters: marsfile (str) – Path to the MARS request file.

prepare_flexpart ¶

This script prepares the final version of the grib files which are then used by FLEXPART.

It converts the bunch of grib files extracted via get_mars_data before, by doing the necessary conversion to obtain consistent grids or the disaggregation of flux data. Finally, the data fields are combined in files per hour available with the naming convention xxYYMMDDHH, where xx should be 2 arbitrary letters (mostly xx is chosen to be “EN”).

This file can also be imported as a module which then contains the following functions:

main

prepare_flexpart

Type: prepare_flexpart.py –help to get information about command line parameters. Read the documentation for usage instructions.

prepare_flexpart.main()[source]¶

Controls the program to prepare FLEXPART input files from MARS data.

This is done if called directly from the command line. Then, arguments and control file are taken as input.

prepare_flexpart.prepare_flexpart(ppid, c)[source]¶

Converts the MARS data into files ready as input for FLEXPART.

Certain fields are converted to a different grid and the flux data are disaggregated. Fields are collected by hour and stored in a file with a specific naming convention.

Parameters

ppid (int) – Contains the ppid number of the current ECMWF job. It will be None if the method was called within this module.
c (ControlFile) – Contains all the parameters of CONTROL file and command line.

tools ¶

This module contains a collection of diverse tasks within flex_extract.

tools.clean_up(c)[source]¶

Remove files from the intermediate directory (inputdir).

It keeps the final FLEXPART input files if program runs without ECMWF API and keywords “ectrans” or “ecstorage” are set to “1”.

Parameters: c (ControlFile) – Contains all the parameters of CONTROL file and command line.

tools.execute_subprocess(cmd_list, error_msg='SUBPROCESS FAILED!')[source]¶

Executes a command via a subprocess.

Error handling is done if an error occures.

Parameters: cmd_list (list of str) – A list of the components for the command line execution. They will be concatenated with blank space for the command to be submitted, like [‘mv’, file1, file2] for mv file1 file2.
Returns: error_msg – Error message if the subprocess fails. By default it will just say “SUBPROCESS FAILED!”.
Return type: str, optional

tools.generate_retrieval_period_boundary(c)[source]¶

Generates retrieval period boundary datetimes from CONTROL information.

Parameters

c (ControlFile) – Contains all the parameters of CONTROL file and command line.

Returns

start_period (datetime) – The first timestamp of the actual retrieval period disregarding the temporary times which were used for processing reasons.
end_period (datetime) – The last timestamp of the actual retrieval period disregarding the temporary times which were used for processing reasons.

tools.get_cmdline_args()[source]¶

Decomposes the command line arguments and assigns them to variables. Apply default values for arguments not present.

Returns: args – Contains the command line arguments from the script / program call.
Return type: Namespace

tools.get_dimensions(info, purefc, dtime, index_vals, start_date, end_date)[source]¶

This function specifies the correct dimensions for x, y, and t.

Parameters

info (dict) – Contains basic informations of the ECMWF grib files, e.g. ‘Ni’, ‘Nj’, ‘latitudeOfFirstGridPointInDegrees’, ‘longitudeOfFirstGridPointInDegrees’, ‘latitudeOfLastGridPointInDegrees’, ‘longitudeOfLastGridPointInDegrees’, ‘jDirectionIncrementInDegrees’, ‘iDirectionIncrementInDegrees’, ‘missingValue’
purefc (int) – Switch for definition of pure forecast mode or not.
dtime (str) – Time step in hours.
index_vals (list of list of str) – Contains the values from the keys used for a distinct selection of GRIB messages in processing the grib files. Content looks like e.g.: index_vals[0]: (‘20171106’, ‘20171107’, ‘20171108’) ; date index_vals[1]: (‘0’, ‘1200’, ‘1800’, ‘600’) ; time index_vals[2]: (‘0’, ‘12’, ‘3’, ‘6’, ‘9’) ; stepRange
start_date (str) – The start date of the retrieval job.
end_date (str) – The end date of the retrieval job.

Returns

(ix, jy, it) – Dimension in x-direction, y-direction and in time.

Return type

tuple of int

tools.get_informations(filename)[source]¶

Extracts basic information from a sample grib file.

This information is needed for later use and the initialization of numpy arrays where data are stored.

Parameters: filename (str) – Name of the file which will be opened to extract basic information.
Returns: data – Contains basic informations of the ECMWF grib files, e.g. ‘Ni’, ‘Nj’, ‘latitudeOfFirstGridPointInDegrees’, ‘longitudeOfFirstGridPointInDegrees’, ‘latitudeOfLastGridPointInDegrees’, ‘longitudeOfLastGridPointInDegrees’, ‘jDirectionIncrementInDegrees’, ‘iDirectionIncrementInDegrees’, ‘missingValue’
Return type: dict

tools.get_list_as_string(list_obj, concatenate_sign=', ')[source]¶

Converts a list of arbitrary content into a single string using a given concatenation character.

Parameters

list_obj (list of *) – A list with arbitrary content.
concatenate_sign (str, optional) – A string which is used to concatenate the single list elements. Default value is “, “.

Returns

str_of_list – The content of the list as a single string.

Return type

str

tools.init128(filepath)[source]¶

Opens and reads the grib file with table 128 information.

Parameters: filepath (str) – Path to file of ECMWF grib table number 128.
Returns: table128 – Contains the ECMWF grib table 128 information. The key is the parameter number and the value is the short name of the parameter.
Return type: dict

tools.make_dir(directory)[source]¶

Creates a directory.

If the directory already exists, an information is printed and the creation skipped. The program stops only if there is another problem.

Parameters: directory (str) – The path to directory which should be created.

tools.my_error(message='ERROR')[source]¶

Prints a specified error message which can be passed to the function before exiting the program.

Parameters: message (str, optional) – Error message. Default value is “ERROR”.

tools.none_or_int(value)[source]¶

Converts the input string into Pythons None-type if it contains string “None”; otherwise it is converted to an integer value.

Parameters: value (str) – String to be checked for the “None” word.
Returns: Return depends on the content of the input value. If it was “None”, then the python type None is returned. Otherwise the string is converted into an integer value.
Return type: None or int(value)

tools.none_or_str(value)[source]¶

Converts the input string into Pythons None type if it contains the string “None”.

Parameters: value (str) – String to be checked for the “None” word.
Returns: Return depends on the content of the input value. If it was “None”, then the Python type None is returned, otherwise the string itself.
Return type: None or value

tools.normal_exit(message='Done!')[source]¶

Prints a specific exit message which can be passed to the function.

Parameters: message (str, optional) – Message for exiting program. Default value is “Done!”.

tools.product(*args, **kwds)[source]¶

Creates combinations of all passed arguments.

This method combines the single characters of the passed arguments with each other in a way that each character of each argument value will be combined with each character of the other arguments as a tuple.

Note

This method is taken from an example at the ECMWF wiki website. https://software.ecmwf.int/wiki/display/GRIB/index.py; 2018-03-16

It was released under the following license: https://confluence.ecmwf.int/display/ECC/License

Example

product(‘ABCD’, ‘xy’) –> Ax Ay Bx By Cx Cy Dx Dy

product(range(2), repeat = 3) –> 000 001 010 011 100 101 110 111

Parameters

*args (list or str) – Positional arguments (arbitrary number).
**kwds (dict) – Contains all the keyword arguments from *args.

Returns

prod – Return will be done with “yield”. A tuple of combined arguments. See example in description above.

Return type

tuple

tools.put_file_to_ecserver(ecd, filename, target, ecuid, ecgid)[source]¶

Uses the ecaccess-file-put command to send a file to the ECMWF servers.

Note

The return value is just for testing reasons. It does not have to be used from the calling function since the whole error handling is done in here.

Parameters

ecd (str) – The path were the file is stored.
filename (str) – The name of the file to send to the ECMWF server.
target (str) – The target queue where the file should be sent to.
ecuid (str) – The user id on ECMWF server.
ecgid (str) – The group id on ECMWF server.

tools.read_ecenv(filepath)[source]¶

Reads the file into a dictionary where the key values are the parameter names.

Parameters: filepath (str) – Path to file where the ECMWF environment parameters are stored.
Returns: envs – Contains the environment parameter ecuid, ecgid, gateway and destination for ECMWF server environments.
Return type: dict

tools.send_mail(users, success_mode, message)[source]¶

Prints a specific exit message which can be passed to the function.

Parameters

users (list of str) – Contains all email addresses which should be notified. It might also contain just the ecmwf user name which wil trigger mailing to the associated email address for this user.
success_mode (str) – States the exit mode of the program to put into the mail subject line.
message (str, optional) – Message for exiting program. Default value is “Done!”.

tools.setup_controldata()[source]¶

Collects, stores and checks controlling arguments from command line, CONTROL file and ECMWF_ENV file.

Returns

c (ControlFile) – Contains all the parameters of CONTROL file and command line.
ppid (str) – Parent process id.
queue (str) – Name of queue for submission to ECMWF (e.g. ecgate or cca )
job_template (str) – Name of the job template file for submission to ECMWF server.

tools.silent_remove(filename)[source]¶

Remove file if it exists. The function does not fail if the file does not exist.

Parameters: filename (str) – The name of the file to be removed without notification.

tools.submit_job_to_ecserver(target, jobname)[source]¶

Uses ecaccess-job-submit command to submit a job to the ECMWF server.

Note

The return value is just for testing reasons. It does not have to be used from the calling function since the whole error handling is done in here.

Parameters

target (str) – The target where the file should be sent to, e.g. the queue.
jobname (str) – The name of the jobfile to be submitted to the ECMWF server.

Returns

job_id – The id number of the job as a reference at the ECMWF server.

Return type

int

tools.to_param_id(pars, table)[source]¶

Transform parameter names to parameter ids with ECMWF grib table 128.

Parameters

pars (str) – Addpar argument from CONTROL file in the format of parameter names instead of IDs. The parameter short names are separated by “/” and passed as one single string.
table (dict) – Contains the ECMWF grib table 128 information. The key is the parameter number and the value is the short name of the parameter.

Returns

ipar – List of addpar parameters from CONTROL file transformed to parameter ids in the format of integer.

Return type

list of int

tools.to_param_id_with_tablenumber(pars, table)[source]¶

Transform parameter names to parameter IDs and add table ID.

Conversion with ECMWF grib table 128.

Parameters

pars (str) – Addpar argument from CONTROL file in the format of parameter names instead of ID. The parameter short names are separated by “/” and passed as one single string.
table (dict) – Contains the ECMWF grib table 128 information. The key is the parameter number and the value is the short name of the parameter.

Returns

spar – List of addpar parameters from CONTROL file transformed to parameter IDs in the format of integer.

Return type

str

checks ¶

This module contains check methods for the CONTROL paramaters.

checks.check_accmaxstep(accmaxstep, marsclass, purefc, maxstep)[source]¶

Guarantees that the accumulation forecast step was set.

Parameters

accmaxstep (str) – The maximum forecast step for the accumulated forecast fields.
marsclass (str) – ECMWF data classification identifier.
purefc (int) – Switch for definition of pure forecast mode or not.
maxstep (str) – The maximum forecast time step in hours from the forecast base time. This is the maximum step for non-flux (accumulated) forecast data.

Returns

accmaxstep – The maximum forecast step for the accumulated forecast fields.

Return type

str

checks.check_acctime(acctime, marsclass, purefc, time)[source]¶

Guarantees that the accumulation forecast times were set.

If not set, setting the value to some of the most commonly used data sets is attempted. Otherwise, an eror is raised.

Parameters

acctime (str) – The starting time for the accumulated forecasts.
marsclass (str) – ECMWF data classification identifier.
purefc (int) – Switch for definition of pure forecast mode or not.

Returns

acctime – The starting time for the accumulated forecasts.

Return type

str

checks.check_acctype(acctype, ftype)[source]¶

Guarantees that the accumulation field type is set.

If not set, it is derived with the old method (TYPE[1]).

Parameters

acctype (str) – The field type for the accumulated forecast fields.
ftype (list of str) – List of field types.

Returns

acctype – The field type for the accumulated forecast fields.

Return type

str

checks.check_addpar(addpar)[source]¶

Check that addpar has correct format of additional parameters in a single string, so that it can be easily appended to the hard-coded parameters retrieved in any case.

Parameters: addpar (str or list of str) – List of additional parameters to be retrieved.
Returns: addpar – List of additional parameters to be retrieved.
Return type: str

checks.check_area(grid, area, upper, lower, left, right)[source]¶

Defines the correct area string.

Checks the format of the four area components wether it is on the order of 1000 or 1. Also checks wether area was already set on command line, then the four components are overwritten. Convert to correct format of the order of magnitude “1” and sets the area parameter (North/West/South/East). E.g.: -5./20./10./10.

Parameters

grid (str) – Contains grid information.
area (str) – Contains area informtion.
upper (str) – The northernmost latitude.
lower (str) – The southernmost latitude.
left (str) – The westernmost longitude.
right (str) – The easternmost longitude.

Returns

grid – Contains grid in format lat/lon. E.g. 0.1/0.1

Return type

str

checks.check_basetime(basetime)[source]¶

Check if basetime is set and contains one of the two possible values (0, 12).

Parameters: basetime (int or str or None) – The time for a half day retrieval. The 12 hours upfront are to be retrieved.
Returns: basetime – The time for a half day retrieval. The 12 hours upfront are to be retrieved.
Return type: int or None

checks.check_dates(start, end)[source]¶

Checks if there is at least a start date for a one-day retrieval.

Checks if end date lies after start date and end date is set.

Parameters

start (str) – The start date of the retrieval job.
end (str) – The end date of the retrieval job.

Returns

start (str) – The start date of the retrieval job.
end (str) – The end date of the retrieval job.

checks.check_grid(grid)[source]¶

Convert grid into correct Lat/Lon format. E.g. ‘0.5/0.5’

Checks on format of original grid. Wether it is in the order of 1000 or 1. Convert to correct grid format and substitute into “Lat/Lon” format string.

Parameters: grid (str) – Contains grid information
Returns: grid – Contains grid in format Lat/lon. E.g. 0.1/0.1
Return type: str

checks.check_job_chunk(job_chunk)[source]¶

Checks that if job chunk is set, the number is positive and nonzero.

Parameters: job_chunk (int) – The number of days for a single job script.
Returns: job_chunk – The number of days for a single job script.
Return type: int

checks.check_len_type_time_step(ftype, ftime, steps, maxstep, purefc)[source]¶

Check if

Parameters

ftype (list of str) – List of field types.
ftime (list of str or str) – The time in hours of the field.
steps (str) – Specifies the forecast time step from forecast base time. Valid values are hours (HH) from forecast base time.
maxstep (int) – The maximum forecast time step in hours from the forecast base time. This is the maximum step for non-flux (not accumulated) forecast data.
purefc (int) – Switch for definition of pure forecast mode or not.

Returns

ftype (list of str) – List of field types.
ftime (list of str) – The time in hours of the field.
steps (str) – Specifies the forecast time step from forecast base time. Valid values are hours (HH) from forecast base time.

checks.check_levels(levelist, level)[source]¶

Defines correct level list and guarantees that the maximum level is one of the available maximum levels.

Parameters

levelist (str) – Specifies the level list. Examples: model level: 1/to/137, pressure levels: 500/to/1000
level (str) – Specifies the maximum level.

Returns

levelist (str) – Specifies the required levels. It has to have a valid correspondence to the selected levtype. Examples: model level: 1/to/137, pressure levels: 500/to/1000
level (str) – Specifies the maximum level. It has to be one of the available maximum level numbers as contained in the variable MAX_LEVEL_LIST in “_config”: [16, 19, 31, 40, 50, 60, 62, 91, 137]

checks.check_logicals_type(c, logicals)[source]¶

Check that the logical variables have correct type integer.

Parameters

c (ControlFile) – Contains all the parameters of CONTROL file and command line.
logicals (list of (str or int)) – Names of the switches that are used to control the flow of the program.

checks.check_mail(mail)[source]¶

Check the string of mail addresses, separate them and convert to list.

Parameters: mail (list of str or str) – Contains email addresses for notifications. It might also contain just the ecmwf user name which will trigger mailing to the associated email address for this user.
Returns: mail – Contains email addresses for notifications. It might also contain just the ecmwf user name which will trigger mailing to the associated email address for this user.
Return type: list of str

checks.check_maxstep(maxstep, steps)[source]¶

Convert maxstep into integer if it is already given. Otherwise, select maxstep by going through the steps list.

Parameters

maxstep (str) – The maximum forecast time step in hours from the forecast base time. This is the maximum step for non flux (accumulated) forecast data.
steps (str) – Specifies the forecast time step from forecast base time. Valid values are hours (HH) from forecast base time.

Returns

maxstep – The maximum forecast time step in hours from the forecast base time. This is the maximum step for non flux (accumulated) forecast data.

Return type

int

checks.check_number(number)[source]¶

Check for correct string format of ensemble member numbers.

Parameters: number (str) – List of ensemble member forecast runs.
Returns: number – String with list of ensemble member forecast runs. E.g. ‘01/02/03/04’
Return type: str

checks.check_pathes(idir, odir, fpdir, fedir)[source]¶

Check if output and flexpart pathes are set.

Parameters

idir (str) – Path to the temporary directory for MARS retrieval data.
odir (str) – Path to the final output directory where the FLEXPART input files will be stored.
fpdir (str) – Path to FLEXPART root directory.
fedir (str) – Path to flex_extract root directory.

Returns

odir (str) – Path to the final output directory where the FLEXPART input files will be stored.
fpdir (str) – Path to FLEXPART root directory.

checks.check_ppid(c, ppid)[source]¶

Sets the current PPID.

Parameters

c (ControlFile) – Contains all the parameters of CONTROL file and command line.
ppid (int or None) – Contains the ppid number provided by the command line parameter of is None otherwise.

checks.check_public(public, dataset, marsclass)[source]¶

Check wether the dataset parameter is set to a public data set.

Parameters

public (int) – Specifies if public data are to be retrieved or not.
dataset (str) – Specific name which identifies the public dataset.

checks.check_purefc(ftype)[source]¶

Check for a pure forecast mode.

Parameters: ftype (list of str) – List of field types.
Returns: True if pure forecasts are to be retrieved. False if there are analysis fields in between.
Return type: True or False

checks.check_queue(queue, gateway, destination, ecuid, ecgid)[source]¶

Check if the necessary ECMWF parameters are set if the queue is one of the QUEUES_LIST (in _config).

Parameters

queue (str) – Name of the queue if submitted to the ECMWF servers. Used to check if ecuid, ecgid, gateway and destination are set correctly and are not empty.
gateway (str) – The address of the gateway server.
destination (str) – The name of the destination of the gateway server for data transfer through ectrans. E.g. name@genericSftp
ecuid (str) – ECMWF user id.
ecgid (str) – ECMWF group id.

checks.check_request(request, marsfile)[source]¶

Check if there is an old MARS request file; if so, remove it.

Parameters

request (int) – Selects the mode of retrieval. 0: Retrieves the data from ECMWF. 1: Prints the mars requests to an output file. 2: Retrieves the data and prints the mars request.
marsfile (str) – Path to the mars request file.

checks.check_step(step)[source]¶

Checks on step format and convert into a list of steps.

If the steps were defined with “to” and “by” they are converted into a list of steps. If the steps were set in a string, it is converted into a list.

Parameters: step (list of str or str) – Specifies the forecast time step from forecast base time. Valid values are hours (HH) from forecast base time.
Returns: step – List of forecast steps in format e.g. [001, 002, …]
Return type: list of str

checks.check_time(ftime)[source]¶

Check if time variable is of type list. Otherwise convert to list.

Parameters: ftime (list of str or str) – The time in hours of the field.
Returns: ftime – The time in hours of the field.
Return type: list of str

checks.check_type(ftype, steps)[source]¶

Check if type variable is of type list and if analysis field has forecast step 0.

Parameters

ftype (list of str or str) – List of field types.
steps (str) – Specifies the forecast time step from forecast base time. Valid values are hours (HH) from forecast base time.

Returns

ftype – List of field types.

Return type

list of str

disaggregation ¶

Disaggregation of deaccumulated flux data from an ECMWF model FG field.

Initially the flux data to be concerned are:

large-scale precipitation
convective precipitation
surface sensible heat flux
surface solar radiation
u stress
v stress

Different versions of disaggregation is provided for rainfall data (darain, modified linear) and the surface fluxes and stress data (dapoly, cubic polynomial).

disaggregation.IA3(g)[source]¶

Interpolation with a non-negative geometric mean based algorithm.

The original grid is reconstructed by adding two sampling points in each data series interval. This subgrid is used to keep all information during the interpolation within the associated interval. Additionally, an advanced monotonicity filter is applied to improve the monotonicity properties of the series.

Note

This work is licensed under the Creative Commons Attribution 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.

Parameters: g (list of float) – Complete data series that will be interpolated having the dimension of the original raw series.
Returns: f – The interpolated data series with additional subgrid points. Its dimension is equal to the length of the input data series times three.
Return type: list of float

References

For more information see article: Hittmeir, S.; Philipp, A.; Seibert, P. (2017): A conservative interpolation scheme for extensive quantities with application to the Lagrangian particle dispersion model FLEXPART., Geoscientific Model Development

disaggregation.dapoly(alist)[source]¶

Cubic polynomial interpolation of deaccumulated fluxes.

Interpolation of deaccumulated fluxes of an ECMWF model FG field using a cubic polynomial solution which conserves the integrals of the fluxes within each timespan. Disaggregation is done for 4 accumluated timespans which generates a new, disaggregated value which is output at the central point of the 4 accumulation timespans. This new point is used for linear interpolation of the complete timeseries afterwards.

Parameters: alist (list of array of float) – List of 4 timespans as 2-dimensional, horizontal fields. E.g. [[array_t1], [array_t2], [array_t3], [array_t4]]
Returns: nfield – Interpolated flux at central point of accumulation timespan.
Return type: array of float

Note

March 2000P. JAMES: Original author
June 2003A. BECK: Adaptations
November 2015Leopold Haimberger (University of Vienna): Migration from Fortran to Python

disaggregation.darain(alist)[source]¶

Linear interpolation of deaccumulated fluxes.

Interpolation of deaccumulated fluxes of an ECMWF model FG rainfall field using a modified linear solution which conserves the integrals of the fluxes within each timespan. Disaggregation is done for 4 accumluated timespans which generates a new, disaggregated value which is output at the central point of the 4 accumulation timespans. This new point is used for linear interpolation of the complete timeseries afterwards.

Parameters: alist (list of array of float) – List of 4 timespans as 2-dimensional, horizontal fields. E.g. [[array_t1], [array_t2], [array_t3], [array_t4]]
Returns: nfield – Interpolated flux at central point of accumulation timespan.
Return type: array of float

Note

March 2000P. JAMES: Original author
June 2003A. BECK: Adaptations
November 2015Leopold Haimberger (University of Vienna): Migration from Fortran to Python