source: flex_extract.git/source/python/mods/prepare_flexpart.py @ f20af73

ctbtodev
Last change on this file since f20af73 was f20af73, checked in by Anne Philipp <anne.philipp@…>, 5 years ago

added CDS API support for ERA5 instead of ECMWFAPI / refactored setup of CONTROL parameter because for local version it wanted to use not installed ECMWF_ENV file.

  • Property mode set to 100755
File size: 6.1 KB
Line 
1#!/usr/bin/env python
2# -*- coding: utf-8 -*-
3#*******************************************************************************
4# @Author: Anne Fouilloux (University of Oslo)
5#
6# @Date: October 2014
7#
8# @Change History:
9#
10#    November 2015 - Leopold Haimberger (University of Vienna):
11#        - using the WebAPI also for general MARS retrievals
12#        - job submission on ecgate and cca
13#        - job templates suitable for twice daily operational dissemination
14#        - dividing retrievals of longer periods into digestable chunks
15#        - retrieve also longer term forecasts, not only analyses and
16#          short term forecast data
17#        - conversion into GRIB2
18#        - conversion into .fp format for faster execution of FLEXPART
19#
20#    February 2018 - Anne Philipp (University of Vienna):
21#        - applied PEP8 style guide
22#        - added documentation
23#        - minor changes in programming style for consistence
24#        - BUGFIX: removed call of clean_up-Function after call of
25#               prepareFlexpart in main since it is already called in
26#               prepareFlexpart at the end!
27#        - created function main and moved the two function calls for
28#          arguments and prepare_flexpart into it
29#
30# @License:
31#    (C) Copyright 2014-2019.
32#    Anne Philipp, Leopold Haimberger
33#
34#    This work is licensed under the Creative Commons Attribution 4.0
35#    International License. To view a copy of this license, visit
36#    http://creativecommons.org/licenses/by/4.0/ or send a letter to
37#    Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.
38#*******************************************************************************
39'''This script prepares the final version of the grib files which are
40then used by FLEXPART.
41
42It converts the bunch of grib files extracted via get_mars_data before,
43by doing the necessary conversion to get consistent grids or the
44disaggregation of flux data. Finally, the data fields are combined
45in files per available hour with the naming convention xxYYMMDDHH,
46where xx should be 2 arbitrary letters (mostly xx is chosen to be "EN").
47
48This file can also be imported as a module which then contains the following
49functions:
50
51    * main
52    * prepare_flexpart
53
54Type: prepare_flexpart.py --help
55to get information about command line parameters.
56Read the documentation for usage instructions.
57'''
58
59# ------------------------------------------------------------------------------
60# MODULES
61# ------------------------------------------------------------------------------
62import datetime
63import os
64import inspect
65import sys
66import socket
67
68# software specific classes and modules from flex_extract
69# add path to local main python path for flex_extract to get full access
70sys.path.append(os.path.dirname(os.path.abspath(
71    inspect.getfile(inspect.currentframe()))) + '/../')
72import _config
73from .checks import check_ppid
74from classes.UioFiles import UioFiles
75from classes.ControlFile import ControlFile
76from .tools import (setup_controldata, clean_up, get_cmdline_args,
77                   read_ecenv, make_dir)
78from classes.EcFlexpart import EcFlexpart
79
80# ------------------------------------------------------------------------------
81# FUNCTION
82# ------------------------------------------------------------------------------
83def main():
84    '''Controls the program to prepare flexpart input files from mars data.
85
86    This is done if it is called directly from command line.
87    Then it also takes program call arguments and control file input.
88
89    Parameters
90    ----------
91
92    Return
93    ------
94
95    '''
96
97    c, ppid, _, _ = setup_controldata()
98    prepare_flexpart(ppid, c)
99    normal_exit('Preparing FLEXPART output files: Done!')
100
101    return
102
103def prepare_flexpart(ppid, c):
104    '''Converts the mars data into flexpart ready input files.
105
106    Specific data fields are converted to a different grid and the flux
107    data are going to be disaggregated. The data fields are collected by
108    hour and stored in a file with a specific FLEXPART relevant naming
109    convention.
110
111    Parameters
112    ----------
113    ppid : int
114        Contains the ppid number of the current ECMWF job. It will be None if
115        the method was called within this module.
116
117    c : ControlFile
118        Contains all the parameters of CONTROL file and
119        command line.
120
121    Return
122    ------
123
124    '''
125    check_ppid(c, ppid)
126
127    # create the start and end date
128    start = datetime.date(year=int(c.start_date[:4]),
129                          month=int(c.start_date[4:6]),
130                          day=int(c.start_date[6:]))
131
132    end = datetime.date(year=int(c.end_date[:4]),
133                        month=int(c.end_date[4:6]),
134                        day=int(c.end_date[6:]))
135
136    # if basetime is 00
137    # assign starting date minus 1 day
138    # since we need the 12 hours upfront
139    # (the day before from 12 UTC to current day 00 UTC)
140    if c.basetime == 0:
141        start = start - datetime.timedelta(days=1)
142
143    print('Prepare ' + start.strftime("%Y%m%d") +
144           "/to/" + end.strftime("%Y%m%d"))
145
146    # create output dir if necessary
147    if not os.path.exists(c.outputdir):
148        make_dir(c.outputdir)
149
150    # get all files with flux data to be deaccumulated
151    inputfiles = UioFiles(c.inputdir, '*OG_acc_SL*.' + str(c.ppid) + '.*')
152
153    # deaccumulate the flux data
154    flexpart = EcFlexpart(c, fluxes=True)
155    flexpart.write_namelist(c)
156    flexpart.deacc_fluxes(inputfiles, c)
157
158    # get a list of all other files
159    inputfiles = UioFiles(c.inputdir, '????__??.*' + str(c.ppid) + '.*')
160
161    # produce FLEXPART-ready GRIB files and process them -
162    # copy/transfer/interpolate them or make them GRIB2
163    flexpart = EcFlexpart(c, fluxes=False)
164    flexpart.create(inputfiles, c)
165    if c.stream.lower() == 'elda':
166        flexpart.calc_extra_elda(c.inputdir, c.prefix)
167    flexpart.process_output(c)
168
169    # make use of a possible conversion to a
170    # specific flexpart binary format
171    if c.grib2flexpart:
172        flexpart.prepare_fp_files(c)
173
174    # check if in debugging mode, then store all files
175    # otherwise delete temporary files
176    if c.debug:
177        print('\nTemporary files left intact')
178    else:
179        clean_up(c)
180
181    return
182
183if __name__ == "__main__":
184    main()
Note: See TracBrowser for help on using the repository browser.
hosted by ZAMG