Changeset 8209738 in flex_extract.git for Source/Python/Mods/get_mars_data.py
- Timestamp:
- May 7, 2020, 3:32:03 PM (4 years ago)
- Branches:
- master, ctbto, dev
- Children:
- b936fd3
- Parents:
- 05e9362
- File:
-
- 1 edited
Legend:
- Unmodified
- Added
- Removed
-
Source/Python/Mods/get_mars_data.py
r0f89116 r8209738 10 10 # November 2015 - Leopold Haimberger (University of Vienna): 11 11 # - moved the getEIdata program into a function "get_mars_data" 12 # - moved the AgurmentParser into a sep erate function13 # - ada tpted the function for theuse in flex_extract14 # - renamed file to get_mars_data12 # - moved the AgurmentParser into a separate function 13 # - adapted the function for use in flex_extract 14 # - renamed source file to get_mars_data 15 15 # 16 16 # February 2018 - Anne Philipp (University of Vienna): … … 22 22 # online documentation) 23 23 # - use of UIFiles class for file selection and deletion 24 # - sep erated get_mars_data function into several smaller pieces:24 # - separated get_mars_data function into several smaller pieces: 25 25 # write_reqheader, mk_server, mk_dates, remove_old, do_retrievment 26 26 # … … 36 36 # Creative Commons, PO Box 1866, Mountain View, CA 94042, USA. 37 37 #******************************************************************************* 38 '''This script extracts MARS data from ECMWF servers.38 '''This script extracts MARS data from ECMWF. 39 39 40 40 At first, the necessary parameters from command line and CONTROL files are … … 44 44 functions: 45 45 46 * main - the main function of the script47 * get_mars_data - overall control of ECMWF data retrievment46 * main - the main function of the script 47 * get_mars_data - overall control of ECMWF data retrievment 48 48 * write_reqheader - writes the header into the mars_request file 49 * mk_server - creates the server connection to ECMWF servers50 * mk_dates - defines the start and end date51 * remove_old - deletes old retrieved grib files52 * do_retriev ement- creates individual retrievals53 54 Type :get_mars_data.py --help49 * mk_server - creates the server connection to ECMWF servers 50 * mk_dates - defines the start and end date 51 * remove_old - deletes old retrieved grib files 52 * do_retrieval - creates individual retrievals 53 54 Type get_mars_data.py --help 55 55 to get information about command line parameters. 56 56 Read the documentation for usage instructions. … … 66 66 from datetime import datetime, timedelta 67 67 68 # software 69 # add path to local main python path for flex_extract to get full access68 # software-specific classes and modules from flex_extract 69 # add path to local main Python path for flex_extract to get full access 70 70 sys.path.append(os.path.dirname(os.path.abspath( 71 71 inspect.getfile(inspect.currentframe()))) + '/../') … … 94 94 # ------------------------------------------------------------------------------ 95 95 def main(): 96 '''Controls the program to get data out of mars.97 98 This is done if it iscalled directly from command line.99 Then it also takes program call arguments and control fileinput.96 '''Controls the program to retrieve data from MARS. 97 98 This is done if called directly from command line. 99 Then, arguments and control file are taken as input. 100 100 101 101 Parameters … … 114 114 115 115 def get_mars_data(c): 116 '''Retrieves the EC data needed for a FLEXPART simulation.117 118 Start and end dates for retrieval period isset. Retrievals119 are divided into s maller periods if necessary anddatechunk parameter116 '''Retrieves the ECMWF data required for a FLEXPART simulation. 117 118 Start and end dates for retrieval period are set. Retrievals 119 are divided into shorter periods if necessary and if datechunk parameter 120 120 is set. 121 121 … … 137 137 138 138 if c.request == 0: 139 print("Retrieving EC data!")139 print("Retrieving ECMWF data!") 140 140 else: 141 141 if c.request == 1: 142 print("Printing marsrequests!")142 print("Printing MARS requests!") 143 143 elif c.request == 2: 144 print("Retrieving EC data and printing marsrequest!")144 print("Retrieving ECMWF data and printing MARS request!") 145 145 write_reqheader(os.path.join(c.inputdir, _config.FILE_MARS_REQUESTS)) 146 146 … … 165 165 166 166 def write_reqheader(marsfile): 167 '''Writes header with column names into marsrequest file.167 '''Writes header with column names into MARS request file. 168 168 169 169 Parameters 170 170 ---------- 171 171 marsfile : str 172 Path to the marsrequest file.172 Path to the MARS request file. 173 173 174 174 Return … … 188 188 189 189 def mk_server(c): 190 '''Creates a server connection with available python API. 191 192 Which API is used depends on availability and the dataset to be retrieved. 193 The CDS API is used for ERA5 dataset no matter if the user is a member or 194 a public user. ECMWF WebAPI is used for all other available datasets. 190 '''Creates a server connection with available Python API. 191 192 The API selected depends on availability and the data set to be retrieved. 193 The CDS API is used for ERA5 data, no matter whether the user is a 194 member-state or a public user. 195 ECMWF WebAPI is used for all other available datasets. 195 196 196 197 Parameters … … 247 248 248 249 def mk_dates(c, fluxes): 249 '''Prepares start and end date depending on flux or non fluxdata.250 251 If forecast for maximum one day (upto 24h) are to be retrieved, then250 '''Prepares start and end date depending on flux or non-flux type of data. 251 252 If forecasts for a maximum of one day (24 h) are to be retrieved, then 252 253 collect accumulation data (flux data) with additional days in the 253 beginning and at the end ( used for complete disaggregation of254 beginning and at the end (needed for complete disaggregation of 254 255 original period) 255 256 256 If forecast data longer than 24h are to be retrieved, then257 If forecast data for more than +24 h are to be retrieved, then 257 258 collect accumulation data (flux data) with the exact start and end date 258 259 (disaggregation will be done for the exact time period with … … 314 315 ---------- 315 316 pattern : str 316 The sub 317 The substring pattern which identifies the files to be deleted. 317 318 318 319 inputdir : str, optional 319 Path to the directory where the retrieved data isstored.320 321 Return 322 ------ 323 324 ''' 325 print('... removing old content of' + inputdir)320 Path to the directory where the retrieved data are stored. 321 322 Return 323 ------ 324 325 ''' 326 print('... removing old files in ' + inputdir) 326 327 327 328 tobecleaned = UioFiles(inputdir, pattern) … … 332 333 333 334 def do_retrievement(c, server, start, end, delta_t, fluxes=False): 334 '''Divides the complete retrieval period insmaller chunks and335 '''Divides the total retrieval period into smaller chunks and 335 336 retrieves the data from MARS. 336 337 … … 351 352 352 353 delta_t : datetime 353 Delta_t + 1 is the maximal time period of a single 354 retrieval. 354 Delta_t + 1 is the maximum time period of a single retrieval. 355 355 356 356 fluxes : boolean, optional … … 365 365 366 366 # since actual day also counts as one day, 367 # we only need to add datechunk - 1 days to retrieval 368 # for a period 367 # we only need to add datechunk - 1 days to retrieval for a period 369 368 delta_t_m1 = delta_t - timedelta(days=1) 370 369
Note: See TracChangeset
for help on using the changeset viewer.