Opened 11 months ago

Last modified 3 months ago

#286 accepted Defect

Bug for local public ERA5 data / flex_extract v7.1.2

Reported by: anphi Owned by: anphi
Priority: blocker Milestone: flex_extract_v7.1.3
Component: flex_extract Version: flex_extract_v7.1
Keywords: Cc:

Description

When working with flex_extract v7.1.2 locally to retrieve ERA5 data from the CS3 website with CDSAPI the requests, a user had problems to use the retrieved files with FLEXPART v10.4.

An example result file and CONTROL file are attached.

Change History (6)

comment:1 Changed 11 months ago by anphi

  • Status changed from new to accepted

comment:2 Changed 11 months ago by anphi

After a quick inspection of the file content, it was clear that there is a bug in retrieving the surface fields. See below for list of fields contained in the example file:

1            ecmf         20180701     an           regular_ll   0            surface      0            sd           grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            msl          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            tcc          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            10u          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            10v          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            2t           grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            2d           grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            z            grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            lsm          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            sdor         grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            cvl          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            cvh          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            fsr          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            sdor         grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            cvl          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            cvh          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            fsr          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            sdor         grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            cvl          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            cvh          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            fsr          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            sdor         grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            cvl          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            cvh          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            fsr          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            sdor         grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            cvl          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            cvh          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            fsr          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            sdor         grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            cvl          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            cvh          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            fsr          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            sdor         grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            cvl          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            cvh          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            fsr          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            sdor         grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            cvl          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            cvh          grid_simple 
1            ecmf         20180701     an           regular_ll   0            surface      0            fsr          grid_simple 

The fields for topography information cvl,cvh,fsr and sdor should appear one time each and not multiple times. And precipitation data is missing, at least for this specific time step.


The problem occurs because I wanted to speed up the retrieval time for public users who want to retrieve ERA5 data from CS3 website. I, therefore, split the retrieval of surface and model level fields, since model level fields are still not available on CS3 and we need to retrieve them from the original MARS archive. The surface fields can be retrieved relatively fast from CS3, but they have different access keywords and storage structure.

There are actually two separate bugs. The first bug is in the conversion routine from MARS keywords to CS3 keywords. Surface fluxes are no longer accessed through the combination of time and step parameters but direct analysis times. The second bug is the accidental fixing of the dtime parameter to 3 in the surface field retrieval. Therefore all surface fields are retrieved every third hour, regardless of what was determined beforehand.

A bugfix will be provided as soon as possible.

comment:3 Changed 11 months ago by anphi

  • Summary changed from Problem with local public ERA5 data retrieval in flex_extract v7.1.2 to BUG: Problem with local public ERA5 data retrieval in flex_extract v7.1.2

comment:4 Changed 11 months ago by anphi

This bug is fixed in dev branch of our git repository now.

The exact changes can be viewed here.

Until the fix is merged into the master branch, I will keep the ticket open.

comment:5 Changed 3 months ago by anphi

  • Milestone set to flex_extract_v7.1.3

comment:6 Changed 3 months ago by pesei

  • Summary changed from BUG: Problem with local public ERA5 data retrieval in flex_extract v7.1.2 to Bug for local public ERA5 data / flex_extract v7.1.2
Note: See TracTickets for help on using tickets.
hosted by ZAMG