Opened 14 months ago

Last modified 3 months ago

#297 accepted Defect

gridcheck_ecmwf.f90: fix of small bug and improvements

Reported by: anphi Owned by: anphi
Priority: critical Milestone: FLEXPART 10
Component: FP coding/compilation Version: FLEXPART 10.4
Keywords: GRIB2 GRIB1 Cc: pesei

Description (last modified by pesei)

  1. check on field dimensions uses undefined variable nx
  1. If ECMWF grib files contain only GRIB2 formatted fields, FLEXPART stops with an error message due to missing parameters from GRIB1 messages.

***ERROR: input file needs to contain GRiB1 formatted messages

Update 2022-02-09: In v10.4 the problem is not that a GRIB1 field is required, but that this messages is emitted if no T2 field is found (which is used to extract certain parameters). Rewriting the error message is probably good enough as a solution.

Change History (16)

comment:1 Changed 10 months ago by anphi

  • Status changed from new to accepted

comment:2 Changed 4 months ago by cluekenwinkels

Hello, I recently started using Flexpart and recieved the same ERROR in my text case. I followed the same steps as described in ticket 303. Is there a solution yet? Could you also give information about the flex_extact version you are using?

comment:3 Changed 4 months ago by pesei

We'll try to offer at least a quick fix soon.

comment:4 Changed 4 months ago by pesei

Closely related: #199 gridcheck: checking array dimensions needs to be improved

comment:5 Changed 4 months ago by anphi

Dear cluekenwinkels,

without any more detail to your specific case I am unable to reproduce the error.
Just from checking the code, I have the suspicion that your EN files are missing the 2m temperature field. I am not going into much detail yet. Could you please provide me with more information?
I take it, you are using Flexpart 10.4? What flex_extract version do you use and do you still have the log file from the extraction of the EN files you are currently using? Please provide it to me (attaching it to this ticket would be good). And could you please send me the grib_ls output from at least one of your EN files please? I need to know what is actually in the files. Maybe you can attach one EN file to this ticket. That would be enough.

Thank you.
Anne

comment:6 Changed 4 months ago by pesei

If your EN file is big, you can upload it here: https://bokubox.boku.ac.at/#f6dfa649844127b8da86fa8856618ce4 (max size 15 GB)
(NB: only I will be able to download)

comment:7 Changed 4 months ago by cluekenwinkels

I followed the documentation for flex_extract 7.1.2, so this should be my version: https://www.flexpart.eu/flex_extract/installation.html#ref-download
The Flexpart version is 10.4.
The error only occurs using ERA5 data as I checked runs with ERA-Interim data which worked fine. The only flex_extract output I have left are the files with names EA########, all other files were deleted by flex_extract. I uploaded one of these files and the error and output files of the corresponding flex_extract run. Where can I find the EN files?
(I tried it with both settings: FORMAT=GRIB1 and FORMAT=GRIB2, and the final error remains the same.)

Last edited 4 months ago by cluekenwinkels (previous) (diff)

comment:8 Changed 4 months ago by pesei

Thank you for the files. the EA-File is what we need (for many years we called the FLEXPART input files EN*, more recentely identiers specific to the data set are used, such as EA for ERA5, just old habit to say EN).

There are two uploads without file name - is this a stopped upload? They are not downloadable, so I deleted them.

comment:9 Changed 4 months ago by cluekenwinkels

Ok perfect. There should be 3 files: the EA*, one .o* and one .e* file (the output and error file respectively). I did not try to upload any other files.

comment:10 Changed 3 months ago by anphi

I need to tell you that the grib file looks completely wrong in the surface section. There are a couple of fields which are too much and others are completely missing. I do not remember seeing this specific combination of surface fields before and I cannot reproduce the full problem right now.

So, generally, there is a problem with flex_extract or your setup but not with FLEXPART.

Please upload the CONTROL file you used for the extraction and maybe you can also upload the run_local.sh file.

And I need to make sure that we are using the same version of flex_extract.
Did you use the tar file from the University of Vienna Phaidra website or did you clone the master branch of the git repository? And can you please confirm that you use 7.1.2 and not 7.1.1?

comment:11 Changed 3 months ago by cluekenwinkels

I used this download link: git clone --single-branch --branch master https://www.flexpart.eu/gitmob/flex_extract
I could not find the version number within the files, but it should yield the most recent one, right?
I cannot upload data anymore to the bokubox link, so here are the first lines of my CONTROL file:

START_DATE 20180809
DTIME 1
TYPE AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN
TIME 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23
STEP 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
ACCTYPE FC
ACCTIME 06/18
ACCMAXSTEP 12
CLASS EA
STREAM OPER
GRID 0.28125  
LEFT -24.
LOWER 9.875
UPPER 74.
RIGHT 60.375
LEVELIST 1/to/137
RESOL 799
ETA 1
FORMAT GRIB1
PREFIX EA
CWC 1
RRINT 1
ECTRANS 1

This should be the first lines of the run_local.sh I used (I changed it since then but the parameters should be right):

QUEUE=' '
START_DATE='20170102'
DATE_CHUNK=None
JOB_CHUNK=None
BASETIME=None
STEP=None
LEVELIST=None
AREA=None
INPUTDIR='./Workspace/path_to_output'
OUTPUTDIR=None
PP_ID=None
JOB_TEMPLATE=' '
CONTROLFILE='/work/bb1170/RUN/b381737/software/flexpart_v10.4_3d7eebf/preprocess/flex_extract/Run/Control/CONTROL_EA5_grib1'
DEBUG=0
REQUEST=0
PUBLIC=1
Last edited 3 months ago by pesei (previous) (diff)

comment:12 Changed 3 months ago by anphi

Thank you! I need some time to figure out what the problem is. I am on it. I'll let you know as soon as it's solved and how you should proceed.

And by the way, you used RRINT=1. This is not yet supported in Flexpart10.4. I currently have a beta version under test so you need to either use RRINT=0 or wait for the release of my beta version.

Best,
Anne

comment:13 Changed 3 months ago by pesei

  • Description modified (diff)

comment:14 Changed 3 months ago by anphi

I solved the issue. The new version is currently in the dev branch. With a git pull followed by a git checkout dev. You should get the dev branch and you should be able to restart your extraction. The file MarsRetrieval.py in the Source/Python/Classes directory should have a new timestamp.

Best
Anne

comment:15 Changed 3 months ago by cluekenwinkels

Hi, thanks already! I will try it out!

Edit: It the simulations work now. Thanks alot!

(there are still a few warnings that occur however it seems that the correct output is produced)

 parId        3063 isec1(6)         143
 ***WARNING: undefined GRiB2 message found!           0           1         254           1
 parId           0 isec1(6)          -1
 ***WARNING: undefined GRiB2 message found!         192         128          27           1
 parId          27 isec1(6)          -1
 ***WARNING: undefined GRiB2 message found!         192         128          28           1
 parId          28 isec1(6)          -1
 ***WARNING: undefined GRiB2 message found!         192         128         244           1
 parId         244 isec1(6)          -1
Last edited 3 months ago by cluekenwinkels (previous) (diff)

comment:16 Changed 3 months ago by anphi

Thanks a lot for this feedback. I very much appreciate it.

A couple of these warnings are expected and we are aware that we need to adapt the output messages.
However, two warnings come unexpected to me.
First, the parameter with ID 254 shouldn't be there and I'd like to investigate this. Second, it seems that the convective precipitation is not recognized but Flexpart. I have a feeling this is due to GRIB1 vs GRIB2 differences, but I will check this out and report back later this week.


Note: See TracTickets for help on using tickets.
hosted by ZAMG