Opened 4 months ago

Last modified 4 months ago

#316 new Defect

Post-processing issue: Negative deposition values

Reported by: jshafer Owned by:
Priority: major Milestone: FLEXPART 10.4.1
Component: FP physics/numerics Version: FLEXPART 10.4
Keywords: Cc:

Description

I am experiencing an issue with my FLEXPART outputs (using the NetCDF format) that I noticed when trying to analyze it with Python.
I am trying to run two species with two sepearate releases in forward mode, for 1 month but single species are causing the same issue.

Some of the wet and dry depositions are showing up as negative values, very small negative values but negative nonetheless.
I have tried changing the settings in the different key user files like particle number released, grid size, species files etc but nothing changes the outcome. The only time I am not getting negative values is when the settling velocity is disable. I looked at the data in the netcdf files in ncview as well which shows the same outcome so it seems to be flexpart related.
I would appreciate any insight to what could be causing this.

Thanks!
Julia

Attachments (10)

SPECIES_2_file.txt (2.6 KB) - added by jshafer 4 months ago.
Attached are the two Species files. Also, below are the minimum values for the wet deposition for species 1 and 2 and the dry deposition for species 1 and 2.
minimum values.txt (357 bytes) - added by jshafer 4 months ago.
Attached are the minimum values after a 1 month run with two species and 5 million particles released.
Negative value map.JPG (70.2 KB) - added by jshafer 4 months ago.
Attached is an example of what the plotting looks like, when plotting both species summed wet and dry deposition
grid_conc_20200120230000_002 (18.9 KB) - added by jshafer 4 months ago.
grid_conc_20200120230000_001 (40.3 KB) - added by jshafer 4 months ago.
header (33.0 KB) - added by jshafer 4 months ago.
COMMAND file FLEXPART.txt (2.7 KB) - added by jshafer 4 months ago.
RELEASE_2_sub.txt (2.4 KB) - added by jshafer 4 months ago.
dates (6.9 KB) - added by jshafer 4 months ago.
Input files.txt (265 bytes) - added by jshafer 4 months ago.

Download all attachments as: .zip

Change History (36)

comment:1 Changed 4 months ago by pesei

  • Component changed from FP post-processing to FP physics/numerics
  • Type changed from Support to Defect

What you describe looks more like a potential bug.

Could you please post the relevant SPECIES files, and also give us the minimum value of the deposition field that you found?

If you have a plot that shows the geographical distribution of the (negative) values it could also be helpful (You can attach it to the ticket).

Changed 4 months ago by jshafer

Attached are the two Species files. Also, below are the minimum values for the wet deposition for species 1 and 2 and the dry deposition for species 1 and 2.

Changed 4 months ago by jshafer

Attached are the minimum values after a 1 month run with two species and 5 million particles released.

Changed 4 months ago by jshafer

Attached is an example of what the plotting looks like, when plotting both species summed wet and dry deposition

comment:2 Changed 4 months ago by jshafer

Thank you so much for your response!

Let me know if attached files make sense and if there is anything else that would be helpful for me to provide.

Thanks,
Julia

comment:3 Changed 4 months ago by pesei

Hello, how far into the simulation (after release start) is the negative value map?

comment:4 Changed 4 months ago by jshafer

It depends slightly on the settings but around 200 hours in.

Thank you,
Julia

comment:5 Changed 4 months ago by ignacio

Hi, Do you see the same negative values in the binary formatted output?
Ignacio

comment:6 Changed 4 months ago by jshafer

Hi,

No I do not seem to be. The negative values vary with time and also with coordinates. Most negative numbers however seem to be located at the outskirts of the plot.

Thank you,
Julia

comment:7 Changed 4 months ago by pesei

Do the negative values occur in regions where conc should be =0 or where they should be >0 ?
And is it verified that the problem is ONLY with netcdf output?

comment:8 Changed 4 months ago by jshafer

It seems like as if it is occuring where I would expect the concentration to be zero or atleast very close to zero. e.g its not occuring close to the point source.
I have only checked the netcdf output, I can try running the model with the default output format.

comment:9 Changed 4 months ago by pesei

Yes, please do so, it will help to delineate the problem.

Changed 4 months ago by jshafer

Changed 4 months ago by jshafer

comment:10 Changed 4 months ago by jshafer

I have run the model with the sparse matrix output the only problem is that I am not familiar with analysing this type of format. I have attached two of the final outputfiles incase you know how to quickly check.

I totally understand if this is to much to ask and I will try to figure out how to analyze it on my own.

Thank you,
Julia

comment:11 Changed 4 months ago by pesei

Thank you. The header file would be need as well for processing.

Changed 4 months ago by jshafer

comment:12 Changed 4 months ago by jshafer

It has now been attached.
Thank you so much for helping me with this!

comment:13 Changed 4 months ago by anphi

Hi,
I can plot the binary files with the tool I built, but I would need the COMMAND and the RELEASES file as well. Could you upload them, please?
Thanks!
Anne

Changed 4 months ago by jshafer

Changed 4 months ago by jshafer

comment:14 Changed 4 months ago by jshafer

Thank you Anne.
I attached the COMMAND and RELEASES files that correlates to the two binary files I attached above.
Both COMMAND file and RELEASES file should have the end date of 20200120 instead of 20200114 (I uploaded them a bit to quickly)

Thank you!
Julia

Last edited 4 months ago by jshafer (previous) (diff)

comment:15 Changed 4 months ago by ignacio

Hi Julia, Could you upload the "dates" output file as well? Ignacio
Also the input files you changed in the options/ directory like OUTGRID could also be useful.

Last edited 4 months ago by ignacio (previous) (diff)

Changed 4 months ago by jshafer

Changed 4 months ago by jshafer

comment:16 Changed 4 months ago by jshafer

I added both AGECLASS and OUTGRID to the file name "input files".
Thanks!

comment:17 Changed 4 months ago by ignacio

Hi Julia,

I did open your binary files and didn't find any negative values. The issue could be related to netcdf compression, but in any case is a numerical issue and does not seem to come from the physical parametrisation.

Would you be able to upload the .nc file with the negative values as well?

Thanks

Ignacio

comment:18 Changed 4 months ago by ignacio

Hi again,

What kind of meteorological input where you using?

Ignacio

comment:19 Changed 4 months ago by jshafer

Hi!

I did not download the data myself but its ECMWF data with 3 hour forecast and with file name "EN" If you need more details I will find out!

Julia

Last edited 4 months ago by jshafer (previous) (diff)

comment:20 Changed 4 months ago by jshafer

Hi,

Any updates regarding the binary file or is the issue soley with NetCDF? I am wondering if I should try reinstalling FLEXPART to solve the bug issue?

Thanks again for all the support!

Julia

comment:21 Changed 4 months ago by pesei

As Ignacio said, he did not find negative values in the binary output. It is strange that you found these negative values - they should not be there and my understanding is that netCDF uses a lossless compression algorithm, thus one would not expect that. Maybe you want to make available one netCDF output file as well for independent check?

As for your work, if you want to be on the safe side, you could use the binary output (there are various codes available for reading it). Otherwise, if the negative values are insignificant, you may choose to ignore them (setting to zero). Keep an eye on whether there are also spurious positive values (very small positive values where zero would be expected).

Last edited 4 months ago by pesei (previous) (diff)

comment:22 Changed 4 months ago by jshafer

Hi,

Thank you so much for the reply.

I totally missed Ignacio's reply, I am very sorry about that!

Is there a way to upload a larger file? The smallest nc file I have contaning negative values (occurs after 200 h ish) is 500 MB.

Thanks again for everything!

Julia

Last edited 4 months ago by jshafer (previous) (diff)

comment:23 Changed 4 months ago by pesei

I'll provide you with a special upload link for large files, thank you!

comment:25 Changed 4 months ago by jshafer

Great!

Done!

Thanks,
Julia

Last edited 4 months ago by jshafer (previous) (diff)

comment:26 Changed 4 months ago by jshafer

Hi again!

I was wondering if you had a chance to look at the netcdf file I uploaded.
Also, I was thinking that to circumvent the issue I could run the model and produce the binary sparse matrix output and then maybe convert it after the fact using maybe Reflexible? Do you think this would be a promising work around?

Thank you!
Julia

Note: See TracTickets for help on using tickets.
hosted by ZAMG