wiki:FpCtbtoWo4PreprocessingVerification

VERIFICATION

The verification is done with the fp files generated with the useAvail option for simplicity and to ensure that they should match with the control data.

The test case data can be found in: http://borealscicomp.com/CTBTO_FLEXPART/LargeFlexpartTestCases/

ALL THE TESTS FOR THE CASES BELOW PASS SUCCESSFULLY WITH THE FP FILES

List of tests:

  • ECMWF global 1.0 degree bwd, 2 hours
    • Generation of the fp files:
      • Wall time to process: EN13062503: 22.2360001 seconds
      • Wall time to dump to: EN13062503_bwd.fp: 5.23000002 seconds
      • Wall time to process: EN13062500: 22.1380005 seconds
      • Wall time to dump to: EN13062500_bwd.fp: 4.90600014 seconds
      • Size of each file: 473 MB
    • Running FLEXPART with the fp files:
      • Wall time to process: fpfiles_fwd_useavailable/EN13062503_bwd.fp : 0.200000003 seconds
      • Wall time to process: fpfiles_fwd_useavailable/EN13062500_bwd.fp : 0.230000004 seconds
    • diff shows zero differences with control
    • testing environment: Test performed. Error = 0.0E+00 Test passed
  • ECMWF global 1.0 degree fwd, 2 hours
    • Generation of the fp files:
      • Wall time to process: EN13062500: 22.1739998 seconds
      • Wall time to dump to: EN13062500_fwd.fp: 0.537000000 seconds
      • Wall time to process: EN13062503: 22.0270004 seconds
      • Wall time to dump to: EN13062503_fwd.fp: 0.532999992 seconds
      • Size of each file: 473 MB
    • Running FLEXPART with the fp files:
      • Wall time to process: met_data/EN13062500_fwd.fp : 0.296000004 seconds
      • Wall time to process: met_data/EN13062503_fwd.fp : 0.273999989 seconds
    • diff shows zero differences with control
    • testing environment: testing environment: Test performed. Error = 0.0E+00 Test passed
  • NCEP global 1.0 degree bwd, 2 hours
    • Generation of the fp files:
      • Wall time to process: GF15021603: 7.11800003 seconds
      • Wall time to dump to: GF15021603_bwd.fp: 0.526000023 seconds
      • Wall time to process: GF15021600: 6.53900003 seconds
      • Wall time to dump to: GF15021600_bwd.fp: 0.554000020 seconds
      • Size of each file: 473 MB
    • diff shows zero differences with control
    • testing environment: Test performed. Error = 0.0E+00 Test passed
  • NCEP global 1.0 degree fwd, 2 hours
    • Generation of the fp files:
      • Wall time to process: GF15021600: 6.86299992 seconds
      • Wall time to dump to: GF15021600_fwd.fp: 0.531000018 seconds
      • Wall time to process: GF15021603: 6.28499985 seconds
      • Wall time to dump to: GF15021603_fwd.fp: 0.542999983 seconds
      • Size of each file: 473 MB
    • diff shows zero differences with control
    • testing environment: Test performed. Error = 0.0E+00 Test passed
  • ECMWF 6 hours 0.5 bwd with fp files
    • Generation of the fp files:
      • Wall time to process: EN15021606: 165.356995 seconds
      • Wall time to dump to: EN15021606_bwd.fp: 3.00000000 seconds
      • Wall time to process: EN15021603: 158.772995 seconds
      • Wall time to dump to: EN15021603_bwd.fp: 2.62299991 seconds
      • Wall time to process: EN15021600: 147.024002 seconds
      • Wall time to dump to: EN15021600_bwd.fp: 2.89299989 seconds
      • Size of each file: 1.9 GB
    • Timings for run FLEXPART with met files
      • Wall time to process: EN15021606: 145.141998 seconds (not used in the simulation- unneded read)
      • Wall time to process: EN15021603: 151.067001 seconds
      • Wall time to process: EN15021600: 139.856995 seconds
    • Timings for run FLEXPART with fp files
      • Wall time to process: met_data/EN15021606_bwd.fp: 0.963999987 seconds
      • Wall time to process: met_data/EN15021603_bwd.fp: 16.5459995 seconds
      • Wall time to process: met_data/EN15021600_bwd.fp: 10.2419996 seconds
    • diff shows zero differences with control
    • testing environment: Test performed. Error = 0.0E+00 Test passed
  • ECMWF 9.5 days 1 deg bwd
    • Generation of the fp files:
      • Wall time to process all 78 files: 6002.89898 seconds
      • Wall time to dump to all 78 files : 43.233 seconds
      • Size of all files: 36 GB (each file 473 MB)
    • Timings for run with met files: 6255 seconds
    • Timings for run with fp files: 290
    • testing environment: Test performed. Error = 0.0E+00 Test passed
    • This particular run has been investigated as well on a time to time basis showing no errors (and thus, no difference to the control files) for any of the simulation times:
 Max error for full volume, full timeseries:  0.0E+00
 RMSE for full volume, full timeseries:       0.0E+00


 Errors for full volume, timestamp 20150519110000:
 RMSE:       0.0E+00
 Max Error:  0.0E+00

 Errors for full volume, timestamp 20150519100000:
 RMSE:       0.0E+00
 Max Error:  0.0E+00

 Errors for full volume, timestamp 20150519090000:
 RMSE:       0.0E+00
 Max Error:  0.0E+00

 ...

 Errors for full volume, timestamp 20150510010000:
 RMSE:       0.0E+00
 Max Error:  0.0E+00

 Errors for full volume, timestamp 20150510000000:
 RMSE:       0.0E+00
 Max Error:  0.0E+00

Comparison runs with fp files generated one at a time and with useAvail

There are different ways of generating the fp files. The one that would be most efficient and less time consuming, generation of one fp file at a time with new incoming data, implies a variable vertical grid coordinate system in FLEXPART. This may lead to some inconsistencies or differences that should be evaluated (benefits or disadvantages) in depth for a range of test-cases. Here, we investigate visually and numerically the differences for the 9 day bwd test run.

As expected, the tests performed with the testing environment using the control data fail because of the time-depending internal vertical coordinate syste, The total global error is 4.9E-01. The RMSE, although small, increases with time as seen in the figure below, whereas the overall error alternates positive and negative.

This relative information needs to be understood together with the concentration plots. The concentration (or residence time) plots show minimum differences due to the varying vertical coordinates. An example is shown below, with concentration averages for the two output vertical layers at the times of maximum "Max error" (both positive and negative) and at the end of the simulation, where the RMSE is as well maximum.

Differences exist but, for this specific case, seem to appear only at the edges of the plume. Further investigation is needed to understand whether such differences may be a problematic area.

Last modified 3 years ago Last modified on Nov 6, 2015, 11:44:13 AM

Attachments (2)

Download all attachments as: .zip

hosted by ZAMG