Changes between Initial Version and Version 1 of FpCtbtoWo4PreprocessingVerification


Ignore:
Timestamp:
Nov 6, 2015, 8:42:18 AM (8 years ago)
Author:
dearn
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • FpCtbtoWo4PreprocessingVerification

    v1 v1  
     1
     2== VERIFICATION ==
     3
     4The verification is done with the fp files generated with the useAvail option for simplicity and to ensure that they should match with the control data.
     5
     6The test case data can be found in:  [http://borealscicomp.com/CTBTO_FLEXPART/LargeFlexpartTestCases/]
     7
     8'''
     9ALL THE TESTS FOR THE CASES BELOW PASS SUCCESSFULLY WITH THE FP FILES'''
     10
     11List of tests:
     12* __ECMWF global 1.0 degree bwd, 2 hours__
     13    * Generation of the fp files:
     14        * Wall time to process: EN13062503:    22.2360001      seconds
     15        * Wall time to dump to: EN13062503_bwd.fp:    5.23000002      seconds
     16        * Wall time to process: EN13062500:    22.1380005      seconds
     17        * Wall time to dump to: EN13062500_bwd.fp:    4.90600014      seconds
     18        * Size of each file: 473 MB
     19    * Running FLEXPART with the fp files:
     20        * Wall time to process: fpfiles_fwd_useavailable/EN13062503_bwd.fp  :   0.200000003      seconds
     21        * Wall time to process: fpfiles_fwd_useavailable/EN13062500_bwd.fp :   0.230000004      seconds
     22    * diff shows zero differences with control
     23    * testing environment: Test performed.  Error = 0.0E+00 Test passed
     24
     25* __ECMWF global 1.0 degree fwd, 2 hours__
     26    * Generation of the fp files:
     27        * Wall time to process: EN13062500:    22.1739998      seconds
     28        * Wall time to dump to: EN13062500_fwd.fp:   0.537000000      seconds
     29        * Wall time to process: EN13062503:    22.0270004      seconds
     30        * Wall time to dump to: EN13062503_fwd.fp:   0.532999992      seconds
     31        * Size of each file: 473 MB
     32    * Running FLEXPART with the fp files:
     33        * Wall time to process: met_data/EN13062500_fwd.fp :   0.296000004      seconds
     34        * Wall time to process: met_data/EN13062503_fwd.fp :   0.273999989      seconds
     35    * diff shows zero differences with control
     36    * testing environment: testing environment: Test performed.  Error = 0.0E+00 Test passed
     37
     38* __NCEP global 1.0 degree bwd, 2 hours__
     39    * Generation of the fp files:
     40        * Wall time to process: GF15021603:    7.11800003      seconds
     41        * Wall time to dump to:  GF15021603_bwd.fp:   0.526000023      seconds
     42        * Wall time to process: GF15021600:    6.53900003      seconds
     43        * Wall time to dump to: GF15021600_bwd.fp:   0.554000020      seconds
     44        * Size of each file: 473 MB
     45    * diff shows zero differences with control
     46    * testing environment: Test performed.  Error = 0.0E+00 Test passed
     47
     48* __NCEP global 1.0 degree fwd, 2 hours__
     49    * Generation of the fp files:
     50        * Wall time to process: GF15021600:    6.86299992      seconds
     51        * Wall time to dump to:  GF15021600_fwd.fp:   0.531000018      seconds
     52        * Wall time to process: GF15021603:    6.28499985      seconds
     53        * Wall time to dump to: GF15021603_fwd.fp:   0.542999983      seconds
     54        * Size of each file: 473 MB
     55    * diff shows zero differences with control
     56    * testing environment: Test performed.  Error = 0.0E+00 Test passed
     57
     58* __ECMWF 6 hours 0.5 bwd with fp files__ 
     59    * Generation of the fp files:
     60        * Wall time to process: EN15021606:    165.356995      seconds
     61        * Wall time to dump to: EN15021606_bwd.fp:    3.00000000      seconds
     62        * Wall time to process: EN15021603:    158.772995      seconds
     63        * Wall time to dump to: EN15021603_bwd.fp:    2.62299991      seconds
     64        * Wall time to process: EN15021600:    147.024002      seconds
     65        * Wall time to dump to: EN15021600_bwd.fp:    2.89299989      seconds
     66        * Size of each file: 1.9 GB
     67    * Timings for run FLEXPART with met files
     68        * Wall time to process: EN15021606:    145.141998      seconds (not used in the simulation- unneded read)
     69        * Wall time to process: EN15021603:    151.067001      seconds
     70        * Wall time to process: EN15021600:    139.856995      seconds
     71    * Timings for run FLEXPART with fp files
     72        * Wall time to process: met_data/EN15021606_bwd.fp:   0.963999987      seconds
     73        * Wall time to process: met_data/EN15021603_bwd.fp:    16.5459995      seconds
     74        * Wall time to process: met_data/EN15021600_bwd.fp:    10.2419996      seconds
     75    * diff shows zero differences with control
     76    * testing environment: Test performed.  Error = 0.0E+00  Test passed
     77
     78* __ECMWF 9.5 days 1 deg bwd__
     79    * Generation of the fp files:
     80        * Wall time to process all 78 files:    6002.89898      seconds
     81        * Wall time to dump to all 78 files :    43.233      seconds
     82        * Size of all files: 36 GB (each file 473 MB)
     83    * Timings for run with met files: 6255 seconds
     84    * Timings for run with fp files: 290
     85    * testing environment:  Test performed.  Error = 0.0E+00  Test passed
     86    * This particular run has been investigated as well on a time to time basis showing no errors (and thus, no difference to the control files) for any of the simulation times:
     87
     88       {{{
     89 Max error for full volume, full timeseries:  0.0E+00
     90 RMSE for full volume, full timeseries:       0.0E+00
     91
     92
     93 Errors for full volume, timestamp 20150519110000:
     94 RMSE:       0.0E+00
     95 Max Error:  0.0E+00
     96
     97 Errors for full volume, timestamp 20150519100000:
     98 RMSE:       0.0E+00
     99 Max Error:  0.0E+00
     100
     101 Errors for full volume, timestamp 20150519090000:
     102 RMSE:       0.0E+00
     103 Max Error:  0.0E+00
     104
     105 ...
     106
     107 Errors for full volume, timestamp 20150510010000:
     108 RMSE:       0.0E+00
     109 Max Error:  0.0E+00
     110
     111 Errors for full volume, timestamp 20150510000000:
     112 RMSE:       0.0E+00
     113 Max Error:  0.0E+00
     114
     115       }}}
     116
     117= Comparison runs with fp files generated one at a time and with useAvail =
     118
     119There are different ways of generating the fp files. The one that would be most efficient and less time consuming, generation of one fp file at a time with new incoming data, implies a variable vertical grid coordinate system in FLEXPART. This may lead to some inconsistencies  or differences that should be evaluated (benefits or disadvantages) in depth for a range of test-cases. Here, we investigate visually and numerically the differences for the 9 day bwd test run.
     120
     121As expected, the tests performed with the testing environment using the control data fail because of the time-depending internal vertical coordinate syste, The total global error is  4.9E-01. The RMSE, although small, increases with time as seen in the figure below, whereas the overall error alternates positive and negative.
     122
     123'''''image missing'''''
     124
     125This relative information needs to be understood together with the concentration plots. The concentration (or residence time) plots show minimum differences due to the varying vertical coordinates. An example is shown below, with concentration averages for the two output vertical layers at the times of maximum "Max error" (both positive and negative) and at the end of the simulation, where the RMSE is as well maximum.
     126
     127'''''image missing'''''
     128
     129Differences exist but, for this specific case, seem to appear only at the edges of the plume. Further investigation is needed to understand whether such differences may be a problematic area.