flex_extract: TypeError: an integer is required

[fcheng]

Dear developers,

I recently used the developer version of flex_extract from Github to download ERA5. Due to the data size, I downloaded the data day by day.

It ran well and successfully produced the files for the FLEXPART model for 2022-09-01 and 2022-09-02, until it ran into an error TypeError?: an integer is required for 2022-09-03 after the MARS retrieve was done.

I'm pasting the error message below. Any help is much appreciated.

Thank you very much,
Franklin

Error message:

...
marsclass: EA
dataset: ERA5
type: AN
levtype: ML
levelist: 1
repres:
date: 20220903/to/20220903
resol: 399
stream: OPER
area: 90.0/-179.5/-90.0/180.0
time: 00/03/06/09/12/15/18/21
step: 00
expver: 1
number: OFF
accuracy: 24
grid: OFF
gaussian:
target: /disk/r059/tfchengac/flex_extract_github_dev/Run/Workspace/EA_temp/ANSH__SL.20220903.834135.834136.grb
param: 152.128
target: /disk/r059/tfchengac/flex_extract_github_dev/Run/Workspace/EA_temp/ANSH__SL.20220903.834135.834136.grb
RETRIEVE ERA5 WITH CDS API!
2024-02-01 22:06:44,492 INFO Welcome to the CDS
2024-02-01 22:06:44,492 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/reanalysis-era5-complete
2024-02-01 22:06:44,823 INFO Request is queued
2024-02-01 23:13:34,739 INFO Request is completed
2024-02-01 23:13:34,739 INFO Downloading https://download-0016.copernicus-climate.eu/cache-compute-0016/cache/data4/adaptor.mars.external-1706800396.2392218-9882-14-77ff4c23-b955-4ad3-8531-f0e19a7ef648.grib to /disk/r059/tfchengac/flex_extract_github_dev/Run/Workspace/EA_temp/ANSH__SL.20220903.834135.834136.grb (3.7M)
0%|          | 0.00/3.68M 
2024-02-01 23:13:38,210 INFO Download rate 1.1M/s
MARS retrieve done ...
Prepare 20220903/to/20220903
... index will be done
... index done
Traceback (most recent call last):
  File "/disk/r059/tfchengac/flex_extract_github_dev/Source/Python/submit.py", line 272, in <module>
    main()
  File "/disk/r059/tfchengac/flex_extract_github_dev/Source/Python/submit.py", line 110, in main
    prepare_flexpart(ppid, c)
  File "/disk/r059/tfchengac/flex_extract_github_dev/Source/Python/Mods/prepare_flexpart.py", line 161, in prepare_flexpart
    flexpart.deacc_fluxes(inputfiles, c)
  File "/disk/r059/tfchengac/flex_extract_github_dev/Source/Python/Classes/EcFlexpart.py", line 1009, in deacc_fluxes
    iid, index_vals = self._mk_index_values(c.inputdir,
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/disk/r059/tfchengac/flex_extract_github_dev/Source/Python/Classes/EcFlexpart.py", line 602, in _mk_index_values
    key_vals = codes_index_get(iid, key)
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/disk/r059/tfchengac/anaconda3/lib/python3.11/site-packages/gribapi/gribapi.py", line 2202, in grib_index_get
    result = grib_index_get_string(indexid, key)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/disk/r059/tfchengac/anaconda3/lib/python3.11/site-packages/gribapi/gribapi.py", line 1482, in grib_index_get_string
    nval = grib_index_get_size(indexid, key)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/disk/r059/tfchengac/anaconda3/lib/python3.11/site-packages/gribapi/gribapi.py", line 1433, in grib_index_get_size
    ih = get_index(indexid)
         ^^^^^^^^^^^^^^^^^^
  File "/disk/r059/tfchengac/anaconda3/lib/python3.11/site-packages/gribapi/gribapi.py", line 184, in get_index
    return ffi.cast("grib_index*", indexid)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/disk/r059/tfchengac/anaconda3/lib/python3.11/site-packages/cffi/api.py", line 300, in cast
    return self._backend.cast(cdecl, source)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: an integer is required

[fcheng]

Hi,

I found that the bug may stem from the recent update of the eccodes-python package. Since version 1.6.0, the package gets the API version as an integer (see ​https://pypi.org/project/eccodes/)

I am not sure how that may cause the bug. But I seem to get rid of the issue by downgrading my eccodes-python to ver 1.5.2, by pip install 'eccodes==1.5.2'.

Future versions of flex_extract may take into account the latest updates of the package. This ticket can be closed. Thank you for your attention.

Cheers,
Franklin