Fallback Downloads included?

[j0nes2k]

Hello,

I did not find it in the listed features, therefore I am asking here: what happens if a file does not (yet) exist on my preferred source? Does Herbie attempt to use the next data source listed and/or is there a configurable set of retries available?

Thank you for your help!

Jonas

[blaylockbk]

Hi @j0nes2k,
Thanks for your question. Using the HRRR model as an example, the default behavior is to look for the requested file at the following sources in this order:

1. AWS
2. NOMADS
3. Google
4. Azure
5. Pando

If the file is not found on AWS, then Herbie will look at NOMADS, and so forth, until the file is found (if there is one found).

You can use the argument priority to specify the order the sources are searched.

# Only look for file at AWS
Herbie('2022-01-01', model='hrrr', product='sfc', fxx=0, priority=['aws'])

# Look at NOMADS first, then Google
Herbie('2022-01-01', model='hrrr', product='sfc', fxx=0, priority=['nomads', 'google'])

There isn't a "retry" feature in Herbie, but you could implement one of your own (below is untested)...

# Retry Herbie at least 5 times if file is not found
for i in range(5):
    print(f"Try to find the GRIB file (attempt={n+1})")
    H = Herbie('2022-01-01', model='hrrr', )
    if H.grib:
        # A grib file was found (because H.grib is not None)
        break
    else:
        # A grib file was not found (because H.grib is None)
        # Try again after 30 seconds
        time. Sleep(30)