Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add xarray backends for fgmax and fgout gridded formats #611

Merged
merged 24 commits into from
Aug 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
b6b6308
initial commit of xarray backends and example use file
kbarnhart May 17, 2024
3119ed5
Merge branch 'master' into add_xarray_backends
kbarnhart May 17, 2024
ef4c778
remove extra words
kbarnhart May 17, 2024
744e0e6
relax assumption of where command is issued (no longer needs to be in…
kbarnhart May 18, 2024
9925e7c
add more information about rioxarray
kbarnhart Jun 5, 2024
51fb840
better import error for rioxarray
kbarnhart Jun 5, 2024
a8bc42c
add B and level
kbarnhart Jun 5, 2024
972d838
Merge branch 'clawpack:master' into add_xarray_backends
kbarnhart Jun 14, 2024
dac2a2b
add clip example
kbarnhart Jul 5, 2024
a930916
Merge branch 'add_xarray_backends' of https://github.com/kbarnhart/ge…
kbarnhart Jul 5, 2024
d83bd23
return ds
kbarnhart Jul 5, 2024
c0ba204
wip
kbarnhart Aug 20, 2024
f51f2f6
Merge branch 'master' into add_xarray_backends
kbarnhart Aug 20, 2024
e21b294
first pass at using revised fgout data structure with backends.
kbarnhart Aug 20, 2024
eb3cd20
add a few more comments.
kbarnhart Aug 20, 2024
92375ff
name updates
kbarnhart Aug 21, 2024
f69c708
bdiff->bdif
kbarnhart Aug 21, 2024
145233c
remove vestigial _qelements
kbarnhart Aug 21, 2024
66d0d63
Add example that should work with/without dask
kbarnhart Aug 21, 2024
eb7981e
improve drytol handling
kbarnhart Aug 21, 2024
d4079f8
don't specify error type.
kbarnhart Aug 21, 2024
61460c1
switch from hard coded drytol to user-specified drytol
kbarnhart Aug 21, 2024
f61d155
update use of dry_tolerance
kbarnhart Aug 21, 2024
987417a
fix exceptions.
kbarnhart Aug 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 87 additions & 0 deletions examples/tsunami/chile2010_fgmax-fgout/use_xarray_backends.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
import glob

try:
import rioxarray
import xarray as xr
except ImportError:
"You must install xarray and rioxarray in order to use the xarray backends"
raise

from clawpack.geoclaw.xarray_backends import FGMaxBackend, FGOutBackend

# epsg code for lat-lon
# Optionally, provide an epsg code to assign the associated coordinate system to the file.
# default behavior assigns no coordinate system. Note that if no coordinate system is provided,
# it will come in a GIS with coordinates associated with row and column number (not the x and y position
# encoded in the netcdf).

epsg_code = 4326

# An example of a fgout grid.
filename = "_output/fgout0001.b0001"
# provide the .bxxx file if binary format is used or the
# .qxxx file if ascii format is used.
# the format, fg number, and frame number are inferred from the filename.

ds = xr.open_dataset(
filename,
engine=FGOutBackend,
backend_kwargs={
"epsg": epsg_code,
"qmap": "geoclaw",
# qmap is the qmap specified to the fgout object in setrun.py see
# the following documentation page for more details.
# https://www.clawpack.org/dev/fgout.html#specifying-q-out-vars
"dry_tolerance": None,
# variables that are not eta and B are masked
# where h>dry_tolerance. To turn this functionality
# off set dry_tolerance = None.
},
)
# ds is now an xarray object. It can be interacted with directly or written to netcdf using
ds.to_netcdf("fgout0001_0001.nc")

# It is possible to combine all fgout files into a single netcdf file
# using xr.open_mfdataset (requires dask) or xr.concat (does not require dask)
# https://docs.xarray.dev/en/stable/generated/xarray.open_mfdataset.html
# https://docs.xarray.dev/en/latest/generated/xarray.concat.html
# for instructions on installing xarray with dask see:
# https://docs.xarray.dev/en/latest/getting-started-guide/installing.html#instructions

fgout_files = glob.glob("_output/fgout0001.b*")

try:
ds_all = xr.open_mfdataset(
fgout_files,
engine=FGOutBackend,
backend_kwargs={"epsg": epsg_code, "qmap": "geoclaw"},
)
except ValueError: # if dask is not available, use xr.concat.
# if dask is not installed xr.open_mfdataset() will fail with something like
# ValueError: unrecognized chunk manager dask - must be one of: []
fgouts = []
for filename in fgout_files:
ds = xr.open_dataset(
filename,
engine=FGOutBackend,
backend_kwargs={"epsg": epsg_code, "qmap": "geoclaw"},
)
fgouts.append(ds)

ds_all = xr.concat(fgouts, dim="time")

# save out.
ds_all.to_netcdf("fgout_all.nc")

# An example of a fgmax grid.
filename = "_output/fgmax0001.txt"
ds = xr.open_dataset(filename, engine=FGMaxBackend, backend_kwargs={"epsg": epsg_code})
ds.to_netcdf("fgmax0001.nc")

# To see the use of clipping, change the tfinal in setrun to something like 2*3600.0
# the fgmax0001_clipped.nc will only be the area where the wave arrived within the considered time.
filename = "_output/fgmax0001.txt"
ds = xr.open_dataset(
filename, engine=FGMaxBackend, backend_kwargs={"epsg": epsg_code, "clip": True}
)
ds.to_netcdf("fgmax0001_clipped.nc")
Loading
Loading