Skip to content

Commit

Permalink
Update cmip6-tas.data.mdx
Browse files Browse the repository at this point in the history
  • Loading branch information
abarciauskas-bgse authored Oct 5, 2023
1 parent c8072d0 commit 244fad3
Showing 1 changed file with 5 additions and 3 deletions.
8 changes: 5 additions & 3 deletions datasets/cmip6-tas.data.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -69,11 +69,13 @@ Source: [https://www.nccs.nasa.gov/services/data-collections/land-based-products

There are 2 datasets listed on the AWS Registry of Open Data for [NEX-GDDP-CMIP6](https://registry.opendata.aws/nex-gddp-cmip6/). First, there is an archive of NetCDF files from about 35 different climate models, each supplying historical and predicted values for up to 9 environment variables, daily, from 1950 to 2100. Second, there is an archive of COGs generated from the corresponding NetCDFs to support visualization via dynamic tiling using COGs. COGs are only available for 2 models. The COG archive also includes monthly aggregatations across those models.

## This is a demo subset of NEX-GDDP-CMIP6
## Dataset Preparation

VEDA hosts a reference file which includes references to a subset of the NEX-GDDP-CMIP6 data, primarily for demo purposes. With this dataset, we demonstrate a method for visualizing the data without creating any data copies just for visualization.
### This dataset represents a subset of NEX-GDDP-CMIP6

To prepare this dataset, we used the [kerchunk](https://fsspec.github.io/kerchunk/) python library to create a file - often called a reference file - which is an index of the chunks of the data stored in the original [NetCDF](https://www.unidata.ucar.edu/software/netcdf/) files. This reference file is used by a dynamic tiling library ([titiler-xarray](https://github.com/developmentseed/titiler-xarray)) to create image tiles from the underlying data. The indexes were generated for the near-surface air temperature variable (TAS) for years 1950-2014, the historical extent of the NEX-GDDP-CMIP6 project.
VEDA is hosting a single JSON file which references a subset of the NEX-GDDP-CMIP6 data in NetCDF. We used the [kerchunk](https://fsspec.github.io/kerchunk/) python library to create a JSON file - often called a reference file - which is an index of the chunks of the data stored in the original [NetCDF](https://www.unidata.ucar.edu/software/netcdf/) files. This can be considered a "virtual" Zarr dataset. The virtual Zarr dataset (aka the kerchunk reference file) is used by a dynamic tiling library ([titiler-xarray](https://github.com/developmentseed/titiler-xarray)) to create image tiles from the underlying data. The indexes were generated for the near-surface air temperature variable (TAS) for years 1950-2014, the historical extent of the NEX-GDDP-CMIP6 project.

We are using these methods to demonstrate a method for visualizing a "virtual" Zarr dataset which allows for dynamic visualization without having to create and maintain any additional copies of data.

</Prose>
</Block>

0 comments on commit 244fad3

Please sign in to comment.