Skip to content

Commit

Permalink
[icoscp-core] Collection metadata updates
Browse files Browse the repository at this point in the history
  • Loading branch information
mirzov committed Sep 12, 2024
1 parent d9ebb2f commit 2bf3349
Show file tree
Hide file tree
Showing 5 changed files with 38 additions and 10 deletions.
2 changes: 1 addition & 1 deletion build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ lazy val netcdf = (project in file("netcdf"))
credentials += Credentials(Path.userHome / ".ivy2" / ".credentials")
)

val metaCoreModule: ModuleID = "se.lu.nateko.cp" %% "meta-core" % "0.7.19"
val metaCoreModule: ModuleID = "se.lu.nateko.cp" %% "meta-core" % "0.7.20"

val osName: String = System.getProperty("os.name") match {
case name if name.startsWith("Linux") => "linux"
Expand Down
32 changes: 28 additions & 4 deletions src/main/python/icoscp_core/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,15 @@

A foundational ICOS Carbon Portal (CP) core products Python library for metadata and data access, designed to work with multiple data repositories who use ICOS Carbon Portal core server software stack to host and serve their data. At the moment, three repositories are supported: [ICOS](https://data.icos-cp.eu/portal/), [SITES](https://data.fieldsites.se/portal/), and [ICOS Cities](https://citydata.icos-cp.eu/portal/).

## Release notes

### Version 0.3.6
Update to support collection metadata enhancements
- `parentCollections` property added to `StaticCollection` class (through automatic code import from meta-core Scala lib jar)
- `coverage` property added to `StaticCollection` (manually re-declared as `Optional[GeoFeatureWithGeo]` to include GeoJSON)
- code examples for `meta.get_collection_meta` updated accordingly
- `StaticCollection.members` property content has been made more lightweight in the case of presence of sub-collections (server change, applied to this library through automatic code import). The values can now be either `PlainStaticObject` or `PlainStaticCollection`, both lightweight classes. They have different fields (`name` vs `title`, respectively), so should be tested at runtime, as collections can contain both objects and other collections.

## Design goals

- offer basic functionality with good performance
Expand Down Expand Up @@ -191,16 +200,31 @@ dobj_meta = meta.get_dobj_meta(dobj_uri)
```

### Fetch metadata for a collection
Some data objects belong to collections. Collections can also contain other collections. Collections can be discovered on the data portal app, or from individual data object metadata (as parent collections), for example:
Some data objects belong to collections. Collections can also contain other collections. Collections can be discovered on the data portal app, or from individual data object metadata (as parent collections), and then traversed to the top of the collection hierarchy, for example:
```Python
dobj = meta.get_dobj_meta('https://meta.icos-cp.eu/objects/hujSGCfmNIRdxtOcEvEJLxGM')
coll_uri = dobj.parentCollections[0].uri
coll_meta = meta.get_collection_meta(coll_uri)
coll_info = dobj.parentCollections[0]
coll_label = coll_info.label
coll_2010 = meta.get_collection_meta(coll_info.uri)
coll_evapo_info = coll_2010.parentCollections[0]
coll_evapo = meta.get_collection_meta(coll_evapo_info.uri)
evapo_years = coll_evapo.members
top_coll_info = coll_evapo.parentCollections[0]
top_coll = meta.get_collection_meta(top_coll_info.uri)
top_subcols = top_coll.members
```

Collections contain optional geo-coverage information in a property `coverage`. The value is from a custom `GeoFeature` class hierarchy, ehnanced with a GeoJSON-containing property `geo`, for example:

```Python
tukuma_coll = meta.get_collection_meta("https://meta.icos-cp.eu/collections/iCa6EQA8ClDKSvXgQMuwFAon")
tukuma_geo = tubuma_coll.coverage
tukuma_geojson = tukuma_geo.geo
```

### Note

Detailed help on the available metadata access methods can be obtained from `help(meta)` call.
Detailed help on the available metadata access methods can be obtained from `help(meta)` call, and then by calling help on the input and return types of the methods of interest.

## Repository-specific functionality

Expand Down
2 changes: 1 addition & 1 deletion src/main/python/icoscp_core/src/icoscp_core/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@
core services, such as authentication, metadata access, and data access.
"""

__version__ = "0.3.5"
__version__ = "0.3.6"
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

@dataclass(frozen=True)
class GeoJsonHolder:
geo: Any
geo: dict[str, Any]

@dataclass(frozen=True)
class FeatureCollectionWithGeo(FeatureCollection, GeoJsonHolder): pass
Expand Down
10 changes: 7 additions & 3 deletions src/main/python/icoscp_core/src/icoscp_core/metaclient.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
from .queries.dataobjlist import DataObjectLite, parse_dobj_lite, dataobj_lite_list
from .queries.dataobjlist import Filter, OrderBy, OrderByProp, CategorySelector
from .queries.stationlist import station_lite_list, parse_station, StationLite
from .metacore import DataObject as VanillaDataObject, CPJson, StaticCollection, parse_cp_json
from .metacore import CPJson, parse_cp_json
from .metacore import DataObject as VanillaDataObject, StaticCollection as VanillaStaticCollection
from .rolemeta import StationWithStaff
from .geofeaturemeta import GeoFeatureWithGeo, Point
from .http import http_request
Expand All @@ -20,12 +21,15 @@

@dataclass(frozen=True)
class DataObject(VanillaDataObject):
coverageGeo: Any
coverageGeo: dict[str, Any]

@dataclass(frozen=True)
class Station(StationWithStaff):
class StaticCollection(VanillaStaticCollection):
coverage: Optional[GeoFeatureWithGeo]

@dataclass(frozen=True)
class Station(StationWithStaff):
coverage: Optional[GeoFeatureWithGeo]

class MetadataClient:
def __init__(self, envri_conf: EnvriConfig):
Expand Down

0 comments on commit 2bf3349

Please sign in to comment.