Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue overview: finalize and verify extension, store and codec mechanisms #169

Closed
4 of 6 tasks
jstriebel opened this issue Nov 18, 2022 · 2 comments
Closed
4 of 6 tasks
Assignees
Labels

Comments

@jstriebel
Copy link
Member

jstriebel commented Nov 18, 2022

This is a high-level issue to track the progress around extensions, stores and codecs to finalize the v3 core spec. Finding the right abstraction levels for those is important, since it provides hints for implementors where to generalize concepts, which might make it easier in the long run to add future extension. In short: Having good extension points helps to grow support for extensions. Personally, I think the current status of the extensions is great, this is just to finalize those.

Please use or open the specific issues to discuss single points here. This is mostly to meta-discuss the overall strategy and overview, please feel free to recommend items that should be on this list, this just represents my current thoughts and neither exhaustive nor all strictly necessary (although I'll try to update it based on progress and comments):

Ideas for other extension points

Those are ideas that were floating around in meetings and issues, we might want to add those extension points

  • WIP Global storage transformer extension mechanism. PR Global storage transformers #182

  • Metadata conventions could help to standardize metadata fields. Metadata mostly comes and is used by

    1. the writing zarr-lib itself
    2. higher-level libraries and viewers (eg. GeoZarr, OME-NGFF, neuroglancer, webKnossos, …)
    3. the user.

    Having a specific section and/or document for metadata convention would foster cross-compatibility between producers and viewers (i. & ii.). I think this should not enforce anything on the zarr level, but is a guideline/convention which might be used implementations. This also popped up in Writer name field #139 and Revise how the domain of an array is specified #144. I'll try to draft a PR for this to have some basis for discussion.

    See ZEP 4

Finalize a spec per extension/store/codec:

To verify that all extension points are specified in full depth we might want to have finalized examples for each of those, also for stores and codecs. It might be useful to either include some of them directly into ZEP 1, or directly open separate ZEPs.

Add templates

Once the points above are finalized we might want to extract templates from them to ease adding new extensions and facilitate homogeneous specs across those.

@jstriebel jstriebel added help wanted Extra attention is needed core-protocol-v3.0 Issue relates to the core protocol version 3.0 spec protocol-extension Protocol extension related issue metadata-convention data-type labels Nov 18, 2022
@jstriebel jstriebel self-assigned this Nov 18, 2022
@jstriebel jstriebel added this to ZEP1 Nov 18, 2022
@jstriebel jstriebel added v3-meta and removed help wanted Extra attention is needed core-protocol-v3.0 Issue relates to the core protocol version 3.0 spec protocol-extension Protocol extension related issue metadata-convention data-type labels Nov 18, 2022
@jstriebel jstriebel moved this to Meta in ZEP1 Nov 18, 2022
@jstriebel
Copy link
Member Author

see also #89

@jstriebel jstriebel changed the title finalize and verify extension, store and codec mechanisms Issue overview: finalize and verify extension, store and codec mechanisms Nov 24, 2022
@jstriebel
Copy link
Member Author

This is mostly done, only #182 seems left over for ZEP 1

@github-project-automation github-project-automation bot moved this from Meta to Done in ZEP1 Feb 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Done
Development

No branches or pull requests

1 participant