Update dandischema

[jwodder]

dandischema 0.10.2 was released yesterday, and it increased the schema version to 0.6.8. As a result, uploads from clients using the new version are failing because "Server requires schema version 0.6.7". Please update the version of dandischema used by the server.

[jwodder]

Ping @dandi/archive-maintainers

[jjnesbitt]

Working on it now

[jjnesbitt]

It seems that the GUI doesn't yet work with the new discriminated union feature in 0.6.8, and we'll need to do some investigation/work to get that fixed. My recommendation in the short term is to pin the dandi cli to use 0.6.7, until we can get the API and client working with that change.

[satra]

@jjnesbitt and @mvandenburgh - could we add a basic programmatic test added for the gui? it seems like we keep running into this concern with schema changes, so that we can evaluate this before we merge such changes in the future?

[mvandenburgh]

@jjnesbitt and @mvandenburgh - could we add a basic programmatic test added for the gui? it seems like we keep running into this concern with schema changes, so that we can evaluate this before we merge such changes in the future?

We actually have a test like that already, it failing was what alerted us to this issue - https://github.com/dandi/dandi-archive/actions/runs/9897632400/job/27343595576?pr=1976. The breakage happened because the CLI upgraded its version of dandischema before we realized it breaks the GUI.

I think going forward, we should develop a more structured approach to schema upgrades so that things are more coordinated between the CLI and API/GUI release. @waxlamp I recall we've discussed this before, but never got anything concrete worked out.

[mvandenburgh]

Oh, I see now the CLI doesn't pin it's version of dandischema, so it wasn't an explicit action that caused this. Still, I think we need to put together some sort of schema upgrade "runbook" here

[jjnesbitt]

I think the core reason for this situation is that dandi/dandi-schema#245 was merged before any testing or verification against the API or GUI was done. I agree with Mike that we should have a well structured playbook for doing schema upgrades.

[candleindark]

One possibly suspect is the inclusion of the $schema key, even though running check-jsonschema --check-metaschema asset.json context.json dandiset.json published-asset.json published-dandiset.json doesn't produce any error. Our schema include some customization for backward compatibility, and the GUI may be pickier.

[dandibot]

🚀 Issue was released in v0.3.88 🚀