[Platform Initiative] Allow users to dynamically build environments in the hub

[Gman0909]

Description

As hubs get a wider user base, different users need different environments to do their work. The same user may need multiple environments at different times too. Long term, we can not rely on admins to continuously update the small set of images available, and for users to have to keep up with modifying their code to work with newer packages. We need a way for users to be able to specify environments (without having to really understand Docker), with a view to later being able to share them with others in a standardized way. This should be done in a way that’s compatible with community standards as well, rather than re-inventing something new. As part of the previous PI, we have deployed jupyterhub-fancy-profiles to VEDA for an improved logged-in environment choosing experience. This builds on top of that, with the goal of making it easier for users to easily & reproducibly create and use whatever environments they want, and eventually share it with others as well (See #5087 ).

Proposal

We improve the Dynamic Image Building feature of jupyterhub-fancy-profiles (courtesy a prior grant from GESIS) until it can be put in front of production users. Once deployed, end users can easily create environments in the form of GitHub repositories (or Zenodo DOIs, or many other versioned sources) and, at some point in future, share it with each other (See #5087 ). These same environments will work on mybinder.org, our existing image building infrastructure and the ephemeral hub set up as part of JH-5 by using the same backend technology to build images (binderhub and repo2docker).

This work has partially been done as part of the DevSeed collaboration, but needs to be completed.

See related issues here and here

The second part of this effort is captured in #5087

Definition of Done

• We have a clear UX allowing users to define, build and share their custom environments
• The UX is tested and validated with end-users
• It has been deployed into a production hub
• It has been shown to be replicably deployable
• It is well documented
• It has been added to our product menu.
• Lessons learned have been documented
• We have communicated the availability of this feature to our communities
• We have communicated the availability of this features to our Sales function