Package example notebooks in JupyterLite

[jasongrout]

This packages many of the example notebooks in a live JupyterLite instance available from the documentation.

CC @martinRenou

[github-actions]

👈 Launch a binder notebook on branch jasongrout/ipywidgets/jlite2

[martinRenou]

It seems that nbsphinx and jupyterlite-sphinx both claim to render ipynb files, and it seemed I was getting some conflicts, so for now I disabled nbsphinx.

Indeed, this is a known issue. I pushed a PR to nbsphinx to try to fix. It probably needs updating: spatialaudio/nbsphinx#637

[martinRenou]

A workaround is to not use the custom parser, and instead use the directives directly

[jasongrout]

A workaround is to not use the custom parser, and instead use the directives directly

Oh, right, and manually override the parser for ipynb files. Nice!

[martinRenou]

You might want to give xeus-python a try in this PR :) Using https://xeus-python-kernel.readthedocs.io/en/latest/configuration.html to pre-install ipywidgets will allow to remove the first

import piplite
await piplite.install('ipywidgets')

cell from the notebooks

[jasongrout]

You might want to give xeus-python a try in this PR :) Using https://xeus-python-kernel.readthedocs.io/en/latest/configuration.html to pre-install ipywidgets will allow to remove the first

Good point. I hear that soon we'll be able to set up an environment soon anyway.

Are there any incompatibilities still with xeus-python vs ipykernel?

[jasongrout]

@martinRenou - I tried switching to xeus-python, but it still seems to require the importing and using piplite (in the current doc build in CI). Is there something wrong with my config now?

[martinRenou]

Are there any incompatibilities still with xeus-python vs ipykernel?

xeus-python is pretty close to ipykernel now, there are no big differences that I'm aware of. But note that ipykernel is not used in JupyterLite. The default kernel in JupyterLite is named pyolite, it's backed by pyodide, not ipykernel.

When it comes to differences between xeus-python and JupyterLite's default kernel "pyolite":

• pyolite is backed by pyodide, xeus-python is not
• from time import sleep works in xeus-python, it does not in pyolite
• xeus-python allows pre-installing packages (see https://xeus-python-kernel.readthedocs.io/en/latest/configuration.html), pyolite does not
• pyolite allows dynamically installing packages with piplite, xeus-python does not

[martinRenou]

@martinRenou - I tried switching to xeus-python, but it still seems to require the importing and using piplite (in the current doc build in CI). Is there something wrong with my config now?

Maybe it's a cache issue? Do you not see xeus-python in the JupyterLab launcher?

[jasongrout]

Maybe it's a cache issue? Do you not see xeus-python in the JupyterLab launcher?

No, I don't. Here's the latest readthedocs CI build from this PR: https://ipywidgets--3491.org.readthedocs.build/en/3491/

Does the jupyterlite config look right? https://github.com/jupyter-widgets/ipywidgets/blob/d0e0c6f5b21d77322d0c0aac7db40500faf4f534/docs/source/jupyterlite_config.json

[jasongrout]

@martinRenou - ah, got it working: I had to install jupyterlite-xeus-python

[jasongrout]

Current state:

• widgets won't render in xeus-python kernel (import ipywidgets does work, though)
• there seems to be no way to install a dev install of ipywidgets in jupyterlite, so this is limited use for building latest docs?
• the default kernel seems to still be pyolite - is there a way to disable the pyolite kernel and just use xeus-python?

[martinRenou]

widgets won't render in xeus-python kernel (import ipywidgets does work, though)

I'll give it a try locally.

there seems to be no way to install a dev install of ipywidgets in jupyterlite, so this is limited use for building latest docs?

Using a dev installation might be tricky indeed, but probably not impossible. I wonder if we could make this case easier on the jupyterlite-xeus-python front.

the default kernel seems to still be pyolite - is there a way to disable the pyolite kernel and just use xeus-python?

Yes. I've done it in the ipycanvas documentation: https://github.com/martinRenou/ipycanvas/blob/master/docs/jupyter-lite.json#L6, you must tell jupyterlite-sphinx that the build directory is the local one https://github.com/martinRenou/ipycanvas/blob/master/docs/conf.py#L9 for it to find this config file.

[bollwyvl]

there seems to be no way to install a dev install of ipywidgets in jupyterlite, so this is limited use for building latest docs?

yeah, that was the explicit goal of the build-time --piplite-wheels and piplite... i haven't been able to invest any effort in getting up to speed on the xeus kernels. maybe you could conda-build it? that seems extra heavy for a per-push docs build.

https://jupyterlite.readthedocs.io/en/latest/howto/python/wheels.html

In pyolite, it's now also theoretically possible to create an ipython profile, though the hard-coded path to /drive might behave in ways we don't expect... we might need some way to do additional mounts into emscripten.

[jasongrout]

I rebased and consolidated a lot of commits to clean things up a bit.

[jasongrout]

Interestingly, the output widget cannot capture exceptions in JupyterLite, even though it does in ipykernel:

from ipywidgets import Output
out = Output(layout={'border': '1px solid red'})
display(out)

with out:
    print('This output is captured')
    raise Exception('This exception is captured in ipykernel, but not in pyolite')

(#3417 tries to enable this in general, but it was surprising that output widgets behave like this anyway in JupyterLite)

@martinRenou @jtpio @bollwyvl - is this expected? If not, I can file an issue somewhere.

[bollwyvl]

output widgets behave

Yeah... no two Output implementations appear to work the same way 😿. I don't know where I'd start looking into this, but having upstream testing our stuff will certainly give us a goal.

[jasongrout]

I'm trying to skip some example notebook files. I have two attempts that are failing:

First, I tried giving an array of files to jupyterlite_contents option of jupyterlite_sphinx, which is translated into command-line --contents options, one for each entry. However, this merges the examples files and the images directory, where I'd like the images directory to be a subdirectory of the top-level directory rather than merged with it.

jupyterlite_contents = [
"examples/Beat Frequencies.ipynb",
...
"examples/images",
]

(i.e., the assets in examples/images are copied to the top-level filesystem, but I'd like them to actually be a subdirectory images).

I also tried going back to the jupyterlite_contents = "examples" option (i.e., --contents examples that preserved the examples/images subdirectory as a subdirectory, but also giving the following config in jupyter_lite_config.json:

  "LiteBuildConfig": {
    "ignore_contents": [
      "examples/Layout Example.ipynb",
      "examples/Layout Templates.ipynb",
      "examples/Variable Inspector.ipynb"
    ],
...

This preserved the examples/images directory as a subdirectory in JupyterLite, but it seems that the ignored_contents directory was ignored, i.e., I still see those files in my contents.

Any suggestions on how I can ignore three specific files in my contents directory?

[jtpio]

Right the file hierarchy appears to be flatten when specifying multiple contents options (also noticed elsewhere). Probably this is an issue with jupyterlite-sphinx.

Any suggestions on how I can ignore three specific files in my contents directory?

I would expect this to work based on the reply in jupyterlite/jupyterlite#709 (comment).

Or maybe it's because it should be a list of regex and some characters must be escaped. For example like with the dynamic default: https://github.com/jupyterlite/jupyterlite/blob/fc5875f0ba8ed34c4d78f6c82d98691eab95ee8b/py/jupyterlite/src/jupyterlite/config.py#L183-L207

[bollwyvl]

Tried out some stuff mentioned above over on #3517:

jasongrout/ipywidgets@jlite2...bollwyvl:ipywidgets:jlite2

[jasongrout]

Tried out some stuff mentioned above over on #3517:

I merged Nick's changes into this PR. Thanks Nick!

[bollwyvl]

Add JupyterLab Github extension

Good luck! I don't know if that's going to work out-of-the-box...

[jasongrout]

I think this PR is at a good place to merge, and future iteration can be on a different PR. CC @ibdafna

[ibdafna]

Excited for this one 😍