Jupyter Server startup really slow when panel is installed

[yuvipanda]

ALL software version info

(notebook) jovyan@jupyter-yuvipanda:~$ pip list | grep -P 'panel|jupyter'
jupyter_client                7.4.8
jupyter_core                  5.1.2
jupyter-panel-proxy           0.1.0
jupyter-resource-usage        0.6.4
jupyter-server                1.23.4
jupyter-server-mathjax        0.2.6
jupyter-server-proxy          3.2.2
jupyter-telemetry             0.1.0
jupyterhub                    3.1.0
jupyterlab                    3.5.1
jupyterlab-git                0.41.0
jupyterlab-pygments           0.2.2
jupyterlab-s3-browser         0.12.0
jupyterlab_server             2.18.0
jupyterlab-widgets            3.0.5
panel                         0.14.2

Description of expected behavior and the observed behavior

When starting a JupyterHub server with a container that has panel installed, sometimes the server process takes so long to startup that it times out and gets killed. Upon further investigation, this was partially due to importing the panel serverextension, which takes approximately 7s on some servers. This is due to importing some heavy packages at startup, including IPython itself (which takes about 2s)

Complete, minimal, self-contained example code that reproduces the issue

(notebook) jovyan@jupyter-yuvipanda:~$ time python -c 'import panel.io.jupyter_server_extension'

real    0m7.538s
user    0m6.600s
sys     0m0.870s

(notebook) jovyan@jupyter-yuvipanda:~$ time python -c 'from ipykernel.comm import Comm'

real    0m2.117s
user    0m1.791s
sys     0m0.285s

Note that these numbers are without an existing .pyc cache files, which are generally removed when building containers. So on second run, I get these numberse:

(notebook) jovyan@jupyter-yuvipanda:~$ time python -c 'import panel.io.jupyter_server_extension'

real    0m3.235s
user    0m2.752s
sys     0m0.486s

still kinda slow, but better.

Jupyter Server seems to import all extension modules even if they are not enabled, so there isn't a way to even temporarily disable the extension - only way to speed up the server startup is to uninstall panel completely.

A suggestion for tackling this would be to do the imports only when necessary, rather than at startup.

[yuvipanda]

2i2c-org/infrastructure#2047 has more of the investigation.

[yuvipanda]

I opened jupyter-server/jupyter_server#1171 to surface this performance information more readily

[philippjfr]

Thanks for posting @yuvipanda! Seems like avoiding the IPython import would make a huge difference here. Happy to find a way around that and cut a 0.14.3 release asap.

[philippjfr]

In #4276 I've now split the code that runs as part of the server extension and the code that is executed inside a kernel which reduces the imports significantly.

[yuvipanda]

Wow amazing that was really quick thank you so much @philippjfr

[yuvipanda]

That's awesome, @philippjfr! LMK if / when you think the next release may be :)

[yuvipanda]

Looking at this right now, with panel 1.2, it still adds about 1.6s to jupyterhub server startup time, which is a couple orders of magnitude moree than some other packagse - for example, JupyterLab adds 0.0387s. I think in some cases panel is now the biggest contributor to server startup time! Think there are any more optimizations we could do?

[philippjfr]

Thanks for letting me know. Sounds like a regression, I'll make sure to add some tests.