Compatibility with Jupyter client 6

[tanguycdls]

Hi, since jupyter client was upgraded to the version 6 nb conda kernel does not seem to work. the python kernels are no longer automatically discovered.

I noticed the following message during its install:
https://github.com/Anaconda-Platform/nb_conda_kernels/blob/03630983f2a9ead420fb0a4c464f544931ba377f/nb_conda_kernels/install.py#L111

How can we install nb conda kernel in Jupyter client 6?
Best

[kevin-bates]

You're correct. The experimental discovery model that happened to reside in 5.x was removed in 6.0 because that model has been formalized to jupyter_kernel_mgmt (JKM) and #151 has been created to track that. For the time being, I suspect you'll need to downgrade jupyter_client to 5.3.4.

The problem is that the model present in 5.x is not what is actually supported by JKM.

cc: @mcg1969 @MSeal

[tanguycdls]

Thank's for the prompt answer ! Yes for now we downgraded to 5.3.4, might be interesting to reflect that in the package specs so that we cannot install nb conda kernels w/ jupyter client 6.0.

If I may why the feature was removed from jupyter ?
EDIT: ok i read the full post !

Thank you for your work !

[MSeal]

@kevin-bates @tanguycdls do you two think we should provide something additional or different somewhere today to allow for execution, or should we instead workout and document how to do what was done before in 5.x in jupyter_client docs?

[kevin-bates]

This is a situation that doesn't have good solutions. Given that users were using the experimental discovery was unfortunate, but I'm not sure that state of affairs was properly communicated. Now, there are nb-conda-kernel installations that are installed where that discovery model is assumed to be in place. Had it not been present on installation, I suspect their installation would not have relied on it. @mcg1969 can speak more to that.

I think the right thing to is to restore the discovery stuff into jupyter_client and immediately implement jupyter/jupyter_client#495. I'm not sure what other side-effects that restoration has on JC 6 (I know it will impact my async kernel mgmt PR that I've spent most of this week updating to 6.x [sigh]).

Another option would be to revert NCK to the previous (and supported) discovery model. I suspect that might be doable out of the box, but I don't know if existing installations (that relied on the experimental approach) can be "told" to use the supported discovery model. However, this approach will likely impact NCK customers and why should they pay for something that NCK isn't at fault for?

This is very unfortunate!

[mcg1969]

I'm sorry I just wasn't following the development on the Jupyter side enough to realize that the model had changed in an incompatible way. I clearly misinterpreted the level of certainty that this was the way the new process would go.

The difficult piece for me is that I've not been able to devote the development time to this that it needs. I absolutely would welcome a PR to correct this and could promise a fast review and merge but I can't promise a delivery date for development of my own.

[kevin-bates]

Thanks for the response Mike. Did you intend to close this issue?

It looks like this might be the line to change: https://github.com/Anaconda-Platform/nb_conda_kernels/blob/03630983f2a9ead420fb0a4c464f544931ba377f/nb_conda_kernels/install.py#L104

If has_entrypoints is false - either hard-coded or incrementing the major version to compare against, then I believe the old approach would be used.

I don't use nb_conda_kernels. @tanguycdls - are you in a position to try that?

If that works out, it's probably easier to address this with a new minor version (2.3.0) - or even revision (2.2.3) of NCK.

[mcg1969]

I did not intend to close. I apologize!

[hadim]

Being a heavy nb_conda_kernels user I also had to downgrade all my installation to jupyter_client==5.3.4.

[hadim]

Actually the downgrading does not work unless I am doing something wrong:

$ conda list|grep jupyter
jupyter-archive           0.5.5                      py_0    conda-forge
jupyter_client            5.3.4                    py37_1    conda-forge
jupyter_conda             3.1.1                      py_1    conda-forge
jupyter_core              4.6.2                    py37_0    conda-forge
jupyterlab                2.0.0                      py_0    conda-forge
jupyterlab_code_formatter 1.1.0                      py_0    conda-forge
jupyterlab_server         1.0.6                      py_0    conda-forge

$ conda list|grep nb_conda
nb_conda_kernels          2.2.2                    py37_0    conda-forge

[mcg1969]

Try removing and reinstalling nb_conda_kernels at this point.

[hadim]

@mcg1969: indeed it works. Thanks.

[tanguycdls]

Thanks for the response Mike. Did you intend to close this issue?

It looks like this might be the line to change:

https://github.com/Anaconda-Platform/nb_conda_kernels/blob/03630983f2a9ead420fb0a4c464f544931ba377f/nb_conda_kernels/install.py#L104

If has_entrypoints is false - either hard-coded or incrementing the major version to compare against, then I believe the old approach would be used.

I don't use nb_conda_kernels. @tanguycdls - are you in a position to try that?

If that works out, it's probably easier to address this with a new minor version (2.3.0) - or even revision (2.2.3) of NCK.

Hi thanks for the help guys !
I will try that tmw. I dont see where has_entrypoints is used in the install ?

Do you have a minimum tests to check if nb conda kernel is working properly ? Querying https://github.com/Anaconda-Platform/nb_conda_kernels/blob/03630983f2a9ead420fb0a4c464f544931ba377f/nb_conda_kernels/discovery.py#L23 is enough ?

[kevin-bates]

Thanks @tanguycdls

I dont see where has_entrypoints is used in the install ?

https://github.com/Anaconda-Platform/nb_conda_kernels/blob/03630983f2a9ead420fb0a4c464f544931ba377f/nb_conda_kernels/install.py#L104-L116
I think has_entrypoints is primarily used for messaging. Because jupyter_client's version is now >= 6, has_entrypoints has moved to True. This is what leads to the logging of the error message, although I don't know what else is messed up - an error message shouldn't cause issues - other than annoyance.

By adjusting the comparison, such that it's now false, we should get back into the position we're in today, although I suspect we'll now hit the debug message "NOTE: entry points not used in Jupyter 6", but then that sets is_enabled_entry = False, so I think the behavior would be as if JC was downgraded.

[mcg1969]

For the sake of expediency I've made the fix in #156 and manually tested it with V6. The automatic tests depend on the defaults channel, though, which does not yet have V6.

[mcg1969]

I'm going to reopen this and invite people to test 2.2.2+10.g1b9e821 and let me know if it works. If so I'll tag 2.2.3

[evfro]

@mcg1969 Hi, thanks for your work! I'm experiencing a weird behavior with the recent version of JuyterLab. I'm not sure, if the problem is on the JupyterLab or nb_conda_kernels side. I initially created an issue there jupyterlab/jupyterlab#8388. But since I discovered this thread I'm starting to think it may be an nb_conda_kernels issue.

In short: with nb_conda_kernels enabled, it's no longer possible to interrupt execution of cells from within the JupyterLab's interface if the kernel is not from the env where jupyterlab is installed. I have provided more details and the reproducing steps in the issue there. Can you have a look as well? Thanks!

[mcg1969]

@evfro, while I'm sure your issue is valid it needs to be created separately. Can you go ahead and do that? I'm going to close this one because we have addressed the original purpose of the issue.

[evfro]

@mcg1969, I have opened a new issue #180, thanks!