uncorrect error of missing ipykernel in dependencies

[aqc-carlodri]

Hi, and thanks for you effort in maintaining this package!

I keep having the following issue. I have a pixi project where I have installed only jupyterlab (4.3.1) and pixi-kernel (0.5.1) with Python 3.12.7. Then I have a folder where I want to store a notebook, where only ipykernel (6.29.5 ) is installed, always with Python 3.12.7.

After starting jupyterlab, if I navigate to the folder with my notebook and I try to start the (pixi) kernel I always get the following error:

Error Starting Kernel

To run the Python (Pixi) kernel, you need to add the ipykernel package to 
your project dependencies. You can do this by running 'pixi add ipykernel'
in your project directory and restarting your kernel.

If you continue to face issues, report them at https://github.com/renan-r-santos/pixi-kernel/issues

whereas the ipykernel package is there....

Can you help me diagnose the issue?

Here follows the output on the jupyterlab console.

Full jupyterlab error output

[W 2024-11-27 08:54:53.969 ServerApp] wrote error: "\nTo run the Python (Pixi) kernel, you need to add the ipykernel package to \nyour project dependencies. You can do this by running 'pixi add ipykernel'\nin your project directory and restarting your kernel.\n\nIf you continue to face issues, report them at https://github.com/renan-r-santos/pixi-kernel/issues\n"
    Traceback (most recent call last):
      File "D:\Documents\jupyterlab-pixi\.pixi\envs\default\Lib\site-packages\jupyter_server\services\sessions\handlers.py", line 90, in post
        s_model = await sm.create_session(
                  ^^^^^^^^^^^^^^^^^^^^^^^^
        ...<5 lines>...
        )
        ^
      File "D:\Documents\jupyterlab-pixi\.pixi\envs\default\Lib\site-packages\jupyter_server\services\sessions\sessionmanager.py", line 285, in create_session
        kernel_id = await self.start_kernel_for_session(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            session_id, path, name, type, kernel_name
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        )
        ^
      File "D:\Documents\jupyterlab-pixi\.pixi\envs\default\Lib\site-packages\jupyter_server\services\sessions\sessionmanager.py", line 343, in start_kernel_for_session
        kernel_id = await self.kernel_manager.start_kernel(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        ...<3 lines>...
        )
        ^
      File "D:\Documents\jupyterlab-pixi\.pixi\envs\default\Lib\site-packages\jupyter_server\services\kernels\kernelmanager.py", line 233, in _async_start_kernel
        kernel_id = await self.pinned_superclass._async_start_kernel(self, **kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "D:\Documents\jupyterlab-pixi\.pixi\envs\default\Lib\site-packages\jupyter_client\multikernelmanager.py", line 283, in _async_start_kernel
        raise km.ready.exception()  # type: ignore[misc]
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "D:\Documents\jupyterlab-pixi\.pixi\envs\default\Lib\site-packages\jupyter_client\multikernelmanager.py", line 232, in _add_kernel_when_ready
        await kernel_awaitable
      File "D:\Documents\jupyterlab-pixi\.pixi\envs\default\Lib\site-packages\jupyter_core\utils\__init__.py", line 198, in ensure_async
        result = await obj
                 ^^^^^^^^^
      File "D:\Documents\jupyterlab-pixi\.pixi\envs\default\Lib\site-packages\jupyter_server\services\kernels\kernelmanager.py", line 784, in wrapped_method
        raise err
      File "D:\Documents\jupyterlab-pixi\.pixi\envs\default\Lib\site-packages\jupyter_server\services\kernels\kernelmanager.py", line 748, in wrapped_method
        out = await method(self, *args, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "D:\Documents\jupyterlab-pixi\.pixi\envs\default\Lib\site-packages\jupyter_server\services\kernels\kernelmanager.py", line 853, in start_kernel
        return await super().start_kernel(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "D:\Documents\jupyterlab-pixi\.pixi\envs\default\Lib\site-packages\jupyter_client\manager.py", line 96, in wrapper
        raise e
      File "D:\Documents\jupyterlab-pixi\.pixi\envs\default\Lib\site-packages\jupyter_client\manager.py", line 87, in wrapper
        out = await method(self, *args, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "D:\Documents\jupyterlab-pixi\.pixi\envs\default\Lib\site-packages\jupyter_client\manager.py", line 435, in _async_start_kernel
        kernel_cmd, kw = await self._async_pre_start_kernel(**kw)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "D:\Documents\jupyterlab-pixi\.pixi\envs\default\Lib\site-packages\jupyter_client\manager.py", line 400, in _async_pre_start_kernel
        kw = await self.provisioner.pre_launch(**kw)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "D:\Documents\jupyterlab-pixi\.pixi\envs\default\Lib\site-packages\pixi_kernel\provisioner.py", line 34, in pre_launch
        environment = ensure_readiness(
            cwd=cwd,
            required_package=required_package,
            kernel_name=kernel_spec.display_name,
        )
      File "D:\Documents\jupyterlab-pixi\.pixi\envs\default\Lib\site-packages\pixi_kernel\pixi.py", line 104, in ensure_readiness
        raise RuntimeError(
        ...<3 lines>...
        )
    RuntimeError:
    To run the Python (Pixi) kernel, you need to add the ipykernel package to
    your project dependencies. You can do this by running 'pixi add ipykernel'
    in your project directory and restarting your kernel.

    If you continue to face issues, report them at https://github.com/renan-r-santos/pixi-kernel/issues


    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "D:\Documents\jupyterlab-pixi\.pixi\envs\default\Lib\site-packages\tornado\web.py", line 1790, in _execute
        result = await result
                 ^^^^^^^^^^^^
      File "D:\Documents\jupyterlab-pixi\.pixi\envs\default\Lib\site-packages\jupyter_server\auth\decorator.py", line 73, in inner
        return await out
               ^^^^^^^^^
      File "D:\Documents\jupyterlab-pixi\.pixi\envs\default\Lib\site-packages\jupyter_server\services\sessions\handlers.py", line 108, in post
        raise web.HTTPError(500, str(e)) from e
    tornado.web.HTTPError: HTTP 500: Internal Server Error (
    To run the Python (Pixi) kernel, you need to add the ipykernel package to
    your project dependencies. You can do this by running 'pixi add ipykernel'
    in your project directory and restarting your kernel.

    If you continue to face issues, report them at https://github.com/renan-r-santos/pixi-kernel/issues
    )

[renan-r-santos]

Hi @aqc-carlodri, thanks for the bug report. In a terminal, could you please navigate to the folder where you stored your notebook, run pixi info --json, and post the output here? That would likely help me understand what is going on. Also, make sure that you have a pixi.toml or pyproject.toml in the same folder of your notebook or at least in one of its parent folders.

Note that ipykernel needs to be installed in the default Pixi environment. pixi-kernel doesn't work with other Pixi environments for now, see #20, but this will soon change as I'm currently working on adding support for that.

[aqc-carlodri]

Thanks for the quick reply. Here is the output of the command you asked to run in the notebook folder (I just patched two identical paths with ... for confidentiality):

{
  "platform": "win-64",
  "virtual_packages": [
    "__win=0=0",
    "__cuda=12.4=0",
    "__archspec=1=skylake"
  ],
  "version": "0.38.0",
  "cache_dir": "D:\\.pixi\\cache",
  "cache_size": null,
  "auth_dir": "C:\\Users\\carlo.dri\\.rattler\\credentials.json",
  "global_info": {
    "bin_dir": "D:\\.pixi\\bin",
    "env_dir": "D:\\.pixi\\envs",
    "manifest": "D:\\.pixi\\manifests\\pixi-global.toml"
  },
  "project_info": {
    "name": "ir_image_processing",
    "manifest_path": "D:\\Documents\\...\\...\\ir_image_processing\\pixi.toml",
    "last_updated": "27-11-2024 08:44:31",
    "pixi_folder_size": null,
    "version": "0.1.0"
  },
  "environments_info": [
    {
      "name": "default",
      "features": [
        "default"
      ],
      "solve_group": null,
      "environment_size": null,
      "dependencies": [
        "ipykernel"
      ],
      "pypi_dependencies": [],
      "platforms": [
        "win-64"
      ],
      "tasks": [],
      "channels": [
        "conda-forge"
      ],
      "prefix": "D:\\Documents\\...\\...\\ir_image_processing\\.pixi\\envs\\default"
    }
  ],
  "config_locations": []
}

And yes, I can confirm the pixi.toml is in the same folder as the notebook, this is the contents:

[project]
channels = ["conda-forge"]
description = "Add a short description here"
name = "ir_image_processing"
platforms = ["win-64"]
version = "0.1.0"

[tasks]

[dependencies]
ipykernel = ">=6.29.5,<7"

[renan-r-santos]

Thanks for the quick reply. This all looks fine to me.
Right now I can only think of two possible reasons why this is failing for you:

1. This line is somehow providing the wrong path. The line just below logs the variable cwd. Could you check JupyterLab logs and see which path is shown? Your original comment didn't include this log entry.
2. Something regarding Windows and files in different drives maybe, when executing pixi info --json here. I don't have a Windows machine to test, but all CI tests run on Windows too, so I'm not sure.

I'm betting on the first option though. If you could go inside the .pixi folder where you installed pixi-kernel and edit provisioner.py to print kwargs.get("cwd", None), that could be helpful too.
Thanks again.

[aqc-carlodri]

The JupyterLab log line you are requesting says this:

[I 2024-11-28 07:50:18.996 ServerApp] The current working directory is D:\Documents\...\...\ir_image_processing

which is the same folder I redacted in the pixi info --json output. It is the folder where the notebook is saved (not the JupyterLab and pixi-kernel packages).

Regarding your last point, the output of kwargs.get("cwd", None) in provisioner.py just below the previous cwd log is:

D:\Documents\...\...\ir_image_processing

which is always the same path as above.

Thanks you for your time.

[aqc-carlodri]

So, I digged a little bit and I think the problem is in the ensure_readiness() function. This is the contents of the pixi_info object:

environments=[Environment(name='default', dependencies=['jupyterlab', 'pixi-kernel', 'python'], pypi_dependencies=[], prefix='D:\\Documents\\jupyterlab-pixi\\.pixi\\envs\\default')] project=Project(manifest_path='D:\\Documents\\jupyterlab-pixi\\pixi.toml')

which seems to be running the pixi info --json command in the directory of the JupyterLab project, not the notebook project.

Digging into a fix now... 🙂

Thoughts:

• the weird thing is that subprocess.run() is seemingly ignoring the cwd argument...
• the .absolute() method could be replaced with .resolve(), but I'm not sure this would help, will test later.

[aqc-carlodri]

So I think this is something related to where pixi decides to run commands, but didn't have time to dig any deeper. However, adding the --manifest-path to pixi info solves the problem for me.

The kernel now starts but I get this warning in the JupyterLab logs:

 WARN Using manifest D:\Documents\jupyterlab-pixi\pixi.toml from `PIXI_PROJECT_MANIFEST` rather than local D:\Documents\...\...\ir_image_processing\pixi.toml

[renan-r-santos]

Hmm... it is working for you now but it is probably using the environment in D:\Documents\jupyterlab-pixi which includes ipykernel as a transitive dependency. Dependencies you add to D:\Documents\...\...\ir_image_processing won't be available in your notebook I believe.
I assume using resolve didn't work right? I'll try to set up a VM with Windows and see if I can reproduce this issue. Ouf of curiosity, do the redacted folder names include spaces or special characters? Just so I could more easily reproduce your environment.

[aqc-carlodri]

no, the redacted paths are all like redacted_path, no spaces, only alphabetic characters and underscore.

Oh yes, and resolve() did not work.

[aqc-carlodri]

And yes, I can confirm that now in the notebook I can't use the dependencies from my notebook folder project.

[danielballan]

I think I am having the same issue, and I think I understand the solution. When JupyterLab itself is run using pixi run ..., pixi sets PIXI_PROJECT_MANIFEST to the pixi.toml that specifies the Jupyter _server_environment.

When pixi-kernel runs pixi subcommands, pixi detects PIXI_PROJECT_MANIFEST in the environment and uses that, skipping the discovery process. That is, PIXI_PROJECT_MANFIEST has precedence over the cwd or parent directories.

When asking about the best way around this in the pixi Discord, I was happy to learn that the next release of pixi will change the precedence order, so that cwd and parent directories are checked before PIXI_PRJOECT_MANFIEST. prefix-dev/pixi#2564

This should enable the use of pixi-kernel within a Jupyter server that was itself run using pixi.

[aqc-carlodri]

Wow @danielballan, yes that clarifies the problem and the solution perspective! Thanks a ton I was getting crazy trying to figure this out.

[danielballan]

Sure thing! @renan-r-santos Would you welcome a PR that adds the manifest path (as extracted from pixi info) to the error messages? I figure that might be useful for related issues, even after pixi resolves this one for us.

[renan-r-santos]

Thanks a lot, both of you! Not knowing why this was happening was bugging me so much.

We use this package at work and we don't get this issue there. I finally figured out why: I unset the PIXI_IN_SHELL environment variable and I didn't remember I did that 🤦. For instance, running (unset PIXI_IN_SHELL; pixi info --json) in a JupyterLab terminal would work fine and not display the warning about PIXI_PROJECT_MANFIEST.

@danielballan Yes, adding useful info to the error messages is always welcome. Regardless, I'm inclined to make a patch release later today or tomorrow removing PIXI_IN_SHELL from the env used in subprocess.run so pixi-kernel can work properly with pixi versions older than the future 0.39.0. If you prefer, I can include that in my PR, whichever way you prefer. Thanks!

[danielballan]

That makes sense to me.

I going to log off for US Thanksgiving, so if you add more detail before Monday, go for it! Otherwise I will sling a PR when I get back. Thanks for this useful project.

[renan-r-santos]

I released pixi-kernel 0.5.2 with a fix for this. It is already available on PyPI and should be available on conda-forge in a few hours. Feel free to reopen this if you still encounter issues. Thanks!

[aqc-carlodri]

Thank you for the quick fix! I have triggered the update PR on the conda-forge feedstock. As I said there, you can add me as maintainer with my other username @carlodri