Unable to download packages from nvidia and pytorch channels

[navot-arad]

I am unable to download any packages from either the nvidia or pytorch channels.

For both channels conda is able to resolve the dependencies, but fails on the download, returning a HTTP 404 error for all packages requested from either the nvidia or pytorch channels.

I am able to install packages without issue from the conda-forge and bioconda channels. I have not checked any others.

nvidia channel example

(tmp) C:\Users\narad>conda install -c nvidia libcublas --override-channels
Channels:
 - nvidia
Platform: win-64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: C:\Users\narad\AppData\Local\miniforge3\envs\tmp

  added / updated specs:
    - libcublas


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    cuda-nvrtc-12.6.68         |                0        34.2 MB  nvidia
    cuda-version-12.6          |                3          16 KB  nvidia
    libcublas-12.6.1.4         |                0       379.0 MB  nvidia
    ------------------------------------------------------------
                                           Total:       413.3 MB

The following NEW packages will be INSTALLED:

  cuda-nvrtc         nvidia/win-64::cuda-nvrtc-12.6.68-0
  cuda-version       nvidia/noarch::cuda-version-12.6-3
  libcublas          nvidia/win-64::libcublas-12.6.1.4-0


Proceed ([y]/n)?


Downloading and Extracting Packages:


CondaHTTPError: HTTP 404 NOT FOUND for url <https://prefix.dev/nvidia/noarch/cuda-version-12.6-3.tar.bz2>
Elapsed: 00:00.695236

An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.

CondaHTTPError: HTTP 404 NOT FOUND for url <https://prefix.dev/nvidia/noarch/cuda-version-12.6-3.tar.bz2>
Elapsed: 00:00.695236

An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.

CondaHTTPError: HTTP 404 NOT FOUND for url <https://prefix.dev/nvidia/win-64/libcublas-12.6.1.4-0.tar.bz2>
Elapsed: 00:00.788516

An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.

CondaHTTPError: HTTP 404 NOT FOUND for url <https://prefix.dev/nvidia/win-64/cuda-nvrtc-12.6.68-0.tar.bz2>
Elapsed: 00:00.784702

An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.

pytorch channel example

(tmp) C:\Users\narad>conda install -c pytorch pytorch -c conda-forge --override-channels
Channels:
 - pytorch
 - conda-forge
Platform: win-64
Collecting package metadata (repodata.json): done
Solving environment: done


==> WARNING: A newer version of conda exists. <==
    current version: 24.3.0
    latest version: 24.7.1

Please update conda by running

    $ conda update -n base -c conda-forge conda



## Package Plan ##

  environment location: C:\Users\narad\AppData\Local\miniforge3\envs\tmp

  added / updated specs:
    - pytorch


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    blas-1.0                   |              mkl           1 KB  conda-forge
    filelock-3.16.0            |     pyhd8ed1ab_0          17 KB  conda-forge
    libblas-3.9.0              |1_h8933c1f_netlib         193 KB  conda-forge
    libcblas-3.9.0             |5_hd5c7e75_netlib          95 KB  conda-forge
    liblapack-3.9.0            |5_hd5c7e75_netlib         2.7 MB  conda-forge
    markupsafe-2.1.5           |  py310ha8f682b_1          26 KB  conda-forge
    mkl-2023.1.0               |   h6a75c08_48682       137.2 MB  conda-forge
    mpmath-1.3.0               |     pyhd8ed1ab_0         428 KB  conda-forge
    networkx-3.3               |     pyhd8ed1ab_1         1.1 MB  conda-forge
    pytorch-2.4.1              |     py3.10_cpu_0       143.8 MB  pytorch
    pytorch-mutex-1.0          |              cpu           3 KB  pytorch
    pyyaml-6.0.2               |  py310ha8f682b_1         153 KB  conda-forge
    sympy-1.13.2               |     pyh04b8f61_3         4.4 MB  conda-forge
    ------------------------------------------------------------
                                           Total:       290.1 MB

The following NEW packages will be INSTALLED:

  blas               conda-forge/win-64::blas-1.0-mkl
  filelock           conda-forge/noarch::filelock-3.16.0-pyhd8ed1ab_0
  jinja2             conda-forge/noarch::jinja2-3.1.4-pyhd8ed1ab_0
  libuv              conda-forge/win-64::libuv-1.48.0-hcfcfb64_0
  markupsafe         conda-forge/win-64::markupsafe-2.1.5-py310ha8f682b_1
  mpmath             conda-forge/noarch::mpmath-1.3.0-pyhd8ed1ab_0
  networkx           conda-forge/noarch::networkx-3.3-pyhd8ed1ab_1
  pytorch            pytorch/win-64::pytorch-2.4.1-py3.10_cpu_0
  pytorch-mutex      pytorch/noarch::pytorch-mutex-1.0-cpu
  pyyaml             conda-forge/win-64::pyyaml-6.0.2-py310ha8f682b_1
  sympy              conda-forge/noarch::sympy-1.13.2-pyh04b8f61_3
  yaml               conda-forge/win-64::yaml-0.2.5-h8ffe710_2

The following packages will be DOWNGRADED:

  libblas                                3.9.0-23_win64_mkl --> 3.9.0-1_h8933c1f_netlib
  libcblas                               3.9.0-23_win64_mkl --> 3.9.0-5_hd5c7e75_netlib
  liblapack                              3.9.0-23_win64_mkl --> 3.9.0-5_hd5c7e75_netlib
  mkl                                 2024.1.0-h66d3029_694 --> 2023.1.0-h6a75c08_48682


Proceed ([y]/n)?


Downloading and Extracting Packages:


CondaHTTPError: HTTP 404 NOT FOUND for url <https://prefix.dev/pytorch/win-64/pytorch-2.4.1-py3.10_cpu_0.tar.bz2>
Elapsed: 00:00.690973

An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.

CondaHTTPError: HTTP 404 NOT FOUND for url <https://prefix.dev/pytorch/win-64/pytorch-2.4.1-py3.10_cpu_0.tar.bz2>
Elapsed: 00:00.690973

An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.

[wolfv]

Thanks! We are not allowed to mirror these packages due to redistribution limitations. However, we should just forward the request to the original conda.anaconda.org servers. We'll fix that, ETA thursday.

[jamesmyatt]

Have you contacted nvidia or the pytorch team to ask about mirroring the packages?

In the meantime, isn't it better to just delete these channels, rather than redirect the downloads?

I think that users will be confused if the requests are actually being served by anaconda.org when they were expecting it to come from prefix.dev. As you're aware, there may be reasons why some users need (or want) to avoid anything to do with Anaconda Inc (including anaconda.org).

[wolfv]

@jamesmyatt - yes, we are talking to NVidia about mirroring the packages. Hopefully we will be able to figure this out.

I totally know about the issues with Anaconda. However, the anaconda.org channels that we are redirecting to should be completely free and if that would change we would immediately stop doing that.

But yes, we are talking to NVidia and we can hopefully figure something out asap.

[jamesmyatt]

anaconda.org channels ... should be completely free

"Should" is doing a lot of heavy lifting here. Certainly free as in beer, but access is controlled by Anaconda and their commercial interests. Don't get me wrong. Anaconda have done and continue to do a great deal for our ecosystem. But it is a single-point-of-failure.

[wolfv]

Yes, I just sent an additional message to our contact at NVidia. We're trying to get this resolved as quickly as possible but we've been told that legal departments take some time.

We are also working on additional things to make things less dependend on anaconda: conda/ceps#91

[jamesmyatt]

Thank you

[navot-arad]

@wolfv you only mention being in contact with nvidia. Does the legal situation with nvidia impact both the nvidia and pytorch channels or is the limitation on redistributing the pytorch channel packages a separate issue?

[wolfv]

I haven't looked into the Pytorch channel but since it distributes open source software I imagine that there are fewer or no legal problems in distributing it.