withHoogle does not respect exactDeps for hoogle itself

[dten]

Describe the bug

we have a nix shell that looks very similar to the example
https://input-output-hk.github.io/haskell.nix/tutorials/development.html#how-to-get-a-development-shell

We use stackage for snapshots and currently target nightly-2024-08-08 which importantly contains a working combination of data-default and http-client-tls

https://www.stackage.org/nightly-2024-08-08/package/data-default-0.7.1.1
https://www.stackage.org/nightly-2024-08-08/package/http-client-tls-0.3.6.3

However if we enable withHoogle: true in the last few days we are unable to build this as it fails to compile with the combination of http-client-tls-0.3.6.4 and data-default-0.8.0 neither of which are in our snapshot.

It seems that cabal itself is deciding which dependency versions to use which I was expecting exactDeps to stop

Expected behavior

hoogle server should respect exactDeps or at least the snapshot

** Extra context **

This is the error you get currently when cabal picks the latest versions

trace: haskell-nix.haskellLib.cleanGit: /nix/store/c7dklpz9nx5ssy2xmvzrk23fy586hfj3-source does not seem to be a git repository,
assuming it is a clean checkout.
error: builder for '/nix/store/3cpvh9705kbb7r61cskfx6sqkb6rq4r5-http-client-tls-lib-http-client-tls-0.3.6.4.drv' failed with exit code 1;
       last 25 log lines:
       >     |
       > 192 |         settings = mkManagerSettingsContext' defaultManagerSettings (Just globalConnectionContext) def msocksHTTP msocksHTTPS
       >     |                                                                                                    ^^^
       >
       > Network/HTTP/Client/TLS.hs:208:81: error: [GHC-39999]
       >     • Could not deduce ‘Default NC.TLSSettings’
       >         arising from a use of ‘def’                                                                                                                                                         >       from the context: MonadIO m
       >         bound by the type signature for:
       >                    newTlsManagerWith :: forall (m :: * -> *).
       >                                         MonadIO m =>
       >                                         ManagerSettings -> m Manager
       >         at Network/HTTP/Client/TLS.hs:202:1-62
       >     • In the third argument of ‘mkManagerSettingsContext'’, namely
       >         ‘def’                                                                                                                                                                               >       In the expression:
       >         mkManagerSettingsContext'
       >           set (Just globalConnectionContext) def msocksHTTP msocksHTTPS
       >       In an equation for ‘settings’:
       >           settings
       >             = mkManagerSettingsContext'
       >                 set (Just globalConnectionContext) def msocksHTTP msocksHTTPS
       >     |
       > 208 |         settings = mkManagerSettingsContext' set (Just globalConnectionContext) def msocksHTTP msocksHTTPS
       >     |                                                                                 ^^^
       For full logs, run 'nix log /nix/store/3cpvh9705kbb7r61cskfx6sqkb6rq4r5-http-client-tls-lib-http-client-tls-0.3.6.4.drv'.
error: 1 dependencies of derivation '/nix/store/hs0jjj18rqk34g73d4vvh6l4877h69w7-http-conduit-lib-http-conduit-2.3.9.1.drv' failed to build
error: 1 dependencies of derivation '/nix/store/qx7kcyb8mxgjkx7aw3ls2krzc84ww0xh-hoogle-lib-hoogle-5.0.18.4.drv' failed to build
error: 1 dependencies of derivation '/nix/store/3fzw19s9wd4fd68b3aa8dx3j8n782vl1-hoogle-exe-hoogle-5.0.18.4.drv' failed to build
error: 1 dependencies of derivation '/nix/store/l2qrihinyardws1bfsq7l4wpjdmzfhi0-hoogle-with-packages.drv' failed to build
error: 1 dependencies of derivation '/nix/store/dsr7ij77hm6r61j07fdr4i5xayyrxckq-ghc-shell-for-packages-ghc-9.8.2-env.drv' failed to build
error: 1 dependencies of derivation '/nix/store/znl4v0r9x9553cybi6ig0f2w3a96lkas-ghc-shell-for-packages.drv' failed to build

[hamishmack]

exactDeps only applies to the dependencies of the project (not to the tools in the shell).

There is upstream issue in hackage that breaks hoogle. Some details are here:

snoyberg/http-client#547
snoyberg/http-client#546

To workaround this issue you can override the hoogle used by adding it to the shell tools. For example something like:

    tools = {
      cabal = "latest"; # Use latest version from hackage
      haskell-language-server.src = pkgs.haskell-nix.sources."hls-2.9"; # Build from GitHub pin in haskell.nix flake inputs
      hoogle.index-state = "2024-10-01T00:00:00Z"; # Workaround for https://github.com/input-output-hk/haskell.nix/issues/2277
    };

This will pin the index-state of hackage used to build hoogle. Pinning the index-state is the hackage equivalent of choosing a stackage resolver.

[dten]

thanks for the workaround tip

with cabal picking whatever it likes still though this makes general reproducability more troublesome with cached results differing from uncached. Is there any scope for this being able to be pinned like exactDeps in the future?

[hamishmack]

I don't think haskell.nix should be responsible for pinning tools like hoogle. Hopefully the current issue in hackage will be resolved quickly. There are a lot of tools people might have in their shell an hoogle is really just one. Keep in mind that each tool is built with its own plan so projects are free to pick whatever version you want.

Haskell.nix has a source pin for hls-2.9 because it currently (at least the last time I tried), does not build for any index-state. It's not ideal as it has to be updated each time there is a new release of HLS. I would like to avoid that for hoogle.

Haskell.nix projects can pin index-state as described or more globally by pinning hackage.nix in the flake.nix file:

  inputs = {
    hackage = {
      url = "github:input-output-hk/hackage.nix";
      flake = false;
    };

    haskell-nix = {
      url = "github:input-output-hk/haskell.nix";
      inputs.hackage.follows = "hackage";
    };

That should globally limit the index-state used by everything to the oldest index-state in the hackage.nix commit. That way nix flake update haskell-nix does not bring in any new (potentially broken) hackage stuff. If you are using nightly version of stackage though, then I dod not think this will work (IIRC stackage.nix references packages in hackage.nix).

Something else I should mention. index-state only limits the dependencies chosen. When picking the latest version of the tool to build we pick the version from hackage.nix. So to more fully pin the tool specify the version as well:

      hoogle.version = "5.0.18.4";
      hoogle.index-state = "2024-10-01T00:00:00Z";

Currently there is no way to choose tools from stackage, but a PR would be welcome. Then the above could be replaced with something like:

      hoogle.resolver = "lts-22.39";

[dten]

Thanks for the info

[hamishmack]

It looks like the underlying issue here is that hoogle is restricting crypton-connection==0.3.2. While it is still possible for cabal to find a plan for hoogle, it includes data-default-0.8 and data-default-class-0.1.2.2. That combination is not compatible, but it is chosen to avoid using an old version of the tls package.

If we add add constraints: data-default-class >=0.2.0.0, tls >=2 the planner fails with:

[__0] trying: hoogle-5.0.18.4 (user goal)
[__1] trying: warp-tls-3.4.7 (dependency of hoogle)
[__2] trying: tls-2.1.2 (dependency of warp-tls)
[__3] trying: data-default-class-0.2.0.0 (dependency of warp-tls)
[__4] next goal: crypton-connection (dependency of hoogle)
[__4] rejecting: crypton-connection-0.4.1 (conflict: hoogle =>
crypton-connection==0.3.2)
[__4] skipping: crypton-connection-0.4.0 (has the same characteristics that
caused the previous version to fail: excluded by constraint '==0.3.2' from
'hoogle')
[__4] rejecting: crypton-connection-0.3.2 (conflict: tls==2.1.2,
crypton-connection => tls>=1.7 && <2.1.0)
[__4] skipping: crypton-connection-0.3.1 (has the same characteristics that
caused the previous version to fail: excludes 'tls' version 2.1.2)
[__4] fail (backjumping, conflict set: crypton-connection, hoogle, tls)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: tls, data-default-class, hoogle,
crypton-connection, warp-tls
Try running with --minimize-conflict-set to improve the error message.

Adding allow-newer: hoogle:crypton-connection resolves this problem.

[hamishmack]

This has been fixed in hackage. #2279 removes the workaround from haskell.nix.