Haddock recompilation avoidance #9177

FinleyMcIlwaine · 2023-08-16T05:19:33Z

Haddock no longer writes compilation files by default, so we do not need to pass tmp dirs for -hidir, -stubdir, and -odir via --optghc. Indeed, we do not want to do so, since it results in recompilation for every invocation of Haddock via Cabal. This commit stops this from happening for Haddock versions 2.28 or greater (when Hi Haddock was introduced).

This commit also prevents the definition of the __HADDOCK_VERSION__ macro when invoking GHC through Haddock by default, which necessary to avoid recompilation during documentation generation, since the macro invalidates existing build results. Since a limited set of users may rely on this macro still, we introduce the --haddock-version-cpp flag and haddock-version-cpp: cabal.project field, which enable the definition of the __HADDOCK_VERSION__ macro when invoking GHC through Haddock.

This commit also renames the --haddock-lib flag to --haddock-resources-dir (and haddock-lib: cabal.project field to haddock-resources-dir:), and adds this flag to the users guide since it was missing an entry. This also allows us to add this field to cabal-install:test:integration-tests2, since it is no longer ambiguous with the --lib flag.

This commit also causes documentation: true or --enable-documentation to imply -haddock for GHC.

Also, since Haddock >= 2.29 is renaming --lib to --resources-dir, this commit switches the flag provided to Haddock using a backwards compatible condition based on the Haddock version.

QA Notes

Consider a Haskell package with dependencies. We should be able to simply configure the package so that doing cabal build followed by cabal haddock does not trigger a recompilation. Use the following deps.cabal file:

cabal-version: 3.6
name: deps
version: 0.1.0

library
  build-depends:
      base
    , acme-missiles
  exposed-modules: Lib

The following cabal.project file:

packages: .

package *
  documentation: True
  haddock-options: --optghc=-ddump-hi-diffs

And any Lib.hs will do:

-- | Lib reexports 'launchMissiles'
module Lib (myMissiles, launchMissiles) where

import Acme.Missiles

-- | Another 'launchMissiles'
myMissiles :: IO ()
myMissiles = launchMissiles

This configuration should avoid recompilation during documentation generation. To verify this, with a package such as above, just run cabal build. Since documentation is enabled, the package will first build then generate Haddocks. Since Haddock is passing -ddump-hi-diffs to GHC, you should see output indicating that no recompilation checks have triggered during the Haddock phase:

Module flags unchanged
Optimisation flags changed; ignoring
HPC flags unchanged
signatures to merge in unchanged
[]
implementing module unchanged
Checking interface for module Prelude base
Module fingerprint unchanged
Checking interface for module Acme.Missiles cm-mssls-0.3-9160cec8
Module fingerprint unchanged
Checking interface for module GHC.Types ghc-prim
Module fingerprint unchanged

This means recompilation was successfully avoided.

NOTE: I am working on making no-tmp-comp-dir the default behavior for Haddock 2.29, but until then it is necessary to avoid recompilation.

Patches conform to the coding conventions.
Any changes that could be relevant to users have been recorded in the changelog.
The documentation has been updated, if necessary.
Manual QA notes have been included.

Fixes #9175

dcoutts · 2023-08-21T10:33:41Z

Is it guaranteed that haddock will not write/overwrite any compilation files? Its ok for haddock to read files from the normal compilation output, but never ok to write them.

So how does this work? Does the cabal haddock command now run the compilation first (with its usual compilation avoidance)?

Haddock no longer writes compilation files by default

I'd feel happier if haddock had a flag that promises to never write compilation files, and instead fail if it would need to do so. This would give us a "belt & braces" solution, in case something gets messed up in future. Otherwise we could end up with some hard-to-debug problems.

FinleyMcIlwaine · 2023-08-21T17:25:26Z

Is it guaranteed that haddock will not write/overwrite any compilation files?

I do think this is guaranteed (for Haddock versions >= 2.28.0, which is when "Hi Haddock" was introduced). Haddock always uses GHC's -fno-code backend (unless the user explicitly enables a different GHC backend), which will "omit code generation (and all later phases) altogether". It does generate code for dependencies of modules containing Template Haskell splices, but as far as I can tell those compilation results are put in a temporary directory by GHC and do not overwrite anything.

Does the cabal haddock command now run the compilation first (with its usual compilation avoidance)?

This PR doesn't add or remove any calls to tools for the cabal haddock command. In that regard, the cabal haddock command still works just as it did before, meaning it will

make sure installed dependency packages are up to date (in the same way the cabal build command does), and
run haddock on the local package.

The important bit is that with "Hi Haddock", Haddock generates documentation from module interfaces that it gets from the GHC API. The GHC API will do the normal compilation avoidance things while retrieving those interfaces.

The problem that this PR addresses is that cabal was passing temporary -hidir directories to GHC through Haddock, meaning Haddock's calls to GHC would always result in recompilation. This patch just stops those temporary directories from being passed.

As an aside: Haddock has been running its GHC session in a temporary directory by default since version 2.23.0, shipped with GHC 8.8, until version 2.29 (pending this change). So, as long as this cabal patch is not used in combination with a GHC 8.6.5 bundled Haddock (version 2.22 or lower), this change should be backwards compatible and avoid overwriting any existing compilation files.

I'd feel happier if haddock had a flag that promises to never write compilation files, and instead fail if it would need to do so.

Is the -fno-code usage enough of a guarantee here? As long as -fno-code does not write compilation files, then Haddock won't either.

michaelpj · 2023-08-22T09:24:47Z

It would be really nice if oblivious users could benefit from this change. As far as I can tell, the main reason why that won't be true is haddock-no-version-cpp. In order for user to benefit by default, we'd have to set that option to false by default. I realise that's a breaking change, but if we don't do this I predict that only a small fraction of users will actually benefit from this change, which seems a shame given that the flag is apparently very rarely necessary.

Personally, I would be willing to even just deprecate __HADDOCK_VERSION__ if it means we get reliable recompilation avoidance for all users. 19 Hackage packages! Maybe we can just purge it from them.

(I took a quick look at some of the search results. It seems like most of the usage is "hide this bit of documentation from Haddock because Haddock has a bug". It seems to me that the version of GHC could often be used as a proxy for the problematic Haddock version in those cases.)

phadej · 2023-08-22T14:59:59Z

This PR doesn't add or remove any calls to tools for the cabal haddock command.

And

so we do not need to pass tmp dirs for -hidir, -stubdir, and -odir via --optghc.

are slightly contradictory. You don't remove any calls, but you do change them, as far as I can see, unconditionally.

For old GHCs & haddock combinations I would expect no changes in behaviour. New Cabal library versions are used
to work with old GHCs & haddocks.

In other words, I don't see any version checks. There ought to be.

FinleyMcIlwaine · 2023-08-22T23:23:34Z

Personally, I would be willing to even just deprecate __HADDOCK_VERSION__ if it means we get reliable recompilation avoidance for all users.

What would the deprecation strategy look like for this? Could I just change haddock-no-version-cpp to haddock-version-cpp and make it default to false?

FinleyMcIlwaine · 2023-08-22T23:26:44Z

I don't see any version checks

I will add a version check that only prevents the tmp dirs being passed for Haddock >= 2.28 (when hi haddock was introduced).

andreabedini · 2023-08-24T12:03:12Z

What would the deprecation strategy look like for this? Could I just change haddock-no-version-cpp to haddock-version-cpp and make it default to false?

Putting on my cowboy hat, I'd say I am ok with this, this is one of the behaviours I want to see changing.

Setting haddock-version-cpp: true gives users a very easy way out. They had the resources to update to the latest version of cabal, they will have the resouces to add that line to cabal.project ☺️ 🤠

FinleyMcIlwaine · 2023-08-24T16:01:18Z

Putting on my cowboy hat, I'd say I am ok with this

I agree 🤠

I've changed this to avoid defining the __HADDOCK_VERSION__ macro by default and introduced the --haddock-version-cpp flag and haddock-version-cpp: cabal.project field, defaulting to False. I updated the PR description QA notes and changelog entry to reflect this change.

phadej · 2023-08-24T16:37:37Z

I kindly ask you to think a bit. __HADDOCK_VERSION__ was there mostly to indicate that the compiler is "haddock". No-one actually checks the actual version.

As far as I can tell the functionality has only been used to circumvent bugs in haddock: https://hackage-search.serokell.io/?q=__HADDOCK_VERSION__

base:

#if !defined(__HADDOCK_VERSION__)
    -- workaround https://github.com/haskell/haddock/issues/680

generics-sop:

#if !(defined(__HADDOCK_VERSION__)) || MIN_TOOL_VERSION_haddock(2,14,0)
  , Proxy(..) -- hidden from old Haddock versions, because it triggers an internal error
#endif

(Note how the tool version is checked, HADDOCK_VERSION is used for whether haddock runs. MIN_TOOL_VERSION_haddock macro is always defined! so the version in HADDOCK_VERSION is actually now redundant.

streamly:

#ifdef __HADDOCK_VERSION__
#undef INSPECTION
#endif
#ifdef INSPECTION
{-# LANGUAGE TemplateHaskell #-}
{-# OPTIONS_GHC -fplugin Test.Inspection.Plugin #-}
#endif

This case is also important to think through: here it's essential to know whether package is compiled by haddock, as it simply won't work. So having haddock compile the source and not setting __HADDOCK_VERSION__ will break this.

Either the way haddock-uses-ghc-lib is buggy, or how haddock processed the result data is buggy. The latter is quite likely to exist also in the future and people may want to change the code for haddock to succeed also with hi, haddock. Do they?

In that light, I don't really see a real need for any configuration. You can be conservative and have haddock-recompile which would always recompile for haddock (not "this will likely trigger recompilation during documentation generation.", but always).

There could be a way to distinguish whether the source is processed by haddock, but haddock-version-cpp is a bit indirect. Also it's documentation says:

This will likely trigger recompilation during documentation generation.

where for bug circumention it should be assertive as:

This will trigger recompilation during documentation generation.

I'd like haddock-recompiles or haddock-separate-compilation like of clear cut option.

And the haddock-recompiles: False case (when supported) should in fact tell haddock+ghc to not try to compile anything. That's the flag @dcoutts asks for. I agree, you should be explicit about that.

There is few examples where people are lazy and add additional imports with __HADDOCK_VERSION__ to make haddock link stuff in docs, but that is just lazyness, they should use qualified names: a bit ugly, but correct way of doing that.

michaelpj · 2023-08-24T17:31:58Z

haddock-version-cpp: cabal.project field, defaulting to False

I note that this can't be project-level configuration. Packages that rely on this may exist on Hackage - how are you supposed to know how to haddock them? How is Hackage supposed to know how to haddock them? If we were going to make it configurable, it seems to me that it would need to be a .cabal file field.

This case is also important to think through: here it's essential to know whether package is compiled by haddock, as it simply won't work.

This case is indeed interesting, because it's about compiler plugins. Perhaps I'm being naive, but I would hope that the upshot of HI Haddock is that most bugs such as this go away: haddock doesn't need to know how to handle compiler plugins, because GHC has already dealt with it.

So hopefully there will be less need to work around Haddock in the future. But also suppose that we made it impossible to ever detect if Haddock is running. That would lose us the ability to work around some Haddock bugs... which would mean that in some edge cases (19! in all of Hackage!) some things might not be able to be documented. Perhaps that's an acceptable cost.

phadej · 2023-08-24T18:28:56Z

19! in all of Hackage!)

It's 19 occurences of __HADDOCK_VERSION__.

Cabal: matches the code to set the flag. false positive.
Cabal-ide-backend: ditto.
base: working around bug Haddock doesn't properly handle stripped classes in .hs-boot files haddock#680 which is closed, so probably can be removed (if not already_
concurrent-extra: imports for links. invalid. (instead of 'mask' use 'Control.Exception.mask' in haddock comments)
contrainers: imports for links
dimensions: unclear
dlist: imports
easytensor: unclear (same author as dimensions)
generics-sop: bug with older haddock
glambda: imports
hackport: Cabal copy
haskell-gi: unclear (code generation)
ngx-export-log: see below. not an issue with hi haddock
paths: import
sop-core: bug with older haddock
streamly: not an issue with hi haddock. detection of "ghci" like mode.
strict-containers: imports
unbound-delays: imports
usb: imports. This one is tricky, as the imports come from potentially different modules.

So:

8 of 19 is imports (almost half!)
3 is Cabal copies
3 fixed bugs
2 ghci-like mode detections
3 other!?
- two (dimensions and easytensor) of which are the same (something weird happening)
- and haskell-gi is code generation, I suspect that ENABLE_OVERLOADING is disabled to actually change documentation to not be littered with overloads. I'm not sure that it's great idea though.

I'd say the cost is very low.

haddock has been buggy, but I don't know how much my (past) bad experience with it
is due it trying to be GHC versus the other bugs.

ngc-export-log:

-- Some tools such as hls, haddock, and ghci run interactive linking against C
-- functions plugin_ngx_http_haskell_log() and plugin_ngx_http_haskell_log_r()
-- when loading Log.hs. In Log.hs, TH declarations from Log/Gen.hs which make
-- calls to those C functions, get instantiated. Obviously, linking fails as
-- soon as we don't have a library to expose the functions because such a
-- library is built by Nginx and we don't want to use Nginx at this step.
--
-- In this workaround, the C functions get replaced by stubs when running by
-- hls or haddock. This prevents interactive linking in Log.hs. It's easy to
-- detect that the code is being run by hls or haddock: the tools define their
-- own C macro declarations __GHCIDE__ and __HADDOCK_VERSION__ respectively. To
-- prevent interactive linking in ghci, pass one of the two macro declarations
-- in an appropriate option, e.g.
--
-- cabal repl --ghc-options=-D__GHCIDE__ --repl-options=-fobject-code

domenkozar · 2024-03-19T06:47:37Z

Hitting memory limits due to haddock, this can't come soon enough.

Kleidukos · 2024-03-28T18:14:04Z

@FinleyMcIlwaine Where are we standing with this PR? Anything you need?

changelog.d/pr-9177

Kleidukos · 2024-04-30T14:05:55Z

I am told the work will be resumed by someone else at Well-Typed, no need to harass Finley any further on Cabal's side. :)

Haddock no longer writes compilation files by default, so we do not need to pass tmp dirs for `-hidir`, `-stubdir`, and `-odir` via `--optghc`. Indeed, we do not *want* to do so, since it results in recompilation for every invocation of Haddock via Cabal. This commit stops this from happening for haddock versions >= 2.28 (when Hi Haddock was introduced). This commit also stops the default definition of the `__HADDOCK_VERSION__` macro when invoking GHC through haddock. Since a very limited set of users may still depend on this macro, we introduce the `--haddock-version-cpp` flag and `haddock-version-cpp:` cabal.project field, which enable the definition of the `__HADDOCK_VERSION__` macro when invoking GHC through Haddock. This will almost guarantee recompilation during documentation generation due to the macro definition. This commit also renames the `--haddock-lib` flag to `--haddock-resources-dir` (and `haddock-lib:` cabal.project field to `haddock-resources-dir:`), and adds this flag to the users guide since it was missing an entry. This also allows us to add this field to `cabal-install:test:integration-tests2`, since it is no longer ambiguous with the `--lib` flag. This commit also causes `documentation: true` or `--enable-documentation` to imply `-haddock` for GHC. Also, since Haddock >= 2.29 is renaming `--lib` to `--resources-dir`, this commit switches the flag provided to Haddock using a backwards compatible condition based on the Haddock version. Adds a changelog entry.

We no longer define the `__HADDOCK_VERSION__` macro when invoking GHC through Haddock, since doing so essentially guarantees recompilation during documentation generation. We audited all uses of `__HADDOCK_VERSION__` in hackage, ensuring there was a reasonable path forward to migrate away from using `__HADDOCK_VERSION__` for each, while generating the same documentation as it did before. If you are a user of `__HADDOCK_VERSION__`, please take a look at the discussion in haskell#9177 and reach out to us if your use case is not covered. Reverts the version-cpp flag introduced in the previous commit to avoid a workaround. Instead, we get rid of the problem (`__HADDOCK_VERSION__`) at its root. See the discussion in haskell#9177

With "Hi Haddock", we want to re-use GHC's compilation output (in particular, the generated interface files) to generate documentation from. However, haddock by default will use a temporary location to build sources, and will therefore fail to find the interface files produced by GHC. To avoid recompilation, we need to instruct Cabal to pass --no-tmp-comp-dir to Haddock so as to not use a temporary directory and thus re-use existing interface files.

In the last commits we started re-using GHC's interface files and objects in haddock in order to avoid recompilation. However, if haddock is run with different flags than GHC (say, for example, `haddock-options: -DSomethingCustom`), it will recompile the interfaces and objects. This commit introduces a guardrail to the process of re-using GHC's compilation files: instead of running haddock directly on the directories where GHC placed its output, copy the directory contents to a temporary directory and point haddock to the objects and interfaces there. Even if recompilation is triggered by haddock, the objects produced by GHC will be left untouched.

jgotoh · 2024-05-31T11:22:16Z

Manual QA was successful on WSL (Ubuntu 22.04, Windows 11).

Here is the log for anyone interested:

Reproduce Bug with cabal latest (3.10.3.0)

Resolving dependencies...
Build profile: -w ghc-9.8.2 -O1
In order, the following will be built (use -v for more details):

acme-missiles-0.3 (lib) (requires download & build)
deps-0.1.0 (lib) (first run)
Downloading acme-missiles-0.3
Downloaded acme-missiles-0.3
Starting acme-missiles-0.3 (lib)
Building acme-missiles-0.3 (lib)
Haddock acme-missiles-0.3 (lib)
Installing acme-missiles-0.3 (lib)
Completed acme-missiles-0.3 (lib)
Configuring library for deps-0.1.0..
Preprocessing library for deps-0.1.0..
Building library for deps-0.1.0..
[1 of 1] Compiling Lib ( Lib.hs, /home/.../dist-newstyle/build/x86_64-linux/ghc-9.8.2/deps-0.1.0/build/Lib.o, /home/.../dist-newstyle/build/x86_64-linux/ghc-9.8.2/deps-0.1.0/build/Lib.dyn_o )
Preprocessing library for deps-0.1.0..
Running Haddock on library for deps-0.1.0..
Old interface file was invalid:
Exception when reading interface file /tmp/.haddock-283725/Lib.hi
/tmp/.haddock-283725/Lib.hi: withBinaryFile: does not exist (No such file or directory)
[1 of 1] Compiling Lib ( Lib.hs, nothing )
Haddock coverage:
100% ( 3 / 3) in 'Lib'
Documentation created:
/home/julian/.../dist-newstyle/build/x86_64-linux/ghc-9.8.2/deps-0.1.0/doc/html/deps/

^ lib was compiled twice.

Reproduce Fix with cabal head

Resolving dependencies...
Build profile: -w ghc-9.8.2 -O1
In order, the following will be built (use -v for more details):

deps-0.1.0 (lib) (first run)
Warning: this is a debug build of cabal-install with assertions enabled.
Configuring library for deps-0.1.0...
Warning: this is a debug build of cabal-install with assertions enabled.
Preprocessing library for deps-0.1.0...
Building library for deps-0.1.0...
[1 of 1] Compiling Lib ( Lib.hs, dist-newstyle/build/x86_64-linux/ghc-9.8.2/deps-0.1.0/build/Lib.o, dist-newstyle/build/x86_64-linux/ghc-9.8.2/deps-0.1.0/build/Lib.dyn_o )
Warning: this is a debug build of cabal-install with assertions enabled.
Preprocessing library for deps-0.1.0...
Running Haddock on library for deps-0.1.0...
Considering whether compilation is required for Lib:
Module flags unchanged
Optimisation flags changed; ignoring
HPC flags unchanged
signatures to merge in unchanged
[]
implementing module unchanged
Checking interface for module Acme.Missiles acme-missiles-0.3-ae693866e57964b306e10d08e46ec57afb464eb0ea11a3e9cd8b2d4d84af4b75
Module fingerprint unchanged
Checking interface for module Prelude base
Module fingerprint unchanged
Checking interface for module GHC.Types ghc-prim
Module fingerprint unchanged
Haddock coverage:
100% ( 3 / 3) in 'Lib'
Documentation created:
/home/..././dist-newstyle/build/x86_64-linux/ghc-9.8.2/deps-0.1.0/doc/html/deps/
Warning: this is a debug build of cabal-install with assertions enabled.

Lib.hs was compiled once ✅

jasagredo · 2024-05-31T11:30:10Z

Unless I am doing something wrong, QA does not pass on Windows/MSYS2, it seems Lib is compiled twice:

➜ $(cygpath -m "C:\\Users\\Javier\\code\\cabal\\dist-newstyle\\build\\x86_64-windows\\ghc-9.8.2\\cabal-install-3.13.0.0\\x\\cabal\\build\\cabal\\cabal.exe") build
Warning: this is a debug build of cabal-install with assertions enabled.
Resolving dependencies...
Build profile: -w ghc-9.8.2 -O1
In order, the following will be built (use -v for more details):
 - deps-0.1.0 (lib) (first run)
Warning: this is a debug build of cabal-install with assertions enabled.
Configuring library for deps-0.1.0...
Warning: this is a debug build of cabal-install with assertions enabled.
Preprocessing library for deps-0.1.0...
Building library for deps-0.1.0...
[1 of 1] Compiling Lib              ( Lib.hs, dist-newstyle\build\x86_64-windows\ghc-9.8.2\deps-0.1.0\build\Lib.o )
Warning: this is a debug build of cabal-install with assertions enabled.
Preprocessing library for deps-0.1.0...
Running Haddock on library for deps-0.1.0...
Warning: The documentation for the following packages are not installed. No
links will be generated to these packages: array-0.5.6.0, base-4.19.1.0,
ghc-bignum-1.3, ghc-prim-0.11.0, stm-2.5.2.1
Considering whether compilation is required for Lib:
  Module flags have changed 723e9f5ce95d715375df6f4102da4501 -> 301ebdaa80354c8d4dc501aa439e15ea
[1 of 1] Compiling Lib              ( Lib.hs, nothing ) [Flags changed]
Haddock coverage:
 100% (  3 /  3) in 'Lib'
Warning: Lib: could not find link destinations for:
        - GHC.Types.IO
Documentation created:
C:\Users\Javier\code\bb\.\dist-newstyle\build\x86_64-windows\ghc-9.8.2\deps-0.1.0\doc\html\deps\
Warning: this is a debug build of cabal-install with assertions enabled.

jasagredo · 2024-06-03T11:38:27Z

I don't see an obvious difference in the flags passed to GHC, more than the call to ghc compilation uses single backslashes and the --optghc args given to haddock have double backslashes:

❯ diff ghcargs optghc-haddockargs
4c4
< dist-newstyle\build\x86_64-windows\ghc-9.8.2\deps-0.1.0\build
---
> dist-newstyle\\build\\x86_64-windows\\ghc-9.8.2\\deps-0.1.0\\build
6c6
< dist-newstyle\build\x86_64-windows\ghc-9.8.2\deps-0.1.0\build
---
> dist-newstyle\\build\\x86_64-windows\\ghc-9.8.2\\deps-0.1.0\\haddock-objs-10336
8c8
< dist-newstyle\build\x86_64-windows\ghc-9.8.2\deps-0.1.0\build
---
> dist-newstyle\\build\\x86_64-windows\\ghc-9.8.2\\deps-0.1.0\\haddock-his-10336
10c10
< dist-newstyle\build\x86_64-windows\ghc-9.8.2\deps-0.1.0\build\extra-compilation-artifacts\hie
---
> dist-newstyle\\build\\x86_64-windows\\ghc-9.8.2\\deps-0.1.0\\build\\extra-compilation-artifacts\\hie
12c12
< dist-newstyle\build\x86_64-windows\ghc-9.8.2\deps-0.1.0\build
---
> dist-newstyle\\build\\x86_64-windows\\ghc-9.8.2\\deps-0.1.0\\build
15,21c15,21
< -idist-newstyle\build\x86_64-windows\ghc-9.8.2\deps-0.1.0\build
< -idist-newstyle\build\x86_64-windows\ghc-9.8.2\deps-0.1.0\build\autogen
< -idist-newstyle\build\x86_64-windows\ghc-9.8.2\deps-0.1.0\build\global-autogen
< -Idist-newstyle\build\x86_64-windows\ghc-9.8.2\deps-0.1.0\build\autogen
< -Idist-newstyle\build\x86_64-windows\ghc-9.8.2\deps-0.1.0\build\global-autogen
< -Idist-newstyle\build\x86_64-windows\ghc-9.8.2\deps-0.1.0\build
< -IC:\msys64\ucrt64\include
---
> -idist-newstyle\\build\\x86_64-windows\\ghc-9.8.2\\deps-0.1.0\\build
> -idist-newstyle\\build\\x86_64-windows\\ghc-9.8.2\\deps-0.1.0\\build\\autogen
> -idist-newstyle\\build\\x86_64-windows\\ghc-9.8.2\\deps-0.1.0\\build\\global-autogen
> -Idist-newstyle\\build\\x86_64-windows\\ghc-9.8.2\\deps-0.1.0\\build\\autogen
> -Idist-newstyle\\build\\x86_64-windows\\ghc-9.8.2\\deps-0.1.0\\build\\global-autogen
> -Idist-newstyle\\build\\x86_64-windows\\ghc-9.8.2\\deps-0.1.0\\build
> -IC:\\msys64\\ucrt64\\include
23c23
< -optPdist-newstyle\build\x86_64-windows\ghc-9.8.2\deps-0.1.0\build\autogen\cabal_macros.h
---
> -optPdist-newstyle\\build\\x86_64-windows\\ghc-9.8.2\\deps-0.1.0\\build\\autogen\\cabal_macros.h
30c30
< C:\Users\Javier\AppData\Local\cabal\store\ghc-9.8.2-7294\package.db
---
> C:\\Users\\Javier\\AppData\\Local\\cabal\\store\\ghc-9.8.2-7294\\package.db
32c32
< C:\Users\Javier\code\bb\dist-newstyle\packagedb\ghc-9.8.2
---
> C:\\Users\\Javier\\code\\bb\\dist-newstyle\\packagedb\\ghc-9.8.2
34c34
< C:\Users\Javier\code\bb\.\dist-newstyle\build\x86_64-windows\ghc-9.8.2\deps-0.1.0\package.conf.inplace
---
> C:\\Users\\Javier\\code\\bb\\.\\dist-newstyle\\build\\x86_64-windows\\ghc-9.8.2\\deps-0.1.0\\package.conf.inplace

However computing the flag fingerprint uses normalise on those paths, so it should be fine?

Mikolaj · 2024-06-06T17:42:09Z

Regarding a backport to 3.12, this PR seems to break the 3.12 Cabal the library API, so it seems too late. I'd vote for 3.14.

* CI: install changelog-d from bindist (#10048) This will avoid build problems when the GHC in the CI environment is updated sooner than expected. Previous breakage: #9177 (comment) (cherry picked from commit d1a6ced) # Conflicts: # .github/workflows/changelogs.yml * !fixup resolve conflicts --------- Co-authored-by: Francesco Gazzetta <[email protected]> Co-authored-by: Artem Pelenitsyn <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

jasagredo · 2024-06-08T15:37:25Z

It works with (a locally patched to trace the module flags) GHC 9.11 (HEAD as of today):

➜ $CABAL build --with-compiler /c/Users/Javier/code/ghc/_build/stage0/bin/ghc                
Warning: this is a debug build of cabal-install with assertions enabled.
Warning: Parsing the index cache failed (Data.Binary.Get.runGet at position
16: Non-matching structured hashes: f46da61e7afa58a5e8fd1d2b6fb79899;
expected: d81bdd513f41b5d7ee4cd28455adadbe). Trying to regenerate the index
cache...
Resolving dependencies...
Build profile: -w ghc-9.11.20240607 -O1
In order, the following will be built (use -v for more details):
 - acme-missiles-0.3 (lib) (requires build)
 - deps-0.1.0 (lib) (first run)
Starting     acme-missiles-0.3 (lib)
Building     acme-missiles-0.3 (lib)
Haddock      acme-missiles-0.3 (lib)
Installing   acme-missiles-0.3 (lib)
Completed    acme-missiles-0.3 (lib)
Warning: this is a debug build of cabal-install with assertions enabled.
Configuring library for deps-0.1.0...
Warning: this is a debug build of cabal-install with assertions enabled.
Preprocessing library for deps-0.1.0...
Building library for deps-0.1.0...
[1 of 1] Compiling Lib              ( Lib.hs, dist-newstyle\build\x86_64-windows\ghc-9.11.20240607\deps-0.1.0\build\Lib.o )
MOD FLAGS: mainis:          Nothing
MOD FLAGS: safeHS:          TrustInfo None
MOD FLAGS: lang:            (Just 0,[5,7,23,44,98,99,101,120,122,125,130])
MOD FLAGS: cpp includes:
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240607\deps-0.1.0\build
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240607\deps-0.1.0\build
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240607\deps-0.1.0\build\autogen
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240607\deps-0.1.0\build\global-autogen
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240607\deps-0.1.0\build
  C:\msys64\ucrt64\include
MOD FLAGS: pic_p_opts:      []
MOD FLAGS: optP sig:        ([],19b7ece0dd17ba28e7ce435ed74624da)
MOD FLAGS: paths:           ["hc"]
MOD FLAGS: prof:            0
MOD FLAGS: ticky:           [False,False,False,False,False]
MOD FLAGS: debugLevel:      0
MOD FLAGS: callerCc empty:  True
MOD FLAGS: optP flags:      ["dist-newstyle\\build\\x86_64-windows\\ghc-9.11.20240607\\deps-0.1.0\\build\\autogen\\cabal_macros.h","-include"]
OPT FLAGS: opt_flags:
  Opt_CallArity
  Opt_Strictness
  Opt_FullLaziness
  Opt_FloatIn
  Opt_Specialise
  Opt_CrossModuleSpecialise
  Opt_CSE
  Opt_StgCSE
  Opt_DoLambdaEtaExpansion
  Opt_IgnoreAsserts
  Opt_DoEtaReduction
  Opt_CaseMerge
  Opt_CaseFolding
  Opt_UnboxSmallStrictFields
  Opt_EnableRewriteRules
  Opt_CmmSink
  Opt_CmmElimCommonBlocks
  Opt_Loopification
  Opt_CfgBlocklayout
  Opt_CprAnal
  Opt_WorkerWrapper
  Opt_SolveConstantDicts
Warning: this is a debug build of cabal-install with assertions enabled.
Preprocessing library for deps-0.1.0...
Running Haddock on library for deps-0.1.0...
Warning: The documentation for the following packages are not installed. No
links will be generated to these packages: array-0.5.6.0, base-4.20.0.0,
ghc-bignum-1.3, ghc-internal-9.1001.0, ghc-prim-0.11.0, stm-2.5.3.0
Considering whether compilation is required for Lib:
MOD FLAGS: mainis:          Nothing
MOD FLAGS: safeHS:          TrustInfo None
MOD FLAGS: lang:            (Just 0,[5,7,23,44,98,99,101,120,122,125,130])
MOD FLAGS: cpp includes:
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240607\deps-0.1.0\build
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240607\deps-0.1.0\build
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240607\deps-0.1.0\build\autogen
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240607\deps-0.1.0\build\global-autogen
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240607\deps-0.1.0\build
  C:\msys64\ucrt64\include
MOD FLAGS: pic_p_opts:      []
MOD FLAGS: optP sig:        ([],19b7ece0dd17ba28e7ce435ed74624da)
MOD FLAGS: paths:           ["hc"]
MOD FLAGS: prof:            0
MOD FLAGS: ticky:           [False,False,False,False,False]
MOD FLAGS: debugLevel:      0
MOD FLAGS: callerCc empty:  True
MOD FLAGS: optP flags:      ["dist-newstyle\\build\\x86_64-windows\\ghc-9.11.20240607\\deps-0.1.0\\build\\autogen\\cabal_macros.h","-include"]
Module flags unchanged
OPT FLAGS: opt_flags:
  Opt_DoLambdaEtaExpansion
  Opt_DoEtaReduction
Optimisation flags changed; ignoring
HPC flags unchanged
signatures to merge in unchanged
[]
implementing module unchanged
Checking interface for module Acme.Missiles acme-missiles-0.3-9d2bfaea19d60d250e0d2dd16d3ccb0608c816c1
Module fingerprint unchanged
Checking interface for module Prelude base
Module fingerprint unchanged
Checking interface for module GHC.Types ghc-prim
Module fingerprint unchanged
Haddock coverage:
 100% (  3 /  3) in 'Lib'
Warning: Lib: could not find link destinations for: 
        - GHC.Types.IO
Documentation created:
C:\Users\Javier\code\bb\.\dist-newstyle\build\x86_64-windows\ghc-9.11.20240607\deps-0.1.0\doc\html\deps\
Warning: this is a debug build of cabal-install with assertions enabled.

Mikolaj · 2024-06-08T17:36:54Z

@jasagredo: so it's good news that it works, but bad news that it didn't help us pinpoint the problem when run with GHC 9.8.2? How could, e.g., @FinleyMcIlwaine or @alt-romes help you diagnose further?

jasagredo · 2024-06-08T18:29:06Z

I will try tomorrow with ghc 9.8. @FinleyMcIlwaine provided me with the tracing patch today at Zurihac.

Mikolaj · 2024-06-09T11:57:31Z

Might this be related by chance? https://gitlab.haskell.org/ghc/ghc/-/issues/24964

jasagredo · 2024-06-09T20:40:56Z

The library had no foreign symbols 🤔. It turns out to be even more puzzling than anticipated.

So a locally built GHC 9.8.2 does not show the bug. The bindist 9.8.2 and 9.6.5 GHCs do show the bug, which makes no sense, even more because in Windows it will use an in-tree clang and libs, so it shouldn't depend on the environment in which it is being built.

I looked at this with Ben too, with no luck. His suggestion was to build GHC again tomorrow with --flavour=release which would be the same as the distributed bindist, but if that shows a difference then it will probably be worrying 😄

jasagredo · 2024-06-10T10:38:38Z

GHC 9.8.2 built with hadrian/build.bat --flavour=release says "Module flags unchanged". Now compiling GHC 9.8.2 with hadrian/build.bat --flavour=release --hash-unit-ids and praying 🙏

Mikolaj · 2024-06-13T14:52:52Z

@jasagredo: Was the answer to the prayers to your liking?

michaelpj · 2024-06-13T16:06:29Z

@alt-romes and I found some sad issues with this at Zurihac. Notably:

Turning on documentation: true will still re-build all your dependencies, despite the previously built interface files containing all the needed information to produce the Haddock. This rebuild will also force a rebuild of your local modules, since the hashes of the dependencies will have changed!

So in practice this only benefits you for changes to your local files, which is a shame. It seems like it gives us the tools to avoid rebuilding dependencies (and your local modules!) when turning on documentation, but we don't currently do things in the right way for that to happen.

(Probably this should be in a new issue...)

phadej · 2024-06-13T16:21:25Z

documentation: true issue is discussed in #8725

While good intention, in practice it doesn't work, and you cannot even override it.

What is worse, if someone doesn't want to enable documentation: True, there seems to be no way to permanently disable this behavior.

phadej · 2024-06-13T16:23:40Z

Duncan alludes:

Perhaps a nice solution (though not an easy solution to implement) would be if we could make the haddocks independently installable: so that building the docs for a dependency did not imply rebuilding the code. For example, similar to how we handle building profiling these days.

With haddocks built from ordinary interface files, that should be more easily doable (but still not easy, as it's new codepaths)

michaelpj · 2024-06-13T16:27:50Z

Yes, Rodrigo and I discussed that as a possibility, I think that would be the most graceful solution.

jasagredo · 2024-06-13T20:27:12Z

@Mikolaj:

@jasagredo: Was the answer to the prayers to your liking?

It was not unfortunately. So any GHC I build locally, even using the same config options as the CI, says "Module flags unchanged".

Every GHC distributed and a GHC I downloaded from a very recent pipeline on GHC's gitlab says "Module flags changed".

I think I will try to apply @FinleyMcIlwaine's patch to GHC-head and get a binary with it from CI, as it seems to be the only building environment that can reproduce this behavior.

Such a mysterious issue.

michaelpj · 2024-06-14T09:31:02Z

I made a new issue for the recompilation issue, since I think it's distinct from the question of whether documentation: True is enabled by default: #10111

jasagredo · 2024-06-14T09:47:33Z

@FinleyMcIlwaine I got a CI GHC with your patch, and bingo!

➜ $CABAL build
Warning: this is a debug build of cabal-install with assertions enabled.
Resolving dependencies...
Build profile: -w ghc-9.11.20240614 -O1
In order, the following will be built (use -v for more details):
 - acme-missiles-0.3 (lib) (requires build)
 - deps-0.1.0 (lib) (first run)
Starting     acme-missiles-0.3 (lib)
Building     acme-missiles-0.3 (lib)
Haddock      acme-missiles-0.3 (lib)
Installing   acme-missiles-0.3 (lib)
Completed    acme-missiles-0.3 (lib)
Warning: this is a debug build of cabal-install with assertions enabled.
Configuring library for deps-0.1.0...
Warning: this is a debug build of cabal-install with assertions enabled.
Preprocessing library for deps-0.1.0...
Building library for deps-0.1.0...
[1 of 1] Compiling Lib              ( Lib.hs, dist-newstyle\build\x86_64-windows\ghc-9.11.20240614\deps-0.1.0\build\Lib.o )
MOD FLAGS: mainis:          Nothing
MOD FLAGS: safeHS:          TrustInfo None
MOD FLAGS: lang:            (Just 0,[5,7,23,44,98,99,101,120,122,125,130])
MOD FLAGS: cpp includes:
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240614\deps-0.1.0\build
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240614\deps-0.1.0\build
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240614\deps-0.1.0\build\autogen
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240614\deps-0.1.0\build\global-autogen
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240614\deps-0.1.0\build
  C:\msys64\ucrt64\include
MOD FLAGS: pic_p_opts:      []
MOD FLAGS: optP sig:        ([],8383540998520b6962a51feacf466dc3)
MOD FLAGS: paths:           ["hc"]
MOD FLAGS: prof:            0
MOD FLAGS: ticky:           [False,False,False,False,False]
MOD FLAGS: debugLevel:      0
MOD FLAGS: callerCc empty:  True
MOD FLAGS: optP flags:      ["dist-newstyle\\\\build\\\\x86_64-windows\\\\ghc-9.11.20240614\\\\deps-0.1.0\\\\build\\\\autogen\\\\cabal_macros.h","-include"]
OPT FLAGS: opt_flags:
  Opt_CallArity
  Opt_Strictness
  Opt_FullLaziness
  Opt_FloatIn
  Opt_Specialise
  Opt_CrossModuleSpecialise
  Opt_CSE
  Opt_StgCSE
  Opt_DoLambdaEtaExpansion
  Opt_IgnoreAsserts
  Opt_DoEtaReduction
  Opt_CaseMerge
  Opt_CaseFolding
  Opt_UnboxSmallStrictFields
  Opt_EnableRewriteRules
  Opt_CmmSink
  Opt_CmmElimCommonBlocks
  Opt_Loopification
  Opt_CfgBlocklayout
  Opt_CprAnal
  Opt_WorkerWrapper
  Opt_SolveConstantDicts
Warning: this is a debug build of cabal-install with assertions enabled.
Preprocessing library for deps-0.1.0...
Running Haddock on library for deps-0.1.0...
Considering whether compilation is required for Lib:
MOD FLAGS: mainis:          Nothing
MOD FLAGS: safeHS:          TrustInfo None
MOD FLAGS: lang:            (Just 0,[5,7,23,44,98,99,101,120,122,125,130])
MOD FLAGS: cpp includes:
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240614\deps-0.1.0\build
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240614\deps-0.1.0\build
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240614\deps-0.1.0\build\autogen
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240614\deps-0.1.0\build\global-autogen
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240614\deps-0.1.0\build
  C:\msys64\ucrt64\include
MOD FLAGS: pic_p_opts:      []
MOD FLAGS: optP sig:        ([],4fccfb1d242a0524c052c01a0ed15cdf)
MOD FLAGS: paths:           ["hc"]
MOD FLAGS: prof:            0
MOD FLAGS: ticky:           [False,False,False,False,False]
MOD FLAGS: debugLevel:      0
MOD FLAGS: callerCc empty:  True
MOD FLAGS: optP flags:      ["dist-newstyle\\build\\x86_64-windows\\ghc-9.11.20240614\\deps-0.1.0\\build\\autogen\\cabal_macros.h","-include"]
  Module flags have changed 0e22040b910adf908347fb5cb2731333 -> e5930a88675b596d7b784723a39a2246
[1 of 1] Compiling Lib              ( Lib.hs, nothing ) [Flags changed]
MOD FLAGS: mainis:          Nothing
MOD FLAGS: safeHS:          TrustInfo None
MOD FLAGS: lang:            (Just 0,[5,7,23,44,98,99,101,120,122,125,130])
MOD FLAGS: cpp includes:
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240614\deps-0.1.0\build
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240614\deps-0.1.0\build
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240614\deps-0.1.0\build\autogen
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240614\deps-0.1.0\build\global-autogen
  dist-newstyle\build\x86_64-windows\ghc-9.11.20240614\deps-0.1.0\build
  C:\msys64\ucrt64\include
MOD FLAGS: pic_p_opts:      []
MOD FLAGS: optP sig:        ([],4fccfb1d242a0524c052c01a0ed15cdf)
MOD FLAGS: paths:           ["hc"]
MOD FLAGS: prof:            0
MOD FLAGS: ticky:           [False,False,False,False,False]
MOD FLAGS: debugLevel:      0
MOD FLAGS: callerCc empty:  True
MOD FLAGS: optP flags:      ["dist-newstyle\\build\\x86_64-windows\\ghc-9.11.20240614\\deps-0.1.0\\build\\autogen\\cabal_macros.h","-include"]
OPT FLAGS: opt_flags:
  Opt_DoLambdaEtaExpansion
  Opt_DoEtaReduction
Haddock coverage:
 100% (  3 /  3) in 'Lib'
Documentation created:
C:\Users\Javier\code\bb\.\dist-newstyle\build\x86_64-windows\ghc-9.11.20240614\deps-0.1.0\doc\html\deps\
Warning: this is a debug build of cabal-install with assertions enabled.

So the diff:

➜ diff first second
12c12
< MOD FLAGS: optP sig:        ([],8383540998520b6962a51feacf466dc3)
---
> MOD FLAGS: optP sig:        ([],4fccfb1d242a0524c052c01a0ed15cdf)
18c18
< MOD FLAGS: optP flags:      ["dist-newstyle\\\\build\\\\x86_64-windows\\\\ghc-9.11.20240614\\\\deps-0.1.0\\\\build\\\\autogen\\\\cabal_macros.h","-include"]
---
> MOD FLAGS: optP flags:      ["dist-newstyle\\build\\x86_64-windows\\ghc-9.11.20240614\\deps-0.1.0\\build\\autogen\\cabal_macros.h","-include"]

jasagredo · 2024-06-14T10:51:25Z

Upstream issue https://gitlab.haskell.org/ghc/ghc/-/issues/24988

We no longer define the `__HADDOCK_VERSION__` macro when invoking GHC through Haddock, since doing so essentially guarantees recompilation during documentation generation. We audited all uses of `__HADDOCK_VERSION__` in hackage, ensuring there was a reasonable path forward to migrate away from using `__HADDOCK_VERSION__` for each, while generating the same documentation as it did before. If you are a user of `__HADDOCK_VERSION__`, please take a look at the discussion in haskell#9177 and reach out to us if your use case is not covered. Reverts the version-cpp flag introduced in the previous commit to avoid a workaround. Instead, we get rid of the problem (`__HADDOCK_VERSION__`) at its root. See the discussion in haskell#9177

FinleyMcIlwaine force-pushed the wip/haddock-avoid-recomp branch from 89cf822 to 35c3992 Compare August 16, 2023 05:25

FinleyMcIlwaine mentioned this pull request Aug 16, 2023

--haddock-lib is not used for dependencies #9053

Closed

FinleyMcIlwaine force-pushed the wip/haddock-avoid-recomp branch 3 times, most recently from df9c87b to 4cc5ca1 Compare August 16, 2023 17:49

Kleidukos added attention: needs-review Cabal: cmd/haddock cabal-install: cmd/haddock labels Aug 16, 2023

FinleyMcIlwaine force-pushed the wip/haddock-avoid-recomp branch from 4cc5ca1 to bacf8a1 Compare August 23, 2023 03:48

FinleyMcIlwaine force-pushed the wip/haddock-avoid-recomp branch from bacf8a1 to b956a41 Compare August 24, 2023 15:52

geekosaur approved these changes Apr 12, 2024

View reviewed changes

changelog.d/pr-9177 Outdated Show resolved Hide resolved

alt-romes self-assigned this May 7, 2024

alt-romes force-pushed the wip/haddock-avoid-recomp branch 4 times, most recently from 791ba5c to fe61ff6 Compare May 8, 2024 09:33

FinleyMcIlwaine and others added 4 commits May 30, 2024 19:49

Mikolaj force-pushed the wip/haddock-avoid-recomp branch from b9cf3ac to a1e14a7 Compare May 30, 2024 19:49

mergify bot merged commit ab4c137 into haskell:master May 30, 2024
47 checks passed

michaelpj mentioned this pull request Jun 14, 2024

Come up with a way to build documentation for a package without rebuilding a lot #10111

Open

andreasabel mentioned this pull request Jan 8, 2025

how come the README example configures with --disable-documentation? haskell-actions/setup#107

Open

andreasabel added the re: enable-documentation Concerning option/field {en,dis}able-documentation label Jan 8, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Haddock recompilation avoidance #9177

Haddock recompilation avoidance #9177

FinleyMcIlwaine commented Aug 16, 2023 •

edited by alt-romes

Loading

dcoutts commented Aug 21, 2023

FinleyMcIlwaine commented Aug 21, 2023

michaelpj commented Aug 22, 2023

phadej commented Aug 22, 2023 •

edited

Loading

FinleyMcIlwaine commented Aug 22, 2023

FinleyMcIlwaine commented Aug 22, 2023 •

edited

Loading

andreabedini commented Aug 24, 2023

FinleyMcIlwaine commented Aug 24, 2023

phadej commented Aug 24, 2023

michaelpj commented Aug 24, 2023

phadej commented Aug 24, 2023

domenkozar commented Mar 19, 2024

Kleidukos commented Mar 28, 2024

Kleidukos commented Apr 30, 2024

jgotoh commented May 31, 2024

jasagredo commented May 31, 2024

jasagredo commented Jun 3, 2024

Mikolaj commented Jun 6, 2024 •

edited

Loading

jasagredo commented Jun 8, 2024 •

edited

Loading

Mikolaj commented Jun 8, 2024

jasagredo commented Jun 8, 2024

Mikolaj commented Jun 9, 2024

jasagredo commented Jun 9, 2024

jasagredo commented Jun 10, 2024

Mikolaj commented Jun 13, 2024

michaelpj commented Jun 13, 2024

phadej commented Jun 13, 2024

phadej commented Jun 13, 2024

michaelpj commented Jun 13, 2024

jasagredo commented Jun 13, 2024

michaelpj commented Jun 14, 2024

jasagredo commented Jun 14, 2024

jasagredo commented Jun 14, 2024

Haddock recompilation avoidance #9177

Haddock recompilation avoidance #9177

Conversation

FinleyMcIlwaine commented Aug 16, 2023 • edited by alt-romes Loading

QA Notes

dcoutts commented Aug 21, 2023

FinleyMcIlwaine commented Aug 21, 2023

michaelpj commented Aug 22, 2023

phadej commented Aug 22, 2023 • edited Loading

FinleyMcIlwaine commented Aug 22, 2023

FinleyMcIlwaine commented Aug 22, 2023 • edited Loading

andreabedini commented Aug 24, 2023

FinleyMcIlwaine commented Aug 24, 2023

phadej commented Aug 24, 2023

michaelpj commented Aug 24, 2023

phadej commented Aug 24, 2023

domenkozar commented Mar 19, 2024

Kleidukos commented Mar 28, 2024

Kleidukos commented Apr 30, 2024

jgotoh commented May 31, 2024

jasagredo commented May 31, 2024

jasagredo commented Jun 3, 2024

Mikolaj commented Jun 6, 2024 • edited Loading

jasagredo commented Jun 8, 2024 • edited Loading

Mikolaj commented Jun 8, 2024

jasagredo commented Jun 8, 2024

Mikolaj commented Jun 9, 2024

jasagredo commented Jun 9, 2024

jasagredo commented Jun 10, 2024

Mikolaj commented Jun 13, 2024

michaelpj commented Jun 13, 2024

phadej commented Jun 13, 2024

phadej commented Jun 13, 2024

michaelpj commented Jun 13, 2024

jasagredo commented Jun 13, 2024

michaelpj commented Jun 14, 2024

jasagredo commented Jun 14, 2024

jasagredo commented Jun 14, 2024

FinleyMcIlwaine commented Aug 16, 2023 •

edited by alt-romes

Loading

phadej commented Aug 22, 2023 •

edited

Loading

FinleyMcIlwaine commented Aug 22, 2023 •

edited

Loading

Mikolaj commented Jun 6, 2024 •

edited

Loading

jasagredo commented Jun 8, 2024 •

edited

Loading