CEP: Recipe serialization in packages #74

wolfv · 2024-04-13T20:55:15Z

Rendered version: ✏️ https://github.com/wolfv/ceps/blob/cep-recipe-serialization/cep-recipe-serialization.md

cep-recipe-serialization.md

jaimergp · 2024-04-15T16:51:53Z

cep-recipe-serialization.md

+target_directory: # optional target directory where the source should be extracted
+```
+
+### System tools


This is implicit, but I guess that this dictionary is a dict[str, str] one right? Tool name string to version string.

schuylermartin45 · 2024-04-16T18:07:57Z

cep-recipe-serialization.md

+## Full Example
+
+```yaml
+recipe:


Just going to throwing this out there, are we worried about how these files scale? I can imagine some very complex multi-output, multi-platform recipes could produce very long rendered_recipe.yaml files.

Well, each package only contains a rendered recipe for a single output, for a single platform. So it's not going to be crazy big. But lockfiles can indeed become relatively large. I think I would expect at most something like ~1 Mb. However, that should compress pretty well (except for the checksums). And the benefits are pretty huge (SBOMs, reproducibility, ...).

It would, however, be an option to only store e.g. the URL + 1 hash. That would reduce the size quite decently. Happy to consider / discuss that.

Ah I missed that there was one file per output. I saw the build section and immediately thought those were individual outputs.

I don't have a strong opinion, I'm just pointing out this as a potential future headache. I know we have some large manifest-type files in conda world (looking at you, repodata.json) and I'm sure we all agree we don't want to repeat that pattern.

wolfv · 2024-04-25T11:40:27Z

cep-recipe-serialization.md

+    constrains: []
+    run_exports: null
+finalized_sources:
+  - url: http://curl.haxx.se/download/curl-8.0.1.tar.bz2


We could add a content hash here (as proposed recently in conda-build).

JeanChristopheMorinPerso · 2024-05-06T17:32:33Z

cep-recipe-serialization.md

+  ```yaml
+  - ${{ compiler('c') }}
+  # becomes
+  - compiler: c


Can we say that this is the "evaluated" form? In my mind this isn't an evaluated form, it's more a serialized form.

I thought ${{ compiler('c') }} got evaluated to (e.g.) gcc_linux-64? Where does compiler: c come from?

We currently have an "intermediate" form in rattler-build. We need that for pin_subpackage because that needs to have the rest of the recipe rendered first.
However, for compiler we can make the function return the final value instead. It would have some other benefits (e.g. that we can use the compiler function in other contexts easily).

JeanChristopheMorinPerso · 2024-05-06T17:33:15Z

cep-recipe-serialization.md

+  ```yaml
+  - ${{ pin_subpackage('libtool', exact=true) }}
+  # becomes
+  - pin_subpackage:


What's the rational behind not fully resolving the value?

The value is fully resolved later on in the resolved dependencies. Internally we keep it unresolved in the recipe to later resolve it during the execution.

JeanChristopheMorinPerso · 2024-05-06T17:38:05Z

cep-recipe-serialization.md

+
+- Type: object
+- Description: Contains information related to hashing.
+  - `hash`: The hash value (e.g., `60d57d3`).


Does this contain the full build string? For example py312h7f4fdc5_0 or is it really just the hash (7f4fdc5)? If it contains only the hash, should there be a suffix entry too?

yes, this contains only the hash. There is a hash_prefix entry indeed. Hash and build-string are two different things :)

JeanChristopheMorinPerso · 2024-05-06T17:39:03Z

cep-recipe-serialization.md

+  - `hash`: The hash value (e.g., `60d57d3`).
+  - `hash_input`: The input values used for generating the hash.
+  - `hash_prefix`: Prefix for the hash, which can include values like `py311`
+    based on the variant configuration.


Is this solely based on the variant or can it be influenced by build.string?

No, the hash is used by default in the build string, but the build string cannot influence the hash directly.

JeanChristopheMorinPerso · 2024-05-06T17:40:25Z

cep-recipe-serialization.md

+- Description: Contains directory paths used during the build process.
+  - `host_prefix`: The host prefix directory with placeholders for the host
+    environment.
+  - `build_prefix`: The build prefix directory for the build environment.


What happens when merge_build_and_host_envs is True?

I feel like we're missing a CEP or, at a minimum, a reference to some other spec or documentation. This list of directories should correspond to the prescribed behaviors for the tools that process the package recipe.

I assume merged build and host means that host_prefix == build_prefix, so both values are the same. Alternatively, host_prefix == null.

JeanChristopheMorinPerso · 2024-05-06T17:57:04Z

cep-recipe-serialization.md

+    sha256: 9609163c14cd49ec651562838f38b88ed2d370e354af312ddc78c2be76c08d37
+```
+
+The following fields are available for each source:


Are they almost a 1 to 1 from CEP-14?

The finalized sources are almost but not exactly. We are adding the actual rev of the git repository (so that we can retrieve the actual commit when rebuilding).

JeanChristopheMorinPerso · 2024-05-06T17:58:02Z

cep-recipe-serialization.md

+```yaml
+url: # URL to the source code
+sha256: # sha256 hash of the downloaded file
+md5: # md5 hash of the downloaded file


Are they mandatory? Should the tool that produce the packages add the field even if it's not set in the original recipe?

cep-recipe-serialization.md

JeanChristopheMorinPerso · 2024-05-06T18:01:20Z

cep-recipe-serialization.md

+
+### System tools
+
+This section records all system tools that were used during build execution and


How does conda-build or rattler-build know which tool was used? Should this be somewhat defined?

I added a used_build_tool.json file in the specification :)

dholth · 2024-05-10T13:26:30Z

Why not .json?
Do we still need info/recipe/meta.yaml ?

chenghlee · 2024-05-14T16:45:48Z

cep-recipe-serialization.md

+We want to specify everything that goes into the final package in a standardized
+format. This defines the structure and contents of the rendered_recipe.yaml file


We might want to avoid using the word "everything" here. From what I gathered below, this CEP covers what goes into the info/recipe directory of the output package(s), and not literally everything that goes into the package.

chenghlee · 2024-05-14T16:47:36Z

cep-recipe-serialization.md

+
+The rendered recipe YAML file is stored in the `info/recipe` folder of the final
+package. Alongside the rendered recipe, we copy the entire "recipe folder".
+If the original recipe was not named `recipe.yaml`, then we rename it inside the package.


If the original recipe was not named recipe.yaml, then we rename it inside the package.

Is there a reason to rename, rather than capture the build command that was used while preserving the original filename? As written, this also suggests the existing (v1) recipes that use meta.yaml would/should have said meta.yaml renamed to recipe.yaml in the output package, and that seems...odd?

We could keep the recipe.yaml name. We should definitely have a consistent name for the rendered recipe (currently rendered_recipe.yaml). I don't have super strong feelings. I do like it if we can find the recipe in a consistent place though so that extracting / inspecting it is easy.

chenghlee · 2024-05-14T16:49:38Z

cep-recipe-serialization.md

+  ```yaml
+  - ${{ compiler('c') }}
+  # becomes
+  - compiler: c


I thought ${{ compiler('c') }} got evaluated to (e.g.) gcc_linux-64? Where does compiler: c come from?

chenghlee · 2024-05-14T17:07:14Z

cep-recipe-serialization.md

+##### `target_platform`
+
+- Type: string
+- Description: The target platform for which the package is being built (e.g.,
+  `osx-arm64`).
+
+##### `host_platform`
+
+- Type: string
+- Description: The host platform used for building the package. Usually
+  equivalent to `target_platform`, except when the `target_platform` is
+  `noarch`. If the `target_platform` is `noarch`, the `host_platform` is the
+  platform where the build is being executed.
+
+##### `build_platform`
+
+- Type: string
+- Description: The platform used for building the package (e.g., `osx-arm64`).


If we are going to use cross-compilation language ("target", "build", "host"), we should spend some space to explain what each of these terms really mean. E.g., describing both host_platform and build_platform as "the platform used for building the package" is confusing.

chenghlee · 2024-05-14T20:09:59Z

cep-recipe-serialization.md

+##### `variant`
+
+- Type: object
+- Description: Dictionary with the variant configuration used for the build.


We should specify where such configuration comes from and what actually gets recorded. E.g., do the values come from conda_build_config.yaml, and if so, do we only preserve those keys that are actually used in processing this recipe, or do we replicate all values from the cbc.yaml?

I think for now it is fine to say "from the variant"? :)

chenghlee · 2024-05-14T21:13:15Z

cep-recipe-serialization.md

+The list of `resolved` dependencies contains a dictionary with the following
+fields (equivalent to `RepoDataRecord`):
+
+```yaml
+build: hb7217d7_0
+build_number: 0
+depends: []
+license: GPL-2.0-or-later
+license_family: GPL
+md5: fe8efc3385f58f0055e8100b07225055
+name: libtool
+sha256: efe50471d2baea151f2a93f1f99c05553f8c88e3f0065cdc267e1aa2e8c42449
+size: 408403
+subdir: osx-arm64
+timestamp: 1672362003513 # when available
+version: 2.4.7
+fn: libtool-2.4.7-hb7217d7_0.conda
+url: https://conda.anaconda.org/conda-forge/osx-arm64/libtool-2.4.7-hb7217d7_0.conda
+channel: https://conda.anaconda.org/conda-forge/
+```


Is there a reason to repeat the contents for RepoDataRecord? Seems to me that the URL and SHA256 checksum of the package would be enough to recreate the various build environments.

yeah, I agree. sha hash and url should be enough. I think it might be interesting to include additional information (e.g. licenses) so that tools could report on what-went-into the package ... Of course, given that we still have access to the packages we could recreate this information from the repodata / packages, too.

I think there might be a few more arguments:

repodata changes over time and this is an accurate snapshot as how it was at the time

it makes it easier to extract a conda-lock file from the rendered recipe

there might be additioanl opportunities for tooling in the future (license scanning, ...)

chenghlee · 2024-05-14T21:14:48Z

cep-recipe-serialization.md

+sha256: # computed sha256 hash of the source (this is added if not in the source description)
+patches: # list of patches that should be applied to the source
+target_directory: # optional target directory where the source should be extracted
+file_name: # optional file name


Wouldn't the filename be part of path?

Filename is more for renaming the file in the work directory. So yes, you could add a file_name argument if you want (and then it would be rendered).

chenghlee · 2024-05-14T21:17:53Z

cep-recipe-serialization.md

+git: # git URL
+rev: # resolved commit hash at build time
+depth: # optional depth to clone the repository
+patches: # list of patches that should be applied to the source


We should clarify where these patches come from. (I suspect, relative to the top-level of the directory, but it should be explicitly stated.)

relative to the recipe directory, yeah.

chenghlee · 2024-05-14T21:19:12Z

cep-recipe-serialization.md

+```yaml
+git: # git URL
+rev: # resolved commit hash at build time
+depth: # optional depth to clone the repository


Is this supposed to correspond to the --depth argument to git clone (which basically truncates history), or to (any) submodule clone/init operations?

Yeah this is to truncate history (conda-build has the same argument).

chenghlee · 2024-05-14T21:20:17Z

cep-recipe-serialization.md

+md5: # md5 hash of the downloaded file
+file_name: # optional file name to rename the downloaded file
+patches: # list of patches that should be applied to the source
+target_directory: # optional target directory where the source should be extracted


See previous comments re: target_directory.

Co-authored-by: Jean-Christophe Morin <[email protected]>

dholth · 2024-06-11T14:50:47Z

cep-recipe-serialization.md

+- Description: Contains settings related to the final package output.
+  - `archive_type`: The type of archive used for packaging (either `tar_bz2` or
+    `conda`).
+  - `compression_level`: The numerical compression level used for packaging


Is this just for fun

No, it's for perfectly reproducible builds (https://github.com/prefix-dev/reproducible-builds)

Overview here: https://prefix-dev.github.io/reproducible-builds/

Would it be a better idea to reproduce the data after compression? I also notice the number of threads has been removed? The zstandard version is recorded somewhere or implied by the tool version?

The number of threads doesnt influence the final artifact so we removed it.

In the ideal world we can re-create the compressed artifact bit-for-bit. Content hashes are also important. But for security reasons and so on I believe it's nicer to be able to really recreate the artifact. At least I believe that's what most people in reproducible-builds.org are trying :)

wolfv · 2024-06-12T07:41:21Z

cep-recipe-serialization.md

+  ```yaml
+  - ${{ pin_subpackage('libtool', exact=true) }}
+  # becomes
+  - pin_subpackage:


The value is fully resolved later on in the resolved dependencies. Internally we keep it unresolved in the recipe to later resolve it during the execution.

wolfv · 2024-06-12T07:54:01Z

cep-recipe-serialization.md

+
+The rendered recipe YAML file is stored in the `info/recipe` folder of the final
+package. Alongside the rendered recipe, we copy the entire "recipe folder".
+If the original recipe was not named `recipe.yaml`, then we rename it inside the package.


We could keep the recipe.yaml name. We should definitely have a consistent name for the rendered recipe (currently rendered_recipe.yaml). I don't have super strong feelings. I do like it if we can find the recipe in a consistent place though so that extracting / inspecting it is easy.

cep-recipe-serialization.md

wolfv · 2024-06-12T08:12:53Z

cep-recipe-serialization.md

+sha256: # computed sha256 hash of the source (this is added if not in the source description)
+patches: # list of patches that should be applied to the source
+target_directory: # optional target directory where the source should be extracted
+file_name: # optional file name


Filename is more for renaming the file in the work directory. So yes, you could add a file_name argument if you want (and then it would be rendered).

wolfv · 2024-06-12T08:14:09Z

cep-recipe-serialization.md

+```yaml
+git: # git URL
+rev: # resolved commit hash at build time
+depth: # optional depth to clone the repository


Yeah this is to truncate history (conda-build has the same argument).

wolfv · 2024-06-12T08:18:50Z

cep-recipe-serialization.md

+        fn: ca-certificates-2024.2.2-hf0a4a13_0.conda
+        url: https://conda.anaconda.org/conda-forge/osx-arm64/ca-certificates-2024.2.2-hf0a4a13_0.conda
+        channel: https://conda.anaconda.org/conda-forge/
+    run_exports:


@baszalmstra - we removed these here, right?

Yep updated this in 00ad601

wolfv · 2024-06-12T08:19:13Z

cep-recipe-serialization.md

+        fn: libzlib-1.3.1-h0d3ecfb_0.conda
+        url: https://conda.anaconda.org/conda-forge/osx-arm64/libzlib-1.3.1-h0d3ecfb_0.conda
+        channel: https://conda.anaconda.org/conda-forge/
+    run_exports:


I updated them in 00ad601

wolfv · 2024-06-12T08:19:31Z

cep-recipe-serialization.md

+          - perl >=5.32.1,<6.0a0 *_perl5
+  host:
+    specs:
+      - source: raw


I think this needs to be updated as well

I updated them in 00ad601

cep-recipe-serialization.md

wolfv

I just did another pass as well.

wolfv · 2024-07-02T12:21:21Z

I changed this CEP a little - most of the CEP is now "implementation specific" and serves as documentation of what rattler-build does.

However, the CEP does require alternative build tools to implement a rendered recipe, and also requires a new file to be added to the package (info/used_build_tool.json).

jezdez · 2024-07-02T13:18:47Z

@conda/steering-council

This vote falls under the "Enhancement Proposal Approval" policy of the conda governance policy,
please vote and/or comment on this proposal at your earliest convenience.

It needs 60% of the Steering Council to vote yes to pass.

To vote, please leave yes, no or abstain as comments below.

If you have questions concerning the proposal, you may also leave a comment or code review.

This vote will end on 2024-07-16, End of Day, Anywhere on Earth (AoE). This is an extended voting period due to summer holiday time in the Northern Hemisphere.

baszalmstra · 2024-07-02T13:22:31Z

yes

wolfv · 2024-07-02T13:42:29Z

mariusvniekerk · 2024-07-02T14:25:11Z

cep-recipe-serialization.md

+
+```yaml
+path: # path to the source
+sha256: # computed sha256 hash of the source (this is only added if not in the source entry in the original recipe)


This can be prohitively expensive to compute

yeah, I think it's still worth it though. We might even add a content hash :)
But we could also add knobs to disable it.

jaimergp

Just a few typos I found.

jaimergp · 2024-07-06T12:06:05Z

cep-recipe-serialization.md

+├── index.json
+├── used_build_tool.json  // new file!
+└── recipe
+    ├── build.bat


Huh, this has changed from bld.bat?

cep-recipe-serialization.md

jaimergp · 2024-07-06T12:09:54Z

cep-recipe-serialization.md

+- Description: Contains directory paths used during the build process.
+  - `host_prefix`: The host prefix directory with placeholders for the host
+    environment.
+  - `build_prefix`: The build prefix directory for the build environment.


I assume merged build and host means that host_prefix == build_prefix, so both values are the same. Alternatively, host_prefix == null.

Co-authored-by: jaimergp <[email protected]>

wolfv · 2024-07-16T10:25:25Z

@marcelotrevisani @jakirkham @CJ-Wright @mbargull last chance to vote!

chenghlee · 2024-07-16T16:29:09Z

For the record: voted "no" because while I like the ideas presented in this CEP, I don't think the CEP as written is ready to be adopted as a specification. IMO, various unanswered questions/comments need to be addressed before adoption, and we should better separate behavioral specifications from implementation details.

mbargull · 2024-07-17T04:45:15Z

I think it is good to have these parts of the build output/process specified which is why I support to have a proposal for it here.
I do agree with Cheng in that this is not yet ready to be put in, though, since the "Specification" section is a bit light in detail and the bigger part of the text, i.e., description of implementation details, are likely to change, so nothing we'd set in stone here.

jakirkham

Thanks Wolf! 🙏

Think these are good improvements. Am voting yes with a few suggestions/questions (tried to use the checkbox, but had issues with it for some reason)

cep-recipe-serialization.md

jakirkham · 2024-07-17T09:50:34Z

cep-recipe-serialization.md

+```js
+{
+  "name": "rattler-build",  // string
+  "version": "0.14.0"       // string
+}
+```


Maybe it is worth extending this in the future with other dependencies (like the solver used, Jinja version, etc.), in which case we may want to nest these somehow

jakirkham · 2024-07-17T09:55:37Z

cep-recipe-serialization.md

+  ```yaml
+  - ${{ pin_subpackage('some-pkg-a', lower_bound="x.x.x") }}
+  - ${{ pin_compatible('python', upper_bound="x.x.x") }}
+  - ${{ pin_compatible('numpy', upper_bound="3.2.1", lower_bound="1.2.3") }}
+  ```


Think currently we use max_pin / min_pin with x.x.x. And use upper_bound / lower_bound with 1.2.3

https://docs.conda.io/projects/conda-build/en/latest/resources/variants.html#pinning-expressions

This is intentionally unified in the recent set of CEPs, please see this thread

jakirkham · 2024-07-17T10:10:11Z

cep-recipe-serialization.md

+- pin_subpackage: some-pkg-a
+  lower_bound: x.x    # not present if set to None
+  upper_bound: x.x.x  # not present if set to None
+  exact: true         # not present if false
+  spec: some-pkg-a >=1.0.0,<2


Am curious how these work together. We have exact and a more relaxed spec as well bounds. What takes precedence?

They should not be allowed together, right?

mbargull · 2024-07-17T10:24:25Z

(tried to use the checkbox, but had issues with it for some reason)

@jakirkham
It's really odd that it doesn't like to accept your edit :/.
I checked "yes" on your behalf.

Co-authored-by: jakirkham <[email protected]>

jakirkham · 2024-07-17T20:46:10Z

Thanks Marcel! 🙏

Yeah this happened to me a couple times. So just used "approved" to mean the same thing. Think it is because I tried on my iPhone with GitHub Mobile (not sure why it can't check a box or why it makes an empty edit though 🤷‍♂️)

wolfv · 2024-07-22T15:25:23Z

The vote is closed, and we have the following result:

Total voters: 15 (valid: 13 = 86.67%)

Yes votes (11 / 84.62%):

@xhochy (Uwe Korn)
@mariusvniekerk (Marius van Niekerk)
@goanpeca (Gonzalo Peña-Castellanos)
@ocefpaf (Filipe Fernandes)
@msarahan (Michael Sarahan)
@jakirkham (John Kirkham)
@jezdez (Jannis Leidel)
@wolfv (Wolf Vollprecht)
@jaimergp (Jaime Rodríguez-Guerra)
@kkraus14 (Keith Kraus)
@baszalmstra (Bas Zalmstra)

No votes (2 / 15.38%)):

@chenghlee (Cheng H. Lee)
@mbargull (Marcel Bargull)

Abstain votes (0 / 0.00%):

Not voted (2):

@CJ-Wright (Christopher J. 'CJ' Wright)
@marcelotrevisani (Marcelo Duarte Trevisani)

Invalid votes (0):

Thus we reached quorum and enough YES votes to mark this as accepted. 🎉

add initial CEP for recipe serialization

037ce02

wolfv force-pushed the cep-recipe-serialization branch from b607b6c to 037ce02 Compare April 15, 2024 07:27

This was referenced Apr 15, 2024

Add new source hashing methods: content_sha256, content_sha384, content_sha512 conda/conda-build#5277

Open

Better hashing of sources conda/conda-build#4762

Open

jaimergp reviewed Apr 15, 2024

View reviewed changes

cep-recipe-serialization.md Show resolved Hide resolved

jaimergp reviewed Apr 15, 2024

View reviewed changes

cep-recipe-serialization.md Outdated Show resolved Hide resolved

jaimergp reviewed Apr 15, 2024

View reviewed changes

schuylermartin45 reviewed Apr 16, 2024

View reviewed changes

update schema

903d562

wolfv commented Apr 25, 2024

View reviewed changes

JeanChristopheMorinPerso reviewed May 6, 2024

View reviewed changes

chenghlee reviewed May 14, 2024

View reviewed changes

wolfv and others added 12 commits May 21, 2024 23:54

Update cep-recipe-serialization.md

4598447

Co-authored-by: Jean-Christophe Morin <[email protected]>

small update

05c9223

remove hash_input and rename hash_prefix to prefix

0cdf2f1

modified finalized dependency section

54ee06b

add pin args to finialized dependencies

b97b76c

remove host_platform

0613baa

remove more hash_input references

4980bcd

bring back host_platform with an explanation

7aef8c2

add channel_priority and solve_strategy

0a5ac43

add optional to channel_priority and solve_strategy

6b93356

rename constrains to constraints

9038141

remove compression_threads

bb8aeb1

wolfv changed the title ~~add initial CEP for recipe serialization~~ CEP: Recipe serialization in packages Jun 10, 2024

dholth reviewed Jun 11, 2024

View reviewed changes

some updates to pin rendering after we changed the arguments in jinja

dd9c74b

wolfv commented Jun 12, 2024

View reviewed changes

baszalmstra and others added 3 commits June 12, 2024 10:48

update examples

00ad601

adjust to latest changes

7b30f7f

update spec

547c386

jezdez added the vote Voting following governance policy label Jul 2, 2024

mariusvniekerk reviewed Jul 2, 2024

View reviewed changes

jaimergp approved these changes Jul 6, 2024

View reviewed changes

Update cep-recipe-serialization.md

a59f339

Co-authored-by: jaimergp <[email protected]>

jakirkham approved these changes Jul 17, 2024

View reviewed changes

Update cep-recipe-serialization.md

840c84a

Co-authored-by: jakirkham <[email protected]>


		### System tools

		This section records all system tools that were used during build execution and

		We want to specify everything that goes into the final package in a standardized
		format. This defines the structure and contents of the rendered_recipe.yaml file

CEP: Recipe serialization in packages #74

Are you sure you want to change the base?

CEP: Recipe serialization in packages #74

Conversation

wolfv commented Apr 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dholth commented May 10, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wolfv left a comment

Choose a reason for hiding this comment

wolfv commented Jul 2, 2024

jezdez commented Jul 2, 2024

baszalmstra commented Jul 2, 2024

wolfv commented Jul 2, 2024 • edited by marcelotrevisani Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jaimergp left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wolfv commented Jul 16, 2024

chenghlee commented Jul 16, 2024

mbargull commented Jul 17, 2024

jakirkham left a comment

Choose a reason for hiding this comment

wolfv commented Apr 13, 2024 •

edited

Loading

wolfv commented Jul 2, 2024 •

edited by marcelotrevisani

Loading