support alternative hashes

[eeedean]

Hi,

for a couple popular libraries there may be multiple trusted hashes. In the verification-metadata.xml it may look like that:

         <artifact name="jackson-databind-2.17.2.pom">
            <sha256 value="d2450698bae90be338f2b99faeda694cd450adb52124213e7a53b41219b54062" origin="Generated by Gradle">
               <also-trust value="7d806b9c7875c2ffa35efcefc6f01984d101fe600bb01926cafa364114f6af69"/>
            </sha256>
         </artifact>

Right now, gradle-dot-nix ignores these <also-trust>-tags. To make it work, one needs to put the hashes into the correct order (switch between <sha256>- and <also-trust>-value). Of course, it would be much more easier to be handled, when gradle-dot-nix would iterate over all possible hashes.

Best regards
Dean

[CrazyChaoz]

Ok, so I've avoided this for quite some time.

The current version

1. loops through the maven urls
2. fetches/downloads artifacts that match with the name
3. calculates the hash for that artifact
4. and only if the hash matches, returns with the artifact

all of that is done in python to avoid having to deal with stuff like nixpkgs/issues/188016

[CrazyChaoz]

I've now looked through the nix source, an the failing of a FOD seems to be of type BuildError, a subtype of Error, and so I should be able to tryEval it away, and loop over all potential hashes for a source

[CrazyChaoz]

still, a LOT of work to implement & to debug

[eeedean]

still, a LOT of work to implement & to debug

Yeah, I guessed so from my own tinkering with gradle-dot-nix. My Python-Fu sucks real bad anyhow, so I rather felt like opening an issue (at least for documentation).

Working around by manually adjusting the verification-metadata.xml works for now at least.

Thanks for looking into it and verifying the existence of the issue. 😸

[CrazyChaoz]

Yeah, I get the issue, but I had hoped it wouldn't occur.

Basically the TODOs boil down to:

• adust the gradle-metadata-to-json.py to use a list for the hashes, instead of a single value
• wrap the conversion-function in default.nix in a tryEval
• filter out all false values
• use the first artifact, since any are fine
• do another loop before the conversion-function for all of the hash options
• Optional: since we then swallow hash mismatches, catching the elemAt with tryEval if there are no valid elements should be done to give better errors

[CrazyChaoz]

You could, probably, maybe, hopefully try out the multi-hash-fetching branch, which should be able to do the multi-hash-fetching.
I have only tried it on small projects so far, so your mileage may vary.

[eeedean]

Hi, I just tried and it did not seem to work for me in my case.

What I have in my dependency-verification.xml:

<component group="com.fasterxml.jackson.module" name="jackson-modules-java8" version="2.17.2">                                  
  <artifact name="jackson-modules-java8-2.17.2.pom">                                                                           
    <sha256 value="fc0839cd9bbac2774ac8cf2aec51f4088a69369575e7a963f972cc07f4fcb8b3" origin="Generated by Gradle">            
      <also-trust value="3f39c55109f51a228817b4b185e4351b9ef0501c09fc1e1a3075ae9547cc2d03"/>                                 
    </sha256>                                                                                                                 
  </artifact>                                                                                                                  
</component>

And the error I get:

error: build of '/nix/store/5iyfp9g7hzjvgfcdcllpvfmq7wwv4xiy-jackson-modules-java8-2.17.2.pom.drv' on 'ssh-ng://builder@linux-builder' failed: hash mismatch in fixed-output derivation '/nix/store/5iyfp9g7hzjvgfcdcllpvfmq7wwv4xiy-jackson-modules-java8-2.17.2.pom.drv':
         specified: sha256-/Ag5zZu6wndKyM8q7FH0CIppNpV156lj+XLMB/T8uLM=
            got:    sha256-PznFUQn1GiKIF7SxheQ1G57wUBwJ/B4aMHWulUfMLQM=

Switching the two hashes around, does still resolve the issue.

Maven Repos I use:

[
  "https://repo.maven.apache.org/maven2",
  "https://plugins.gradle.org/m2",
  "https://repo.spring.io/milestone"
]

[CrazyChaoz]

do you have some good (public) repo i could use for a bit of trial-and-error debugging ?

[eeedean]

Hi,

I just updated my play-repo for this kinda setup.

I used your current commit from the feature branch 873b7cdb43be8921c6fa91efd73a874e5c0325ed.

gradle/verification-metadata.xml

      <artifact name="jackson-modules-java8-2.17.2.pom">
        <sha256 value="fc0839cd9bbac2774ac8cf2aec51f4088a69369575e7a963f972cc07f4fcb8b3" origin="Generated by Gradle">
          <also-trust value="3f39c55109f51a228817b4b185e4351b9ef0501c09fc1e1a3075ae9547cc2d03"/>
        <!--<sha256 value="3f39c55109f51a228817b4b185e4351b9ef0501c09fc1e1a3075ae9547cc2d03" origin="Generated by Gradle">-->
          <!--<also-trust value="fc0839cd9bbac2774ac8cf2aec51f4088a69369575e7a963f972cc07f4fcb8b3"/>-->
        </sha256>
      </artifact>

You can just comment in the lines, i commented out and comment out the other two in order to switch the hashes.
When switched, it works, when not switched, it wont work.

nix build does build the spring boot application, which does therefore try to fetch the dependencies.

Thanks for your efforts! I'm very glad, there is some progress in nixification of the Java Ecosystem.

[CrazyChaoz]

alright, time to ditch the current approach:
NixOS/nix#356 (comment)
i get a BuildError from the failing derivation, so a tryEval ACTUALLY DOESN'T CATCH IT

[eeedean]

Hi,

just had a chat at NixCon 2024 with @mschwaig and we seem to agree, that the necessity for the feature is actually present only, because private, authenticated repositories, aren't supported by gradle-dot-nix.

If .netrc inclusion was supported, there would be no need, as I could just put the same repositories into gradle-dot-nix, as I got in my settings.gradle in my project.

If you wanted to be very compliant to the gradle-API, it may be preferable, to »also« support the alternative hash feature, however from my perspective, it is just a very poor and (not really) cheap solution to the lack of authentication support.
To me, this feature loses any value, as soon as authenticated repositories work.

Having multiple valid hashes to the "same" binary dependency is also kinda wrong from a supply chain security perspective.


Would have loved to talk to you in person about that stuff, maybe we can meet up at another NixCon or something 🙃.

Best regards
Dean