Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Write Properties files in a reproducible way #4583

Merged
merged 1 commit into from
Jan 10, 2025

Conversation

Zlika
Copy link
Contributor

@Zlika Zlika commented Jan 2, 2025

Java does not write java.util.Properties files in a reproducible way: there is a timestamp, the order of the lines is not reproducible, and the new line character used is system-dependent.
This PR fixes this problem by removing the timestamp, ordering the lines in the alphabetical order and using a system-independent new line character.
This is a follow-up of #4546 and #4569.

Copy link

github-actions bot commented Jan 2, 2025

Test Results

  606 files  ±0    606 suites  ±0   4h 21m 55s ⏱️ + 4m 38s
  434 tests ±0    422 ✅  - 3   7 💤 ±0  0 ❌  - 2  5 🔥 +5 
1 302 runs  ±0  1 274 ✅  - 4  22 💤 ±0  0 ❌  - 2  6 🔥 +6 

For more details on these errors, see this check.

Results for commit 74067e9. ± Comparison against base commit e052363.

This pull request removes 1 and adds 1 tests. Note that renamed tests count towards both.
org.eclipse.tycho.test.reproducible.ReproducibleBuildTest ‑ test
org.eclipse.tycho.test.reproducible.ReproducibleBuildTest ‑ testReproducible

♻️ This comment has been updated with latest results.

@laeubi
Copy link
Member

laeubi commented Jan 3, 2025

and the new line character used is system-dependent.

Last time I followed a discussion with the maven folks, they said that newlines are always a problem for reproducible build and one should rebuild on the same system architecture, so maybe not a big deal but probably also don't harm much.

@Zlika
Copy link
Contributor Author

Zlika commented Jan 3, 2025

and the new line character used is system-dependent.

Last time I followed a discussion with the maven folks, they said that newlines are always a problem for reproducible build and one should rebuild on the same system architecture, so maybe not a big deal but probably also don't harm much.

That's true, but when maven-archiver fixes the pom.properties file (in https://github.com/apache/maven-archiver/blob/master/src/main/java/org/apache/maven/shared/archiver/PomPropertiesUtil.java) it also uses a system-independent newline character. Anyway, trying to reuse the existing newline characters would make the code more complex, so it's simpler to use a system-independent newline character here. It would be dangerous to try to blindly fix all the text files to use a system-independent newline character, but when we need to rewrite a file it makes sens to take the opportunity to do it.

@Zlika Zlika force-pushed the reproducible-properties branch from 29f63ab to 51a7c6c Compare January 4, 2025 10:04
@Zlika
Copy link
Contributor Author

Zlika commented Jan 4, 2025

Hi @laeubi, I took into account all your remarks. Thanks.

@Zlika Zlika force-pushed the reproducible-properties branch from 51a7c6c to a6e837a Compare January 5, 2025 15:15
@merks
Copy link
Contributor

merks commented Jan 5, 2025

Perhaps the same conclusion as in eclipse-equinox/p2#589 (comment) applies here, i.e., with Java 21 the properties are sorted and you can set a fixed reproducible timestamp via java.properties.date...

@Zlika
Copy link
Contributor Author

Zlika commented Jan 5, 2025

Perhaps the same conclusion as in eclipse-equinox/p2#589 (comment) applies here, i.e., with Java 21 the properties are sorted and you can set a fixed reproducible timestamp via java.properties.date...

Using a system property to set the date is quite messy, but it's an interesting workaround when tycho calls external tools like p2. It's a pity that this feature was only added in Java 18.

@laeubi
Copy link
Member

laeubi commented Jan 10, 2025

@Zlika can you adjust the util method to use Path API then I think this can be merged so we have also the support for java 21 at laest in the core Tycho.

@Zlika Zlika force-pushed the reproducible-properties branch from a6e837a to 92e4fbc Compare January 10, 2025 08:46
@Zlika
Copy link
Contributor Author

Zlika commented Jan 10, 2025

@Zlika can you adjust the util method to use Path API then I think this can be merged so we have also the support for java 21 at laest in the core Tycho.

Done.

Copy link
Member

@laeubi laeubi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few adjustments and I think we are done!

@Zlika Zlika force-pushed the reproducible-properties branch from 92e4fbc to 74067e9 Compare January 10, 2025 17:32
Copy link
Member

@laeubi laeubi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks fine and consistent to me now.

@laeubi laeubi added the backport-to-tycho-4.0.x Can be added to a PR to trigger an automatic backport of the change label Jan 10, 2025
@laeubi laeubi enabled auto-merge (rebase) January 10, 2025 17:36
@laeubi laeubi merged commit 30a6a70 into eclipse-tycho:main Jan 10, 2025
12 of 16 checks passed
@eclipse-tycho-bot
Copy link

💚 All backports created successfully

Status Branch Result
tycho-4.0.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation and see the Github Action logs for details

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-to-tycho-4.0.x Can be added to a PR to trigger an automatic backport of the change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants