Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add digits argument to String::num_scientific and fix serializing #96676

Closed

Conversation

aaronfranke
Copy link
Member

@aaronfranke aaronfranke commented Sep 7, 2024

Supersedes PR #86951, fixes #78204.

This PR adds a digits argument to String::num_scientific and various Variant methods, and uses this to serialize numbers with more precision.

We need to serialize with more precision to ensure that a serialized number can be deserialized into the same number. For example, for the number 123456789, the closest 32-bit float is 123456792. In master this is serialized as 1.23457e8, which becomes 123457000, over 200 off from the closest 32-bit float. With this PR, if a 32-bit float, it will be serialized with 9 digits as 123456792, which can be read back as exactly 123456792. 32-bit floats have 6 reliable digits, but 9 are needed to serialize to decimal in order to read back with full precision.

For an example with 64-bit floats, I have 1.234567898765432123456789e30 included in the test cases. The closest 64-bit float is 1.23456789876543218850569440461e30 (differs at the 8 which used to be a 2). This gets serialized as 1.2345678987654322e+30 which is deserialized to exactly 1.23456789876543218850569440461e30. 64-bit floats have 14 reliable digits, but 17 are needed to serialize to decimal in order to read back with full precision.

Note that this will result in more decimals being printed unnecessarily in some cases. For example, 3.4 becomes 3.4000001 as a 32-bit float and 3.3999999999999999 as a 64-bit float. A better but inefficient algorithm would be to keep increasing the digits until the value matches, but this would require constantly re-writing and re-parsing. I am not sure how this can be improved while still being efficient, but I am sure it's possible. If somebody wants to investigate how this can be improved before this PR is merged, that would be welcome.

Note that this does not affect the version bound to GDScript, which uses 6 digits just like it already does in master. I tried adding the argument to the bindings and it seems to break GDExtension compatibility.

Note that writing the unreliable digits should not be done when printing to the user, this is just meant for serializing.

Note that the docs have special code that uses 6 digits, since those are intended to be displayed to the user.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that get_construct_string() is also used in modules/gdscript/gdscript_editor.cpp.

@aaronfranke aaronfranke force-pushed the num-sci-digits-serialize branch from 11d16ff to 61f94d0 Compare October 5, 2024 01:47
@aaronfranke aaronfranke marked this pull request as draft November 1, 2024 20:49
@aaronfranke aaronfranke removed this from the 4.4 milestone Nov 2, 2024
@aaronfranke
Copy link
Member Author

I opened PR #98750 which is a much better alternative to this PR.

@aaronfranke aaronfranke closed this Nov 2, 2024
@aaronfranke aaronfranke deleted the num-sci-digits-serialize branch December 5, 2024 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

var_to_str rounds floats, losing massive precision in the process
4 participants