You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The Float value produced is not equal to what fromIntegral @_ @Float 655361200000 returns.
It appears that decimal accumulates floating-point errors during parsing so that the final Float result is wrong. (Given the limited precision of Float, error is expected but the result is further from the perfectly accurate value than necessary.) The error probably accumulates from lossy arithmetic performed for each input character, including the (* 10).
Double is likely similarly affected, but I have not constructed a counter-example, which probably requires more digit than Float counter-example.
The text was updated successfully, but these errors were encountered:
Wouldn't it help if decimal first parses into a non-lossy type such as Integer and then used fromInteger? For the same reason the float parser uses Scientific under the hood.
The
Float
value produced is not equal to whatfromIntegral @_ @Float 655361200000
returns.It appears that
decimal
accumulates floating-point errors during parsing so that the finalFloat
result is wrong. (Given the limited precision ofFloat
, error is expected but the result is further from the perfectly accurate value than necessary.) The error probably accumulates from lossy arithmetic performed for each input character, including the(* 10)
.Double
is likely similarly affected, but I have not constructed a counter-example, which probably requires more digit thanFloat
counter-example.The text was updated successfully, but these errors were encountered: