Skip to content

Commit

Permalink
Fix for K=64
Browse files Browse the repository at this point in the history
  • Loading branch information
awf committed Jun 6, 2024
1 parent 05e91a3 commit b669834
Showing 1 changed file with 2 additions and 3 deletions.
5 changes: 2 additions & 3 deletions src/gfloat/round.py
Original file line number Diff line number Diff line change
Expand Up @@ -62,16 +62,15 @@ def round_float(
# Extract exponent
expval = int(np.floor(np.log2(vpos)))

assert expval > -1024 + p # not yet tested for float64 near-subnormals

# Effective precision, accounting for right shift for subnormal values
if fi.has_subnormals:
expval = max(expval, 1 - bias)

# Lift to "integer * 2^e"
expval = expval - p + 1

fsignificand = vpos * 2.0**-expval
# use ldexp instead of vpos*2**-expval to avoid overflow
fsignificand = math.ldexp(vpos, -expval)

# Round
isignificand = math.floor(fsignificand)
Expand Down

0 comments on commit b669834

Please sign in to comment.