Add lua-5.4 to list of libluas to consider #1118
Conversation
modified: configure Cast SIZE_MAX to lua_number before comparing it to n (a variable of that type) to avoid compile-time warning. modified: vis-lua.c
donaldcallen
changed the title
| @@ -496,7 +496,7 @@ static size_t getpos(lua_State *L, int narg) { | |||
|
|
|||
| static size_t checkpos(lua_State *L, int narg) { | |||
| lua_Number n = luaL_checknumber(L, narg); | |||
| if (n >= 0 && n <= SIZE_MAX && n == (size_t)n) | |||
| if (n >= 0 && n <= (lua_Number) SIZE_MAX && n == (size_t)n) | |||
There was a problem hiding this comment.
This has come up before but I can't remember where. I'm just not sure this
is correct. The problem, as the compiler warns about, is that the value
of SIZE_MAX changes when it is implicitly cast to lua_Number. The
value it changes to is dependent on the version of lua that vis is being
built against. The goal is to make sure that n falls in the range
[0, SIZE_MAX]. This shouldn't depend on the lua version.
There was a problem hiding this comment.
I'm pretty sure your doubts about this code are right, with or without my cast (which just papers over the problem by eliminating the compiler warning).
Consider 64-bit machines. The issue is that SIZE_MAX is 0xFFFFFFFFFFFFFFFF on 64-bit machines, obviously requiring 64 bits to represent precisely. The default representation for lua_Number is double on 64-bit machines, which have 53 bits of precision. So SIZE_MAX can't be represented precisely as a double; it gets rounded up to the next higher integer that can be represented precisely as a double, namely 0x10000000000000000, which only requires 1 bit of significand precision as a double. The next such integer is 0x10000000000000000+0x800. 0x800 = 2048 = 2^(64-53). So when you get above (2^53)-1, things start to get imprecise and the kind of comparison the code does gets less and less reliable.
I think the solution to this is to check that n <=min(SIZE_MAX, (2^)-1) for whatever word size machine you are dealing with. This reduces the number of positions the editor can handle. Not a problem on 64-bit machines, where Lua uses doubles by default, so the limit is still astronomical. It could be a problem on 32-bit machines, but solvable where necessary by configuring Lua to use doubles, not floats.
I'll split the pull request into two tomorrow, as you requested.
modified: configure
Cast SIZE_MAX to lua_number before comparing it to n (a variable of that type) to avoid compile-time warning.
modified: vis-lua.c