Some improvement ideas for the future

[robotboy655]

Sorry to pile on these in 1 issue, I just need to write these down before I forget:

• TOOL_Hooks and Strutures/TOOL pages conflict with their definition for the TOOL global. There are probably other similar cases.
  • This results in files that define TOOLs to not inherit the Tool class (because the structure is written first), causing many "Undefined Symbol" warnings
• We need to find a good way to import Vector/Angle/Color/etc fields (Vector.x,Vector.y, etc) from the wiki
  • Same goes for metamethods, like _unm(). Currently negating a vector is seen as it turning into a number. Is this an issue with LuaLS itself?
• Find a good format to define function callbacks on the wiki, so they can be imported here and other similar projects
  • Same goes for arguments that accept multiple types

edit by luttje:

• Perhaps find a good format to include specifications for table element types? E.g: Player[] as the return type of player.GetAll()

[luttje]

No worries, it's fine to pile it in one issue. It's workable this way, since this isn't too big a project.

You may already have considered ideas like these, but here's my thoughts on the topic of annotating multiple types and callbacks on the wiki.

With backwards compatibility

• For multiple possible types the wiki could add an attribute to arg, like alternativeTypes="table", e.g:

  <arg name="flags"
       type="number"
       alternativeTypes="table"
       default="FCVAR_NONE">Flags of the convar, see <page>Enums/FCVAR</page>, either as bitflag or as table.</arg>

  One could even consider allowing multiple types to be specified in that new attribute, by comma separation or something, for even more alternate types on an attribute.
• For callback functions I'd approach it similarly, just adding an attribute to the arg element, e.g:

  <arg name="callback"
       type="function"
       functionArguments="Player ply, string cmd, table args, string argStr">The function to run when the...</arg>

This would ensure backwards compatibility with anyone relying on type or to contain only one value.

Some downsides:

• Hard to add descriptions to alternative types or to function arguments in this way.
  • This seems too cumbersome to ask wiki editors to work with: functionArguments="Player ply the player that ran the concommand, string cmd The command that was ran by the player, table args etc..., string argStr etc..."
• I'm sure there's a function in gmod that changes behavior depending on which type is passed first. That would be hard to document with the above proposal (because adding descriptions is hard this way)

Breaking changes

I guess the "markup/XML way" for this would be adding child elements that annotate more data. However I don't see a way to implement that without omitting backward compatibility for anyone scraping the wiki. Additionally it would require a bunch of work on all sides.

I do believe something like this would work best for the wiki. Since data structured like this could be used by the wiki to generate HTML for developers consulting the wiki manually.

Some mockups of how that could look:

• The wiki could detect if an arg element has a type attribute and default to current behaviour. However if it's missing it'll look inside the element for either:
  • argTypes which contains a set of argType elements of which the user can choose one for this argument:
    e.g: for the https://wiki.facepunch.com/gmod/Global.CreateConVar flags argument:

  <arg name="flags"
     default="FCVAR_NONE">
     <argTypes>
        <argType type="number">
          <description>Flags of the convar, see <page>Enums/FCVAR</page></description>
        </argType>
        <argType type="table">
          <description>Flags of the convar, see <page>Enums/FCVAR</page> as a table</description>
        </argType>
    </argTypes>
  </arg>

  • argType which contains detailed information on an argument
    e.g: for https://wiki.facepunch.com/gmod/concommand.Add callback:

  <arg name="callback">
    <argType type="function">
      <description>The function to run when the command is ran.</description>
      <args>
        <arg name="ply"
            type="Player">
          <description>The player who ran the command.</description>
        </arg>
        <arg name="cmd"
            type="string">
          <description>The command that was ran.</description>
        </arg>
        <arg name="args"
            type="table">
          <description>The arguments that were passed to the command.</description>
        </arg>
        <arg name="argStr"
            type="string">
          <description>The arguments that were passed to the command as a string.</description>
        </arg>
      </args>
    </argType>
  </arg>

Combination without breaking changes

The wiki could add a new element in root, e.g: <metadata> which contains elements with an id/ref. A new attribute on arg could point to that to indicate there's more information there, e.g:

...
<args>
   <arg name="callback"
        type="function"
        metadata="callback">...<arg>
   ...
</args>
...
<metadata>
   <functionParameters id="callback">...</functionParameters>
</metadata>

However this last option is really oriented on those scraping the wiki, whilst the before mentioned options could also be used by the wiki itself to generate useful html for users of the wiki.

[TIMONz1535]

fields and metamethods can be easily added manually
but you need to add some kind of tags to parse the wiki tables (because its just a raw text)

---@class VMatrix
---@operator add(VMatrix): VMatrix
---@operator mul(Vector): Vector
---@operator mul(VMatrix): VMatrix
---@operator sub(VMatrix): VMatrix
---@operator unm: VMatrix

---@class Angle
---@field p number
---@field y number
---@field r number
---@field pitch number
---@field yaw number
---@field roll number
---@field x number
---@field z number
---@field [1] number
---@field [2] number
---@field [3] number
---@operator add(Angle): Angle
---@operator div(number): Angle
---@operator mul(number): Angle
---@operator sub(Angle): Angle
---@operator unm: Angle

---@class Vector
---@field x number
---@field y number
---@field z number
---@field [1] number
---@field [2] number
---@field [3] number
---@operator add(Vector): Vector
---@operator div(number|Vector): Vector
---@operator mul(number|Vector): Vector
---@operator sub(Vector): Vector
---@operator unm: Vector

in general there are only Vector, Angle and VMatrix. It is unlikely that anyone will ever change them.

[luttje]

@TIMONz1535 Thanks for thinking with us.

I think you've got a point that the way Vector, Color, etc. work is unlikely to change. Perhaps our attention is better spent elsewhere and we can just implement your manually written up definitions in the meantime.

Parsing the markdown table like Vector has is doable, but seems like a waste of time.

[robotboy655]

I have implemented a scraper friendly definitions for function callbacks:
https://wiki.facepunch.com/gmod/PathFollower:Compute
https://wiki.facepunch.com/gmod/concommand.Add

And for function overloads:
https://wiki.facepunch.com/gmod/Global.Angle

It's currently only on these pages as a test for now.

[luttje]

Wow awesome, thats super useful and a clean implementation! Thanks so much for your continuous work on all this @robotboy655

[robotboy655]

I have been thinking about how to deal with AccessorFunc and Entity:NetworkVar creating functions that are not being seen by the language server, and my conclusion is the following:

1. Either we open an issue/PR for this in the VSCode extension repo. I have noticed that simply doing this:

function Entity:NetworkVar(type, slot, name, extended)
    self.SetBallSize = function( s, test ) end
    self.GetBallSize = function( s ) return false end
end

Makes it detect the custom functions, but doing this:

function Entity:NetworkVar(type, slot, name, extended)
    self["Set" .. "BallSize"] = function( s, test ) end
    self["Get" .. "BallSize"] = function( s ) return false end
end

does not. It might be possible to fix on their end.

2. Or we try and see if using plugins (https://luals.github.io/wiki/plugins/) could be a solution

[luttje]

@robotboy655 I had been looking at the LuaLS plugins (for #50), but sadly they're very limited and only seem to output Lua to a LOGPATH/diffed.lua, from their wiki:

Introduction

Plugins allow you to create a custom syntax that will then be output to a separate file. They cannot be used to report custom diagnostics.

To me, this is a super confusing functionality in LuaLS. If the diffed file was output to a location that would be indexed for LuaLS autocomplete this would make sense, but I can't find that that's the case.

[luttje]

@robotboy655 I'd like to correct my previous statement. It seems I (and the LuaLS docs) were wrong about plugins not being able to customize diagnostics:

1. I created a plugin.lua in the .vscode directory of my project:

   function OnTransformAst(uri, ast)
       if (not uri:match("gamemode/cl_init.lua$")) then
           return
       end
   
       -- I'm not sure what the structure of the AST is, so I'm simply brute forcing my way to demonstrate this is useful
       local function changeAst(ast, level)
   	    if (level > 3) then
   		    return
   	    end
   
           for k, v in pairs(ast) do
               if (type(v) == "table") then
                   changeAst(v, level + 1)
               elseif (v == "Xtest") then
   			    ast[k] = "Changed"
               end
           end
       end
   
       for k, v in ipairs(ast) do
   	    changeAst(v, 0)
       end
   end

2. I added "Lua.runtime.plugin": ".vscode/plugin.lua" to .vscode/settings.json
3. I stop and restart the Lua Language server through the command palette (or by restarting VSCode)
4. I added a function named 'Xtest' in my gamemode/cl_init.lua
5. Hovering over the function results in:
   

This means we can modify diagnostics with plugins.

Sadly I can't find a way to automatically load the plugin from the glua-api-snippets addon. Placing the plugin.lua in the addon folder doesn't seem to work for me (nor any of the other addons that are in the LuaLS-addons repo)

[TIMONz1535]

since 3.8.0 they added

NEW plugin: add OnTransFormAst interface (@fesily)
NEW plugin: add OnNodeCompileFunctionParam interface (@fesily)
NEW plugin: add ResolveRequire interface (@Artem Dzhemesiuk)
NEW plugin: support multi plugins (@fesily)
setting: Lua.runtime.plugin can be string|string[]
setting: Lua.runtime.pluginArgs can be string[]|table<string, string>

also some interest part LuaLS/lua-language-server#2484

NEW generic pattern (@fesily)

---@generic T
---@param t Cat.`T`
---@return T
local function f(t) end

local t = f('Smile') --> t is `Cat.Smile`

I thought this was what we needed, but it just refers to the existing syntax in the class. That is, it cannot create/modify class fields on the fly.

Well, but I realized that even before 3.8.0 it was possible to get any class through a @generic

---@generic T
---@param metaName `T` The object type to retrieve the meta table of.
---@return T # The corresponding meta table.
function _G.FindMetaTable(metaName) end

local entMeta = FindMetaTable("Entity") -- entMeta: Entity

[luttje]

I've added a small note to the original issue above, it's about adding a way to specify table element types. Right now functions like player.GetAll are documented to return table on the wiki instead of the more complete Player[]. The latter would greatly improve typing on subsequent loops, because lua-language-server would type the array index and elements properly.

Currently the wiki has a structure like this for the return type of player.GetAll

<rets>
	<ret name="" type="table">All <page>Player</page>s currently in the server.</ret>
</rets>

If Rubat has some time in the future and feels that this is worth it, perhaps they could add some attributes similar to this:

<rets>
	<ret name="" type="table" keytype="number" valuetype="Player">All <page>Player</page>s currently in the server.</ret>
</rets>

Perhaps even a sequential="true" so we can differentiate a sequential table like Player[] from a keyvalue table which can be unsequential like table<number, Player>

[luttje]

Another slightly related note, but it would be cool if functions like util.TraceLine could be documented about their structs:

<args>
	<arg name="traceConfig" type="table" struct="Trace">
		A table of data that configures the Trace.  
			
		For the table's format and available options see the <page>Structures/Trace</page> page.
	</arg>
</args>
<rets>
	<ret name="" type="table" struct="TraceResult">
		A table of information detailing where and what the Trace line intersected, or `nil` if the trace is being done before the <page>GM:InitPostEntity</page> hook.

		For the table's format and available options see the <page>Structures/TraceResult</page> page.
	</ret>
</rets>

[TIMONz1535]

I use strict integer types "type.castNumberToInteger": false,, and noticed that some Lua builtin function differ with plugin declarations

https://wiki.facepunch.com/gmod/math.ceil

---[SHARED AND MENU] Ceils or rounds a number up.
---
---[(View on wiki)](https://wiki.facepunch.com/gmod/math.ceil)
---@param number number The number to be rounded up.
---@return number # ceiled numbers
function math.ceil(number) end

vs sumneko's

---
---Returns the smallest integral value larger than or equal to `x`.
---
---[View documents](command:extension.lua.doc?["en-us/51/manual.html/pdf-math.ceil"])
---
---@param x number
---@return integer
---@nodiscard
function math.ceil(x) end

there is no integer for wiki, but we can specify the custom type as you suggested above
oh, there's also @nodiscard