Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Named function type returns #67

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
197 changes: 197 additions & 0 deletions docs/syntax-named-function-type-returns.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,197 @@
# Named function type returns

## Summary

In alignment with [named function type arguments](./syntax-named-function-type-args.md), introduce syntax to describe names of returned values for function types.

## Motivation

Luau does not include semantic information when describing a function returning a tuple of values. This is especially a problem when they can't be distinguished by type alone:

```Lua
-- are the euler angles returned in order of application, or alphabetical (XYZ) order?
calcEulerAngles: () -> (number, number, number)

-- which one is forearm? which one is upper arm?
solveIK: () -> (CFrame, CFrame)

-- is this quaternion in WXYZ or XYZW format?
toQuat: () -> (number, number, number, number)
```

We previously decided to solve a similar problem with function parameters by introducing named function type arguments, and saw multiple benefits:

- better documentation for uncommented functions
- lower propensity for names to fall out of sync with implementation
- improved comprehension for long and dense type signatures
- a consistent location for external tools to extract identifiers

This was all done with minimal extra design work, and done without introducing new responsibilities to other parts of the language.

```Lua
rotateEulerAngles: (y: number, x: number, z: number) -> ()

applyIK: (upperArm: CFrame, forearm: CFrame) -> ()

fromQuat: (x: number, y: number, z: number, w: number) -> ()
```

Function type returns face almost identical problems today, and stand to gain almost identical benefits.

By providing symmetry with named function type arguments, and introducing names for returns, we can:

- consistently apply the pattern of named list members, reducing the mental overhead of learners and less technical Luau users
- provide a single, well-defined location for placing human-readable identifiers usable by LSP inlay types, autofill, and hover tips
- encourage the colocation of semantic information alongside code to ensure documentation and runtime are updated in concert, and aren't prone to non-local editing mistakes
dphblox marked this conversation as resolved.
Show resolved Hide resolved
- improve the legibility of complex type signatures by adding human affordances that would not otherwise be present
- do all this with minimal extension to the language and perfect backwards compatibility

```Lua
calcEulerAngles: () -> (y: number, x: number, z: number)

solveIK: () -> (upperArm: CFrame, forearm: CFrame)

toQuat: () -> (x: number, y: number, z: number, w: number)
```

## Design

This proposal mirrors the existing named function type argument syntax and allows it to be used in return position. This allows returned tuples to be annotated with their contents.

This is added to all locations where return types can be defined:
```Lua
-- function type syntax
x: () -> (return1: number, return2: number)

-- function declaration
local function x(): (return1: number, return2: number)
```

This syntax is fully backwards compatible as only type names, generic names, or type packs are allowed in return position.

Broadly, return names behave identically to parameter names.

Function type comparisons will ignore the return names, this proposal doesn't change the semantics of the language and how typechecking is performed.

```Lua
-- names of returns are ignored
type Red = () -> (foo: number?, bar: number?)
type Blue = () -> (frob: number?, garb: number?)
local x: Red = something :: Blue
```

Assignments and other expressions will ignore the return names, with the same behaviour and type safety as today.

However, return names can be interpreted by linters to provide superior linting as a purely additive benefit:

```Lua
local function doStuff(): (red: number, blue: number)
-- ...
end

local function receiveStuff(blue: number, red: number)
-- ...
end

-- lint: `doStuff()` returns identically-named values in a different order, change these names to silence
local blue, red = doStuff()

-- lint: `doStuff` sends identically named & typed values to `receiveStuff`, but in the wrong order
receiveStuff(doStuff())
```



## Drawbacks

There is a philosophical disagreement over the purpose of the Luau static type system:
- This proposal was written on the grounds that Luau should have internal syntactic consistency and cross-pollination. Runtime, type checking, and documentation features should enmesh and enrich each other as a unified whole, to ensure everything is kept in sync and is easily comprehensible.
dphblox marked this conversation as resolved.
Show resolved Hide resolved
- The primary argument against this proposal is the belief that Luau's type checker should be a "pure proof system", cleanly separated from documentation concerns at every point, so that the type system can exist in a pure realm unconcerned with semantic information.
dphblox marked this conversation as resolved.
Show resolved Hide resolved

This feels like a core tension about the direction of type checking features as a whole. We should check in with the community on this.

There is a particular concern that introducing comprehension aids into the type language would imply meaning where there is none. Would users think that return names have functional meaning?

There are other places in Luau where we have previously introduced comprehension aids in the past:

```Lua
-- number literals could have been kept pure, but we chose to add meaningless delimiters for easier comprehension
local ONE_BILLION = 1000000000
local ONE_BILLION = 1_000_000_000

-- generics could have been referenced abstractly, but we chose to give them human-readable names
type Foo<2> = (<1>, <2>) -> <1> | <2>
type Foo<OK, Fallback> = (OK, Fallback) -> OK | Fallback

-- parameters in function types didn't have to have named parameters, but we decided to add them
type Func = (foo: number, bar: number) -> ()
```

In particular, it is established convention that Luau does not rearrange the contents of lists bounded by parentheses, even when list items are named:

```Lua
local function poem(red: number, blue: number)
print("roses are", red, "violets are", blue)
end

local blue, red = "blue", "red"

poem(blue, red) --> roses are blue violets are red
```

So, this proposal posits that there is already established precedent for such features, and that users understand how comprehension aids function in Luau today.

A common concern is whether these comprehension aids would mislead people into believing that names are significant when considering type compatibility.
Copy link

@hgoldstein hgoldstein Nov 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IME: the way programmers interact with type systems is generally "I'm going to write code how I want until there are no errors." This is especially true for gradual type systems: they're being applied to code written without static typing in mind (trying to run mypy or Pyre on a legacy codebase can be an exercise in frustration because of this).

If I had to guess: I would expect that someone writes code like:

local function maybeMap<T, U>(elem: T?, f: (item: T) -> (result: U)): U?
  -- ...
end

local function blah(foo: number): (bar: string)
  -- ...
end

local function maybeStringify(x: number?): string?
  return maybeMap(x, blah)
end

... and then realizes later there was a mismatch. Or at least the vast majority of folks will do so.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what mismatch you mean here. I noticed the example wouldn't typecheck because maybeMap should prob return U? rather than U, but that's not got anything to do with return names.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what mismatch you mean here.

The signature of maybeMap states it takes a function f of type (item: T) -> (result: U), but when we invoke it, we do so with blah who's type is (foo: number) -> (bar: string). The mismatch is in the names of the params / return types.

I noticed the example wouldn't typecheck because maybeMap should prob return U? rather than U, but that's not got anything to do with return names.

Yeah that has to do with me making a small mistake 😅

Copy link
Author

@dphblox dphblox Nov 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see. Sometimes I do this intentionally, where e.g. a generic maybeMap function uses generic naming, but when I define a mapping callback inline, it might use names more descriptive based on local information, e.g.:

local maybeChar = tryGetCharacter()

local maybeHumanoid = maybeMap(maybeChar, function(char)
   return char:FindFirstChildWhichIsA("Humanoid")
end)

So I wouldn't think a discrepancy like that is always a problem.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's my point: unless you have named parameters, people don't try to match the naming.

OCaml has named parameters that start with tildes, they're a part of the type of the function:

(* math.mli *)
val divide : ~dividend:int -> ~divisor:int -> int
(* math.ml *)
let divide ~dividend ~divisor = dividend / divisor
let halve number = divide ~dividend:number ~divisor:2

There's support for punning, so one can write code like:

let f ast_node = (* do something with an ast_node here *) in
(* `~f` is both the named parameter _and_ the local variable *)
List.map ~f list_of_ast_nodes 

Copy link
Author

@dphblox dphblox Nov 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. So I think that starts to drift into somewhat adjacent territory around things like named parameter support (as opposed to today's positional parameters) which is out of scope since that's a much heavier change.

I would think of both argument names and return names as "sensible defaults" which document the type generically, without restricting how they're used downstream - it manifests mostly in LSP features.

It's OK for people to not match the names exactly (though it's certainly curious if they rearrange them, because that's probably a mistake, hence the lints).


However, we already have features that clearly indicate we don't do this.

```Lua
-- names of generics are ignored
type Foo = <A, B>(A) -> B
type Bar = <X, Y>(X) -> Y
local x: Foo = something :: Bar
dphblox marked this conversation as resolved.
Show resolved Hide resolved

-- names of parameters are ignored
type Red = (foo: number?, bar: number?) -> number
type Blue = (frob: number?, garb: number?) -> number
local x: Red = something :: Blue
hgoldstein marked this conversation as resolved.
Show resolved Hide resolved
```

So, there is no reason why named function type returns would be any different.
dphblox marked this conversation as resolved.
Show resolved Hide resolved

The remaining questions are non-technical:

- Are Luau types a proof system, or a documentation system, or both? What approach makes sense?
- Is it right for types to be annotated for the comprehension of human readers and writers, or is this excess that doesn't need to be there?
- What does it mean for an identifier to be meaningful? Is it meaningful if linters parse for it, or only if it functionally changes runtime behaviour?

Discussion is invited on these questions - this proposal does not prescribe an answer, but consensus between the community and Luau should be reached, and can help direct future proposals.

## Alternatives

### Add meaning to comments

In response to this proposal, the idea of making comments meaningful was proposed. Instead of being treated as non-functional text, comments would be interpreted using a kind of "documentation syntax". Concrete facts about parameters and return types, including identifiers, would be inferred from them.

This works well for verbose documentation such as explanations or diagrams, which would be inappropriate to try and embed inline. However, there are a number of drawbacks to this approach too.

From discussions with users of Luau, uncommented code is still often written with Luau types for LSP benefits such as good autocomplete and improved linting, especially during accelerated phases of development such as prototyping. Whether these people would adopt comments for better LSP features is an open question - we should be mindful of the friction required to personalise LSP results.

This also raises further non-technical questions:

- should comments be given meaning? are they meaningful if linters parse their contents to provide lints? if a parser skips comments, are they losing non-decorative information?
- what documentation syntax do we use? what rules does it follow? how does it fit into Luau's overall design, if it fits at all?
- what is the scope of documentation syntax? where are the edges of its responsibilities?
- what about previous proposals where we decided against placing features into comments? what do they recommend here?

### Don't do anything

It is technically possible for people to "annotate" returned values today using comments. However, there are multiple very tangible downsides to this.

There is no widely-agreed-upon convention for comment format, and as such, any LSP, linter, or docsite generator looking to adopt standardised rules for interpreting returned names would face an uphill battle.

For a community project to implement support, they would need to discover all of the formats for documentation comments in use by Luau developers, and implement support for all of them by hand. With no official specification to go off, this would invite fragmentation.

There is a very real possibility that not implementing *any* standard way of annotating return type names will discourage people from documenting them at all, reducing the quality of documentation in the Luau ecosystem.

The upshot is that Luau would remain unopinionated and free to ignore comments completely. We punt the problem to the community and all consumers of Luau, who will have to invent their own methods of documenting this info if it is desired.