diff --git a/docs/docs.md b/docs/docs.md index 25dabd93d..0f9d21622 100644 --- a/docs/docs.md +++ b/docs/docs.md @@ -6,14 +6,14 @@ - [C Types.](#c-types) - [Control Flow.](#control-flow) - [Functions.](#functions) -- [Modules.](#modules) -- [FFI.](#ffi) +- [Memory.](#memory) - [Error Handling.](#error-handling) - [Concurrency.](#concurrency) - [Dynamic Typing.](#dynamic-typing) - [Metaprogramming.](#metaprogramming) +- [Modules.](#modules) +- [FFI.](#ffi) - [libcyber.](#libcyber) -- [Memory.](#memory) - [CLI.](#cli) @@ -3810,29 +3810,361 @@ void myNodeFinalizer(CLVM* vm, void* obj) { # Memory. -* [ARC.](#arc) - * [Reference counting.](#reference-counting) + + +
+ +* [Structured memory.](#structured-memory) + * [Value ownership.](#value-ownership) + * [Copy semantics.](#copy-semantics) + * [Cloning.](#cloning) + * [Moving.](#moving) + * [References.](#references) + * [Exclusive reference.](#exclusive-reference) + * [`self` reference.](#self-reference) + * [Lifted values.](#lifted-values) + * [Deferred references.](#deferred-references) + * [Implicit lifetimes.](#implicit-lifetimes) + * [Reference lifetimes.](#explicit-reference-lifetimes) + * [Shared ownership.](#shared-ownership) + * [Deinitializer.](#deinitializer) + * [Pointer interop.](#pointer-interop) + + +* [Automatic memory.](#automatic-memory) + * [ARC.](#arc) * [Object destructor.](#object-destructor) - * [Optimizations.](#optimizations) + * [Retain optimizations.](#retain-optimizations) * [Closures.](#closures-1) * [Fibers.](#fibers-1) -* [Heap.](#heap) - * [Weak references.](#weak-references) - * [Cycle detection.](#cycle-detection) + * [Heap objects.](#heap-objects) + * [GC.](#gc) * [Manual memory.](#manual-memory) * [Memory allocations.](#memory-allocations) * [Runtime memory checks.](#runtime-memory-checks) +
[^top](#table-of-contents) -Cyber provides memory safety by default. +Cyber provides memory safety by default with structured and automatic memory. +Manual memory is also supported but discouraged. + +## Structured memory. +Cyber uses single value ownership and variable scopes to determine the lifetime of values. +When lifetimes are known at compile-time, the memory occupied by values do not need to be manually managed which prevents memory bugs such as: +* Use after free. +* Use after invalidation. +* Free with wrong allocator. +* Double free. +* Memory leaks. +* Null pointer dereferencing. -## ARC. -Cyber uses ARC or automatic reference counting to manage memory. -ARC is deterministic and has less overhead compared to a tracing garbage collector. Reference counting distributes memory management, which reduces GC pauses and makes ARC suitable for realtime applications. One common issue in ARC implementations is reference cycles which Cyber addresses with [Weak References](#weak-references) and it's very own [Cycle Detection](#cycle-detection). +At the same time, structured memory allows performant code to be written since it provides safe semantics to directly reference values and child values. -### Reference counting. -In Cyber, there are [primitive and object](#basic-types) values. Primitives don't need any memory management, since they are copied by value and no heap allocation is required (with the exception of primitives being captured by a [closure](#closures-1). +### Value ownership. +Every value in safe memory has a single owner. +An owner can be a variable that binds to a value. Otherwise, the owner can be a parent value or the value itself. +The owner is responsible for deinitializing or dropping the value when it has gone out of scope (no longer reachable). +For example, at the end of the block, a variable can no longer be accessed so it drops the value that it owns: +```cy +var a = 123 +print a --> 123 +-- Deinit `a`. +``` +In this case, there is nothing to deinitialize since the value is an integer. + +If the value was a `String`, the deinit logic would release (-1) on a reference counted byte buffer since strings are just immutable views over byte buffers: +```cy +var a = 'hello' +print a --> hello +-- Deinit `a`. +-- `a.buf` is released. +-- `a.buf` is freed. +``` +Since the string buffer's reference count reaches 0, it's freed as well. + +Finally, let's take a look at `ListValue` which manages a dynamically sized array of elements: +```cy +var a = ListValue[int]{1, 2, 3} +print a --> {1, 2, 3} +-- Deinit `a`. +-- `a.buf` is freed. +``` +When `a` is deinitialized, the buffer that holds the 3 integer elements is freed. +You may have surmised that it's named `ListValue` because it's a value type (it can only be passed around by copying itself). The object type, [`List`](#lists), wraps `ListValue` and can be passed around by reference. + +The concept of a value having a single owner is very simple yet powerful. +A value can represent any data structure from primitives to dynamically allocated buffers. +A value always knows **how** to deinitialize itself, and the owner knows **when** to deinitialize the value. +Later, we'll see that this same concept also applies to [shared ownership](#shared-ownership). + +### Copy semantics. +By default, values are passed around by copying (shallow copying), but now all values can perform a copy. + +A primitive, such as an integer, can always be copied: +```cy +var a = 123 +var b = a +-- Deinit `b`. +-- Deinit `a`. +``` +After a copy, a new value is created and given the owner of `b`. At the end of the block, both `a` and `b` are deinitialized (which does nothing since they are just primitives). + +Strings are also copyable since they are immutable views over byte buffers: +``` +var a = 'hello' +var b = a +-- Deinit `b`. +-- `b.buf` is released. +-- Deinit `a`. +-- `a.buf` is released. +-- `a.buf` is freed. +``` +The copy `b` also reuses the byte buffer of `a` by retaining (+1) on the reference counted byte buffer. The byte buffer is finally freed once there are no references pointing to it. + +Unlike the integer and string, a `ListValue` can not be copied since doing so requires duping a heap allocated buffer which is considered expensive: +```cy +var a = ListValue[int]{1, 2, 3} +var b = a --> error: Can not copy `ListValue`. Can only be cloned or moved. +``` +Instead `a` can only be [cloned](#cloned) or [moved](#moving). + +By default, a declared value type is copyable if all of it's members are also copyable: +```cy +type Foo struct: + a int + b String + +var a = Foo{a=123, b='hello'} +var b = a +``` +Since integers and strings are both copyable, `Foo` is also copyable. + +`Foo` is non-copyable if it contains at least one non-copyable member: +```cy +type Foo struct: + a int + b String + c ListValue[int] + +var a = Foo{a=123, b='hello'} +var b = a --> error: Can not copy `Foo`. Can only be moved. +``` + +`Foo` is also non-copyable if it contains unsafe types such as pointers or pointer slices: +```cy +type Foo struct: + a int + b String + c *Bar + d [*]float +``` + +`Foo` can implement `Copyable` to override the default behavior and define it's own copy logic: +```cy +type Foo struct: + with Copyable + a int + b String + c *Bar + d [*]float + + func copy(self) Foo: + return .{ + a = self.a, + b = self.b, + c = self.c, + d = self.d, + } +``` + +Likewise, `Foo` can implement `NonCopyable` which indicates that it can never be copied: +```cy +type Foo struct: + with NonCopyable + a int + b String +``` + +### Cloning. +Some value types are not allowed to be copied by default and must be cloned instead: +```cy +var a = ListValue[int]{1, 2, 3} +var b = a.clone() +``` + +Any `Copyable` type is also `Cloneable`. For example, performing a clone on an integer will simply perform a copy: +```cy +var a = 123 +var b = a.clone() +``` + +A value type can implement `Cloneable` to override the default behavior and define it's own clone logic: +```cy +type Foo struct: + with Cloneable + a int + b String + + func clone(self) Foo: + return .{ + a = self.a + 1, + b = self.b, + } +``` + +Likewise, `Foo` can implement `NonCloneable` which indicates that it can never be cloned: +```cy +type Foo struct: + with NonCloneable + a int + b String +``` + +### Moving. +Values can be moved, thereby transfering ownership from one variable to another: +```cy +var a = 123 +var b = move a +print a --> error: `a` does not own a value. +``` + +Some types such as `ListValue` can not be passed around by default without moving (or cloning) the value: +```cy +var a = ListValue[int]{1, 2, 3} +print computeSum(move a) --> 6 +``` +In this case, the list value is moved into the `computeSum` function, so the list is deinitialized in the function before the function returns. + +### References. +References are safe pointers to values. +Unlike unsafe pointers, a reference is never concerned with when to free or deinitialize a value since that responsibility always belongs to the value's owner. +They are considered safe pointers because they are guaranteed to point to their values and never outlive the lifetime of their values. + +References grant **stable mutability** which allows a value to be modified as long as it does not invalidate other references. +**Multiple** references can be alive at once as long as an [exclusive reference](#exclusive-reference) is not also alive. + +The `&` operator is used to obtain a reference to a value: +```cy +var a = 123 +var ref = &a +ref.* = 234 +print a --> 234 +``` + +A reference can not outlive the value it's referencing: +```cy +var a = 123 +var ref = &a +if true: + var b = 234 + ref = &b --> error: `ref` can not oulive `b`. +``` + +A reference type is denoted as `&T` where `T` is the type that the reference points to: +```cy +var a = 123 + +func inc(a &int): + a.* = a.* + 1 +``` + +References allow stable mutation: +```cy +var a = ListValue[int]{1, 2, 3} +var third = &a[2] +third.* = 300 +print a --> {1, 2, 300} +``` +The element that `third` points to can be mutated because it does not invalidate other references. + +References however can not perform a unstable mutation. +An unstable mutation requires an exclusive reference: +```cy +var a = ListValue[int]{1, 2, 3} +var ref = &a +ref.append(4) --> error: Expected exclusive reference. +``` + +### Exclusive reference. +An exclusive reference grants **full mutability** which allows a value to be modified even if could potentially invalidate unsafe pointers. + +A **single** exclusive reference can be alive as long as no other references are also alive. +Since no other references (safe pointers) are allowed to be alive at the same time, no references can become invalidated. + +The `&!` operator is used to obtain an exclusive reference to a value. +An exclusive reference type is denoted as `&!T` where `T` is the type that the reference points to. + +`ListValue` is an example of a type that requires an exclusive reference for operations that can resize or reallocate its dynamic buffer: +```cy +var a = ListValue[int]{1, 2, 3} +a.append(4) +print a --> {1, 2, 3, 4} +``` +Note that invoking the method `append` here automatically obtains an exclusive reference for `self` without an explicit `&!` operator. + +If another reference is alive before `append`, the compiler will not allow an exclusive reference to be obtained from `a`. +Doing so would allow `append` to potentially reallocate its dynamic buffer, thereby invalidating other references: +```cy +var a = ListValue[int]{1, 2, 3} +var third = &a[2] +a.append(4) --> error: Can not obtain exclusive reference, `third` is still alive. +print third +``` + +### `self` reference. +By default `self` has a type of `&T` when declared in a value type's method: +```cy +type Pair struct: + a int + b int + + func sum(self) int: + return self.a + self.b +``` + +If `self` requires an exclusive reference, then it must be prepended with `!`: +```cy +type Pair struct: + a int + b int + + func sum(!self) int: + return self.a + self.b +``` + +Invoking methods automatically obtains the correct reference as specified by the method: +```cy +var p = Pair{a=1, b=2} +print p.sum() --> 3 +``` + +### Lifted values. +> _Planned Feature_ + +### Deferred references. +> _Planned Feature_ + +### Implicit lifetimes. +> _Planned Feature_ + +### Reference lifetimes. +> _Planned Feature_ + +### Shared ownership. +> _Planned Feature_ + +### Deinitializer. +> _Planned Feature_ + +### Pointer interop. +> _Planned Feature_ + +## Automatic memory. +Cyber uses an ARC/GC hybrid to automatically manage objects instantiated from object types. Value types typically do not need to be automatically managed unless they were lifted by a [closure](#closures-1) or a dynamic container. + +### ARC. +ARC also known as automatic reference counting is deterministic and has less overhead compared to a tracing garbage collector. Reference counting distributes memory management, which reduces GC pauses and makes ARC suitable for realtime applications. One common issue in ARC implementations is reference cycles which Cyber addresses with a [GC](#gc) supplement when it is required. Objects are managed by ARC. Each object has its own reference counter. Upon creating a new object, it receives a reference count of 1. When the object is copied, it's **retained** and the reference count increments by 1. When an object value is removed from it's parent or is no longer reachable in the current stack frame, it is **released** and the reference count decrements by 1. @@ -3848,8 +4180,12 @@ If the destructor is invoked by the GC instead of ARC, cyclable child references Since objects freed by the GC either belongs to a reference cycle or branched from one, the GC will still end up invoking the destructor of all unreachable objects. This implies that the destructor order is not reliable, but destructors are guaranteed to be invoked for all unreachable objects. -### Optimizations. -The compiler can reduce the number of retain/release ops since it can infer value types even though they are dynamically typed to the user. Arguments passed to functions are only retained depending on the analysis from the callsite. +### Retain optimizations. +When the lifetime of an object's reference is known on the stack, a large amount of retain/release ops can be avoided. +For example, calling a function with an object doesn't need a retain since it is guaranteed to be alive when the function returns. +This leaves only cases where an object must retain to ensure correctness such as escaping the stack. + +When using dynamic types, the compiler can omit retain/release ops when it can infer the actual type even though they are dynamically typed to the user. ### Closures. When primitive variables are captured by a [closure](#closures), they are boxed and allocated on the heap. This means they are managed by ARC and cleaned up when there are no more references to them. @@ -3857,17 +4193,18 @@ When primitive variables are captured by a [closure](#closures), they are boxed ### Fibers. [Fibers](#fibers) are freed by ARC just like any other object. Once there are no references to the fiber, it begins to release it's child references by unwinding it's call stack. -## Heap. -Many object types in Cyber are small enough to be at or under 40 bytes. To take advantage of this, Cyber can reserve object pools to quickly allocate and free these small objects with very little bookkeeping. Bigger objects are allocated and managed by `mimalloc` which has proven to be a fast and reliable general-purpose heap allocator. +### Heap objects. +Many object types are small enough to be at or under 40 bytes. To take advantage of this, object pools are reserved to quickly allocate and free these small objects with very little bookkeeping. Bigger objects are allocated and managed by `mimalloc` which has proven to be a fast and reliable general-purpose heap allocator. -### Weak references. -> _Planned Feature_ +### GC. +The garbage collector is only used if the program may contain objects that form reference cycles. This property is statically determined by the compiler. Since ARC frees most objects, the GC's only responsibility is to free abandoned objects that form reference cycles. +This reduces the amount of work for GC marking since only cyclable objects (objects that may contain a reference cycle) are considered. + +Weak references are not supported for object types because objects are intended to behave like GC objects (the user should not be concerned with reference cycles). If weak references do get supported in the future, they will be introduced as a `Weak[T]` type that is used with an explicit reference counted `Rc[T]` type. -### Cycle detection. -The cycle detector is also considered a GC and frees abandoned objects managed by ARC. Although weak references can remove cycles altogether, Cyber does not force you to use them and provides a manual GC as a one-time catch all solution. -> _Incomplete Feature: Only the main fiber stack is cleaned up at the moment._ +Currently, the GC can be manually invoked. However, the plan is for this to be automatic by either running in a separate thread or per virtual thread by running the GC incrementally. -To invoke the GC, call the builtin function: `performGC`. +To invoke the GC, call the builtin function: `performGC`. *Incomplete Feature: Only the main fiber stack is cleaned up at the moment.* ```cy func foo(): -- Create a reference cycle. diff --git a/docs/gen-docs.cy b/docs/gen-docs.cy index 8cd80bb38..0db9d54b7 100644 --- a/docs/gen-docs.cy +++ b/docs/gen-docs.cy @@ -93,7 +93,7 @@ hljs.registerLanguage('cy', function() { keyword: [ 'func', 'mod', 'for', 'coinit', 'coresume', 'coyield', 'use', 'await', 'context', 'return', 'if', 'else', 'as', 'while', 'var', 'let', 'dynobject', 'object', 'struct', 'cstruct', 'with', 'caught', - 'break', 'continue', 'switch', 'pass', 'or', 'and', 'not', 'is', 'error', 'throws', + 'break', 'continue', 'switch', 'pass', 'or', 'and', 'not', 'is', 'error', 'throws', 'move', 'true', 'false', 'none', 'throw', 'try', 'catch', 'recover', 'enum', 'type', 'case', 'trait' ], type: [ diff --git a/docs/style.css b/docs/style.css index 8298c0c61..95a1c2424 100644 --- a/docs/style.css +++ b/docs/style.css @@ -313,6 +313,11 @@ blockquote p { padding: 0px; } +p { + margin-block-start: 1.3em; + margin-block-end: 1.3em; +} + blockquote em { color: #fb6767; } diff --git a/src/ast.zig b/src/ast.zig index 5ca2df592..00a851083 100644 --- a/src/ast.zig +++ b/src/ast.zig @@ -44,7 +44,6 @@ pub const NodeType = enum(u7) { enumMember, error_lit, expandOpt, - expand_ptr, exprStmt, falseLit, forIterStmt, @@ -75,9 +74,15 @@ pub const NodeType = enum(u7) { octLit, opAssignStmt, passStmt, - pointer_slice, + ptr, + ptr_slice, range, raw_string_lit, + + // Ref type or address of operator. + ref, + + ref_slice, returnExprStmt, returnStmt, root, @@ -114,7 +119,7 @@ pub const AttributeType = enum(u8) { host, }; -const PointerSlice = struct { +const PtrSlice = struct { elem: *Node align(8), pos: u32, }; @@ -124,7 +129,17 @@ const ExpandOpt = struct { pos: u32, }; -const ExpandPtr = struct { +const RefSlice = struct { + elem: *Node align(8), + pos: u32, +}; + +const Ref = struct { + elem: *Node align(8), + pos: u32, +}; + +const Ptr = struct { elem: *Node align(8), pos: u32, }; @@ -586,6 +601,8 @@ fn NodeData(comptime node_t: NodeType) type { .await_expr => AwaitExpr, .binExpr => BinExpr, .binLit => Span, + .ref => Ref, + .ref_slice => RefSlice, .breakStmt => Token, .caseBlock => CaseBlock, .callExpr => CallExpr, @@ -610,7 +627,6 @@ fn NodeData(comptime node_t: NodeType) type { .enumMember => EnumMember, .error_lit => Span, .expandOpt => ExpandOpt, - .expand_ptr => ExpandPtr, .exprStmt => ExprStmt, .falseLit => Token, .forIterStmt => ForIterStmt, @@ -641,7 +657,8 @@ fn NodeData(comptime node_t: NodeType) type { .octLit => Span, .opAssignStmt => OpAssignStmt, .passStmt => Token, - .pointer_slice => PointerSlice, + .ptr => Ptr, + .ptr_slice => PtrSlice, .range => Range, .raw_string_lit => Span, .returnExprStmt => ReturnExprStmt, @@ -747,9 +764,7 @@ pub const Node = struct { .enumMember => self.cast(.enumMember).name.pos(), .error_lit => self.cast(.error_lit).pos-6, .expandOpt => self.cast(.expandOpt).pos, - .expand_ptr => self.cast(.expand_ptr).pos, .exprStmt => self.cast(.exprStmt).child.pos(), - .pointer_slice => self.cast(.pointer_slice).pos, .falseLit => self.cast(.falseLit).pos, .floatLit => self.cast(.floatLit).pos, .forIterStmt => self.cast(.forIterStmt).pos, @@ -779,8 +794,12 @@ pub const Node = struct { .octLit => self.cast(.octLit).pos, .opAssignStmt => self.cast(.opAssignStmt).left.pos(), .passStmt => self.cast(.passStmt).pos, + .ptr => self.cast(.ptr).pos, + .ptr_slice => self.cast(.ptr_slice).pos, .range => self.cast(.range).pos, .raw_string_lit => self.cast(.raw_string_lit).pos, + .ref => self.cast(.ref).pos, + .ref_slice => self.cast(.ref_slice).pos, .returnExprStmt => self.cast(.returnExprStmt).pos, .returnStmt => self.cast(.returnStmt).pos, .root => self.cast(.root).stmts[0].pos(), @@ -883,7 +902,7 @@ pub const UnaryOp = enum(u8) { }; test "ast internals." { - try t.eq(std.enums.values(NodeType).len, 98); + try t.eq(std.enums.values(NodeType).len, 101); try t.eq(@sizeOf(NodeHeader), 1); } @@ -1472,17 +1491,17 @@ pub const Encoder = struct { try w.writeAll("..."); try w.writeByte('}'); }, - .pointer_slice => { + .ptr_slice => { try w.writeAll("[*]"); - try self.write(w, node.cast(.pointer_slice).elem); + try self.write(w, node.cast(.ptr_slice).elem); }, .expandOpt => { try w.writeByte('?'); try self.write(w, node.cast(.expandOpt).param); }, - .expand_ptr => { + .ptr => { try w.writeAll("*"); - try self.write(w, node.cast(.expand_ptr).elem); + try self.write(w, node.cast(.ptr).elem); }, .comptimeExpr => { try self.write(w, node.cast(.comptimeExpr).child); diff --git a/src/builtins/bindings.zig b/src/builtins/bindings.zig index dc2ae51d0..9287b6bc5 100644 --- a/src/builtins/bindings.zig +++ b/src/builtins/bindings.zig @@ -254,18 +254,18 @@ pub fn listSetIndex(vm: *cy.VM) Value { return Value.Void; } -pub fn listSlice(vm: *cy.VM) anyerror!Value { +pub fn List_slice(vm: *cy.VM) anyerror!Value { const list = vm.getValue(0).asHeapObject(); const range = vm.getValue(1).castHeapObject(*cy.heap.Range); const inner = cy.ptrAlignCast(*cy.List(Value), &list.list.list); if (range.start < 0) { - return vm.prepPanic("Out of bounds."); + return error.OutOfBounds; } if (range.end > inner.len) { - return vm.prepPanic("Out of bounds."); + return error.OutOfBounds; } if (range.end < range.start) { - return vm.prepPanic("Out of bounds."); + return error.OutOfBounds; } const elems = inner.buf[@intCast(range.start)..@intCast(range.end)]; @@ -275,6 +275,25 @@ pub fn listSlice(vm: *cy.VM) anyerror!Value { return cy.heap.allocListDyn(vm, elems); } +pub fn ListValue_slice(vm: *cy.VM) anyerror!Value { + const list = vm.getValue(0).asHeapObject(); + const slice_t: cy.TypeId = @intCast(vm.getInt(1)); + const range = vm.getValue(2).castHeapObject(*cy.heap.Range); + const inner = cy.ptrAlignCast(*cy.List(Value), &list.list.list); + if (range.start < 0) { + return error.OutOfBounds; + } + if (range.end > inner.len) { + return error.OutOfBounds; + } + if (range.end < range.start) { + return error.OutOfBounds; + } + + const ptr = inner.buf.ptr + @as(usize, @intCast(range.start)); + return vm.allocRefSlice(slice_t, ptr, @intCast(range.end - range.start)); +} + pub fn listInsert(vm: *cy.VM) Value { const index: i64 = @intCast(vm.getInt(1)); const value = vm.getValue(2); diff --git a/src/builtins/builtins.zig b/src/builtins/builtins.zig index 76bba2538..946b309a5 100644 --- a/src/builtins/builtins.zig +++ b/src/builtins/builtins.zig @@ -45,6 +45,7 @@ const funcs = [_]C.HostFuncEntry{ func("performGC", zErrFunc(performGC)), func("ptrcast_", zErrFunc(ptrcast)), func("print", print), + func("refcast", zErrFunc(refcast)), func("queueTask", zErrFunc(queueTask)), func("runestr", zErrFunc(runestr)), func("sizeof_", sizeof), @@ -116,7 +117,7 @@ const funcs = [_]C.HostFuncEntry{ // List func("List.$index", bindings.listIndex), - func("List.$indexRange", zErrFunc(bindings.listSlice)), + func("List.$indexRange", zErrFunc(bindings.List_slice)), func("List.$setIndex", bindings.listSetIndex), func("List.append", zErrFunc(bindings.listAppend)), func("List.appendAll", zErrFunc(bindings.listAppendAll)), @@ -132,6 +133,20 @@ const funcs = [_]C.HostFuncEntry{ // ListIterator func("ListIterator.next_", zErrFunc(bindings.listIteratorNext)), + // Slice + // func("Slice.endsWith", Slice_endsWith), + // func("Slice.find", Slice_find), + // func("Slice.split", zErrFunc(sliceSplit)), + // func("Slice.startsWith", Slice_startsWith), + // func("Slice.trim", sliceTrim), + + // RefSlice + // func("RefSlice.endsWith", RefSlice_endsWith), + // func("RefSlice.find", RefSlice_find), + // func("RefSlice.split", zErrFunc(sliceSplit)), + // func("RefSlice.startsWith", RefSlice_startsWith), + // func("RefSlice.trim", sliceTrim), + // Tuple func("Tuple.$index", bindings.tupleIndex), @@ -231,7 +246,7 @@ const funcs = [_]C.HostFuncEntry{ // DefaultMemory func("DefaultMemory.alloc", zErrFunc(DefaultMemory_alloc)), - func("DefaultMemory.free", zErrFunc(DefaultMemory_free)), + func("DefaultMemory.free", zErrFunc(DefaultMemory_free)), }; const types = [_]C.HostTypeEntry{ @@ -293,8 +308,8 @@ pub const BuiltinsData = struct { OptionTuple: cy.TypeId, OptionMap: cy.TypeId, OptionString: cy.TypeId, - PointerVoid: cy.TypeId, - SliceByte: cy.TypeId, + PtrVoid: cy.TypeId, + PtrSliceByte: cy.TypeId, }; pub fn create(vm: *cy.VM, r_uri: []const u8) C.Module { @@ -350,7 +365,7 @@ fn onLoad(vm_: ?*C.VM, mod: C.Sym) callconv(.C) void { const void_t = C.newType(vm_, bt.Void); defer C.release(vm_, void_t); - assert(C.expandTemplateType(pointer_tmpl, &void_t, 1, &data.PointerVoid)); + assert(C.expandTemplateType(pointer_tmpl, &void_t, 1, &data.PtrVoid)); const list_tmpl = chunk_sym.getMod().getSym("List").?.toC(); @@ -362,10 +377,10 @@ fn onLoad(vm_: ?*C.VM, mod: C.Sym) callconv(.C) void { const list_iter_tmpl = chunk_sym.getMod().getSym("ListIterator").?.toC(); assert(C.expandTemplateType(list_iter_tmpl, &dynamic_t, 1, &temp)); - const slice_tmpl = chunk_sym.getMod().getSym("Slice").?.toC(); + const ptr_slice_tmpl = chunk_sym.getMod().getSym("PtrSlice").?.toC(); const byte_t = C.newType(vm_, bt.Byte); defer C.release(vm_, byte_t); - assert(C.expandTemplateType(slice_tmpl, &byte_t, 1, &data.SliceByte)); + assert(C.expandTemplateType(ptr_slice_tmpl, &byte_t, 1, &data.PtrSliceByte)); // Verify all core types have been initialized. if (cy.Trace) { @@ -446,9 +461,11 @@ pub fn listFill(vm: *cy.VM) Value { return vm.allocListFill(vm.getValue(0), @intCast(vm.getInt(1))) catch cy.fatal(); } +pub fn refcast(vm: *cy.VM) anyerror!Value { + return vm.getValue(0); +} + pub fn ptrcast(vm: *cy.VM) anyerror!Value { - const ptr_t: cy.TypeId = @intCast(vm.getInt(0)); - _ = ptr_t; return vm.getValue(1); } @@ -754,7 +771,7 @@ pub fn DefaultMemory_alloc(vm: *cy.VM) anyerror!Value { const ptr_v = Value.initRaw(@intCast(@intFromPtr(ptr))); const data = vm.getData(*BuiltinsData, "builtins"); - return vm.allocObjectSmall(data.SliceByte, &.{ ptr_v, Value.initInt(@intCast(size)) }); + return vm.allocObjectSmall(data.PtrSliceByte, &.{ ptr_v, Value.initInt(@intCast(size)) }); } pub fn DefaultMemory_free(vm: *cy.VM) anyerror!Value { @@ -766,6 +783,18 @@ pub fn DefaultMemory_free(vm: *cy.VM) anyerror!Value { return Value.Void; } +fn Slice_startsWith(vm: *cy.VM) Value { + const slice = vm.getObject(*cy.heap.Array, 0).getSlice(); + const needle = vm.getArray(1); + return Value.initBool(std.mem.startsWith(u8, slice, needle)); +} + +fn Slice_endsWith(vm: *cy.VM) Value { + const slice = vm.getObject(*cy.heap.Array, 0).getSlice(); + const needle = vm.getArray(1); + return Value.initBool(std.mem.endsWith(u8, slice, needle)); +} + fn arrayConcat(vm: *cy.VM) Value { const slice = vm.getArray(0); const rslice = vm.getArray(1); @@ -815,7 +844,7 @@ fn arrayInsert(vm: *cy.VM) anyerror!Value { return Value.initNoCycPtr(new); } -fn arrayFind(vm: *cy.VM) Value { +fn Slice_find(vm: *cy.VM) Value { const obj = vm.getObject(*cy.heap.Array, 0); const slice = obj.getSlice(); const needle = vm.getArray(1); @@ -1200,7 +1229,7 @@ fn pointerGet(vm: *cy.VM) anyerror!Value { const addr: usize = @intFromPtr(ptr) + @as(usize, @intCast(uoff)); const val = @as(*?*anyopaque, @ptrFromInt(addr)).*; const data = vm.getData(*BuiltinsData, "builtins"); - return vm.allocPointer(data.PointerVoid, val); + return vm.allocPointer(data.PtrVoid, val); }, else => { return error.InvalidArgument; diff --git a/src/builtins/builtins_vm.cy b/src/builtins/builtins_vm.cy index 64f9da439..ba956847f 100644 --- a/src/builtins/builtins_vm.cy +++ b/src/builtins/builtins_vm.cy @@ -51,6 +51,8 @@ func ptrcast(#D type, val #S) *D: --| Queues a callback function as an async task. @host func queueTask(fn any) void +@host func refcast(#T type, ptr *T) &T + --| Converts a rune to a string. @host func runestr(val int) String @@ -179,7 +181,33 @@ type float #float64_t: @host type type _ +-- @host type ListValue[T type] struct: +-- @host func $index(self, idx int) &T + +-- func $index(self, range Range) []T: +-- return self.indexRange(([]T).id(), range) + +-- @host func indexRange(self, slice_t int, range Range) RefSlice(self, []T) + +-- --| Appends a value to the end of the list. +-- @host func append(self, val T) void + +-- --| Appends the elements of another list to the end of this list. +-- @host func appendList(self, o ListValue[T]) void + +-- --| Appends the elements of another list to the end of this list. +-- @host func appendSlice(self, slice []T) void + +-- --| Inserts a value at index `idx`. +-- @host func insert(self, idx int, val T) void + +-- --| Returns a new iterator over the list elements. +-- func iterator(self) ListIterator[T]: +-- return self.iterator_((ListIterator[T]).id()) + @host type List[T type] _: + -- val ListValue[T] + @host func $index(self, idx int) T @host='List.$indexRange' @@ -528,7 +556,61 @@ func FutureResolver.new(#T type) FutureResolver[T]: @host -func FutureResolver.new_(#T type, future_t int, ret_t int) FutureResolver[T] -type Slice[T type] struct: +type Ref[T type] #int64_t + +type RefSlice[T type] struct: + ptr *T + n int + + func $index(self, idx int) &T: + return refcast(T, self.ptr[idx]) + + func $index(self, range Range) []T: + var ptr_slice = self.ptr[range.start..range.end] + return .{ ptr=ptr_slice.ptr, n=ptr_slice.n } + + func $setIndex(self, idx int, val T) void: + self.ptr[idx] = val + + -- --| Returns whether the array ends with `suffix`. + -- @host func endsWith(self, suffix []T) bool + + -- --| Returns the first index of `needle` in the slice or `none` if not found. + -- @host func find(self, needle []T) ?int + + -- --| Returns the first index of any `bytes` in `set` or `none` if not found. + -- @host func findAny(self, set []T) ?int + + -- --| Returns the first index of `needle` in the slice or `none` if not found. + -- @host func findScalar(self, needle T) ?int + + func iterator(self) RefSliceIterator[T]: + return RefSliceIterator[T]{slice=self, next_idx=0} + + func len(self) int: + return self.n + + -- --| Returns a list of arrays split at occurrences of `sep`. + -- @host func split(self, sep []T) List[Array] + + -- --| Returns whether the array starts with `prefix`. + -- @host func startsWith(self, prefix []T) bool + + -- --| Returns a slice with ends trimmed from `delims`. `mode` can be .left, .right, or .ends. + -- @host func trim(self, mode symbol, delims []T) []T + +type RefSliceIterator[T type]: + slice []T + next_idx int + + func next(self) ?T: + if self.next_idx >= self.slice.len(): + return none + var res = self.slice[self.next_idx] + self.next_idx += 1 + return res + +type PtrSlice[T type] struct: ptr *T n int @@ -541,9 +623,60 @@ type Slice[T type] struct: func $setIndex(self, idx int, val T) void: self.ptr[idx] = val + --| Returns whether the array ends with `suffix`. + func endsWith(self, suffix [*]T) bool: + if suffix.len() > self.len(): + return false + for self[self.len()-suffix.len()..] -> elem, i: + if elem != suffix[i]: + return false + return true + + -- --| Returns the first index of `needle` in the slice or `none` if not found. + -- @host func find(self, needle [*]T) ?int + + -- --| Returns the first index of any `bytes` in `set` or `none` if not found. + -- @host func findAny(self, set [*]T) ?int + + --| Returns the first index of `needle` in the slice or `none` if not found. + func findScalar(self, needle T) ?int: + for self -> elem, i: + if elem == needle: return i + return none + + func iterator(self) PtrSliceIterator[T]: + return PtrSliceIterator[T]{slice=self, next_idx=0} + func len(self) int: return self.n + -- TODO: Consider returning iterator instead. + -- --| Returns a list of arrays split at occurrences of `sep`. + -- @host func split(self, sep [*]T) List[Array] + + --| Returns whether the array starts with `prefix`. + func startsWith(self, prefix [*]T) bool: + if prefix.len() > self.len(): + return false + for self[..prefix.len()] -> elem, i: + if elem != prefix[i]: + return false + return true + + -- --| Returns a slice with ends trimmed from `delims`. `mode` can be .left, .right, or .ends. + -- @host func trim(self, mode symbol, delims [*]T) [*]T + +type PtrSliceIterator[T type]: + slice [*]T + next_idx int + + func next(self) ?T: + if self.next_idx >= self.slice.len(): + return none + var res = self.slice[self.next_idx] + self.next_idx += 1 + return res + type IMemory trait: func alloc(self, len int) [*]byte func free(self, buf [*]byte) void diff --git a/src/compiler.zig b/src/compiler.zig index 48c5b9bbc..f10c2c6e3 100644 --- a/src/compiler.zig +++ b/src/compiler.zig @@ -956,9 +956,11 @@ fn reserveSyms(self: *Compiler, core_sym: *cy.sym.Chunk) !void{ self.sema.option_tmpl = core.getSym("Option").?.cast(.template); self.sema.array_tmpl = core.getSym("Array").?.cast(.template); self.sema.pointer_tmpl = core.getSym("pointer").?.cast(.template); + self.sema.ref_tmpl = core.getSym("Ref").?.cast(.template); self.sema.list_tmpl = core.getSym("List").?.cast(.template); self.sema.table_type = core.getSym("Table").?.cast(.object_t); - self.sema.slice_tmpl = core.getSym("Slice").?.cast(.template); + self.sema.ptr_slice_tmpl = core.getSym("PtrSlice").?.cast(.template); + self.sema.ref_slice_tmpl = core.getSym("RefSlice").?.cast(.template); } if (chunk != self.main_chunk) { // Check for illegal top level statements. diff --git a/src/heap.zig b/src/heap.zig index 1983d0181..3ec0018c1 100644 --- a/src/heap.zig +++ b/src/heap.zig @@ -1513,6 +1513,7 @@ pub const VmExt = struct { pub const allocPointer = Root.allocPointer; pub const allocInt = Root.allocInt; pub const allocBoxValue = Root.allocBoxValue; + pub const allocRefSlice = Root.allocRefSlice; pub const allocSlice = Root.allocSlice; pub const allocEmptyObjectSmall = Root.allocEmptyObjectSmall; pub const allocEmptyObject = Root.allocEmptyObject; @@ -1758,6 +1759,19 @@ pub fn allocPointer(self: *cy.VM, type_id: cy.TypeId, ptr: ?*anyopaque) !Value { return Value.initNoCycPtr(obj); } +pub fn allocRefSlice(self: *cy.VM, type_id: cy.TypeId, ptr: ?*anyopaque, len: usize) !Value { + const obj = try allocPoolObject(self); + obj.object = .{ + .typeId = type_id, + .rc = 1, + .firstValue = undefined, + }; + const fields = obj.object.getValuesPtr(); + fields[0] = Value.initRaw(@intFromPtr(ptr)); + fields[1] = Value.initInt(@intCast(len)); + return Value.initNoCycPtr(obj); +} + pub fn allocSlice(self: *cy.VM, type_id: cy.TypeId, ptr: ?*anyopaque, len: usize) !Value { const obj = try allocPoolObject(self); obj.object = .{ diff --git a/src/lib.zig b/src/lib.zig index 0a7ad7a10..84d8dc8a7 100644 --- a/src/lib.zig +++ b/src/lib.zig @@ -662,7 +662,7 @@ export fn clSymbol(vm: *cy.VM, str: c.Str) Value { export fn clNewPointerVoid(vm: *cy.VM, ptr: ?*anyopaque) Value { const bt_data = vm.getData(*cy.builtins.BuiltinsData, "builtins"); - return cy.heap.allocPointer(vm, bt_data.PointerVoid, ptr) catch fatal(); + return cy.heap.allocPointer(vm, bt_data.PtrVoid, ptr) catch fatal(); } export fn clNewType(vm: *cy.VM, type_id: cy.TypeId) Value { diff --git a/src/parser.zig b/src/parser.zig index 99063cbb5..c98784276 100644 --- a/src/parser.zig +++ b/src/parser.zig @@ -826,6 +826,7 @@ pub const Parser = struct { .left_bracket, .star, .question, + .ampersand, .pound, .void_k, .type_k, @@ -2758,9 +2759,19 @@ pub const Parser = struct { .star => { self.advance(); const elem = (try self.parseTermExpr2Opt(.{ .parse_record_expr = false })) orelse { - return self.reportError("Expected pointer child type.", &.{}); + return self.reportError("Expected right child.", &.{}); }; - return try self.ast.newNodeErase(.expand_ptr, .{ + return try self.ast.newNodeErase(.ptr, .{ + .elem = elem, + .pos = self.tokenSrcPos(start), + }); + }, + .ampersand => { + self.advance(); + const elem = (try self.parseTermExpr2Opt(.{ .parse_record_expr = true })) orelse { + return self.reportError("Expected right child.", &.{}); + }; + return try self.ast.newNodeErase(.ref, .{ .elem = elem, .pos = self.tokenSrcPos(start), }); @@ -2914,7 +2925,17 @@ pub const Parser = struct { const elem = (try self.parseTermExpr2Opt(.{ .parse_record_expr = false })) orelse { return self.reportError("Expected pointer slice child type.", &.{}); }; - return try self.ast.newNodeErase(.pointer_slice, .{ + return try self.ast.newNodeErase(.ptr_slice, .{ + .elem = elem, + .pos = self.tokenSrcPos(start), + }); + } else if (next_tag == .right_bracket) { + self.advance(); + self.advance(); + const elem = (try self.parseTermExpr2Opt(.{ .parse_record_expr = false })) orelse { + return self.reportError("Expected reference slice child type.", &.{}); + }; + return try self.ast.newNodeErase(.ref_slice, .{ .elem = elem, .pos = self.tokenSrcPos(start), }); @@ -2941,14 +2962,6 @@ pub const Parser = struct { .left_brace => { return @ptrCast(try self.parseInitLiteral()); }, - .ampersand => { - self.advance(); - const child = try self.parseTermExpr2(.{}); - return self.ast.newNodeErase(.unary_expr, .{ - .child = child, - .op = .address_of, - }); - }, .minus => { self.advance(); const child = try self.parseTermExpr2(.{}); @@ -3102,13 +3115,11 @@ pub const Parser = struct { .dot_dot => { // Start omitted. self.advance(); - const end = (try self.parseExpr(.{})) orelse { - return self.reportError("Expected range end.", &.{}); - }; + const opt_end = try self.parseExpr(.{}); return self.ast.newNodeErase(.range, .{ .start = null, - .end = end, + .end = opt_end, .inc = true, .pos = self.tokenSrcPos(start), }); diff --git a/src/sema.zig b/src/sema.zig index 40dffdc17..2c306dc77 100644 --- a/src/sema.zig +++ b/src/sema.zig @@ -389,15 +389,24 @@ pub fn semaStmt(c: *cy.Chunk, node: *ast.Node) !void { { const iterable_n: *ast.Node = @ptrCast(stmt.iterable); const iterable_init = try c.semaExprCstr(stmt.iterable, bt.Dyn); - const iterable_v = try declareLocalNameInit(c, "$iterable", bt.Dyn, iterable_init, iterable_n); + const iterable_v = try declareHiddenLocal(c, "$iterable", bt.Dyn, iterable_init, iterable_n); const iterable = try semaLocal(c, iterable_v.id, node); const iterator_init = try c.semaCallObjSym0(iterable.irIdx, "iterator", iterable_n); - const iterator_v = try declareLocalNameInit(c, "$iterator", bt.Dyn, iterator_init, iterable_n); + const iterator_v = try declareHiddenLocal(c, "$iterator", bt.Dyn, iterator_init, iterable_n); const iterator = try semaLocal(c, iterator_v.id, node); const counter_init = try c.semaInt(-1, node); - _ = try declareLocalNameInit(c, "$counter", bt.Integer, counter_init, node); + const counterv = try declareHiddenLocal(c, "$counter", bt.Integer, counter_init, node); + + const ClosureData = struct { + stmt: *ast.ForIterStmt, + counterv: LocalResult, + }; + var closure_data = ClosureData{ + .stmt = stmt, + .counterv = counterv, + }; // Loop block. const loop_stmt = try c.ir.pushEmptyStmt(c.alloc, .loopStmt, node); @@ -405,37 +414,38 @@ pub fn semaStmt(c: *cy.Chunk, node: *ast.Node) !void { { // if $iterator.next() -> each, i const S = struct { - fn body(c_: *cy.Chunk, data: *anyopaque) !void { - const stmt_: *ast.ForIterStmt = @ptrCast(@alignCast(data)); - if (stmt_.count) |count| { + fn body(c_: *cy.Chunk, data_: *anyopaque) !void { + const data: *ClosureData = @ptrCast(@alignCast(data_)); + if (data.stmt.count) |count| { // $counter += 1 const int_t = c_.sema.getTypeSym(bt.Integer); - const sym = try c_.mustFindSym(int_t, "$infix+", @ptrCast(stmt_)); - const func_sym = try requireFuncSym(c_, sym, @ptrCast(stmt_)); - const one = try c_.semaInt(1, @ptrCast(stmt_)); - const res = try getOrLookupVar(c_, "$counter", @ptrCast(stmt_)); - const counter_ = try semaLocal(c_, res.local, @ptrCast(stmt_)); - const right = try c_.semaCallFuncSymRec2(func_sym, @ptrCast(stmt_), counter_, - &.{one}, &.{ @ptrCast(stmt_) }, .any, @ptrCast(stmt_)); - const irStart = try c_.ir.pushEmptyStmt(c_.alloc, .setLocal, @ptrCast(stmt_)); - const counter_info = c_.varStack.items[res.local].inner.local; + const sym = try c_.mustFindSym(int_t, "$infix+", @ptrCast(data.stmt)); + const func_sym = try requireFuncSym(c_, sym, @ptrCast(data.stmt)); + const one = try c_.semaInt(1, @ptrCast(data.stmt)); + + const counter_ = try semaLocal(c_, data.counterv.id, @ptrCast(data.stmt)); + const right = try c_.semaCallFuncSymRec2(func_sym, @ptrCast(data.stmt), counter_, + &.{one}, &.{ @ptrCast(data.stmt) }, .any, @ptrCast(data.stmt)); + const irStart = try c_.ir.pushEmptyStmt(c_.alloc, .setLocal, @ptrCast(data.stmt)); + const info = c_.varStack.items[data.counterv.id].inner.local; c_.ir.setStmtData(irStart, .setLocal, .{ .local = .{ - .id = counter_info.id, + .id = info.id, .right = right.irIdx, }}); // $count = $counter const count_name = c_.ast.nodeString(count); - _ = try declareLocalNameInit(c_, count_name, bt.Integer, counter_, @ptrCast(stmt_)); + _ = try declareLocalInit(c_, count_name, bt.Integer, counter_, @ptrCast(data.stmt)); } - try semaStmts(c_, stmt_.stmts); - _ = try c_.ir.pushStmt(c_.alloc, .contStmt, @ptrCast(stmt_), {}); + try semaStmts(c_, data.stmt.stmts); + _ = try c_.ir.pushStmt(c_.alloc, .contStmt, @ptrCast(data.stmt), {}); } }; const next = try c.semaCallObjSym0(iterator.irIdx, "next", node); const opt = try c.semaOptionExpr2(next, node); - try semaIfUnwrapStmt2(c, opt, node, stmt.each, S.body, stmt, &.{}, @ptrCast(stmt)); + + try semaIfUnwrapStmt2(c, opt, node, stmt.each, S.body, &closure_data, &.{}, @ptrCast(stmt)); } const block = try popLoopBlock(c); c.ir.setStmtData(loop_stmt, .loopStmt, .{ @@ -664,7 +674,7 @@ fn semaIfUnwrapStmt2(c: *cy.Chunk, opt: ExprResult, opt_n: *ast.Node, opt_unwrap try pushBlock(c, @ptrCast(node)); { // $opt = - const opt_v = try declareLocalNameInit(c, "$opt", opt.type.toDeclType(), opt, opt_n); + const opt_v = try declareHiddenLocal(c, "$opt", opt.type.toDeclType(), opt, opt_n); const get_opt = try semaLocal(c, opt_v.id, opt_n); // if !isNone($opt) @@ -689,7 +699,7 @@ fn semaIfUnwrapStmt2(c: *cy.Chunk, opt: ExprResult, opt_n: *ast.Node, opt_unwrap return c.reportErrorFmt("Unsupported unwrap declaration: {}", &.{v(unwrap.type())}, unwrap); } const unwrap_init = try semaAccessEnumPayload(c, get_opt, "some", unwrap); - const unwrap_v = try declareLocalNameInit(c, unwrap_name, unwrap_t, unwrap_init, unwrap); + const unwrap_v = try declareLocalInit(c, unwrap_name, unwrap_t, unwrap_init, unwrap); const unwrap_local = try semaLocal(c, unwrap_v.id, unwrap); if (unwrap.type() == .seqDestructure) { @@ -702,7 +712,7 @@ fn semaIfUnwrapStmt2(c: *cy.Chunk, opt: ExprResult, opt_n: *ast.Node, opt_unwrap }); index.type.id = bt.Any; const dvar_init = try semaIndexExpr2(c, unwrap_local, unwrap, index, unwrap, unwrap); - _ = try declareLocalNameInit(c, name, bt.Dyn, dvar_init, unwrap); + _ = try declareLocalInit(c, name, bt.Dyn, dvar_init, unwrap); } } } @@ -1096,7 +1106,8 @@ fn semaIndexExpr(c: *cy.Chunk, left: *ast.Node, left_res: ExprResult, expr: Expr left, left_res, array.args, expr.getRetCstr(), expr.node); - if (!expr.use_addressable and c.sema.isPointerType(res.type.id) and expr.target_t != res.type.id) { + const ptr_or_ref = c.sema.isPointerType(res.type.id) or c.sema.isRefType(res.type.id); + if (!expr.use_addressable and ptr_or_ref and expr.target_t != res.type.id) { const child_t = c.sema.getPointerChildType(res.type.id); const loc = try c.ir.pushExpr(.deref, c.alloc, child_t, expr.node, .{ .expr = res.irIdx, @@ -2765,8 +2776,8 @@ const LocalResult = struct { ir_id: u32, }; -fn declareLocalNameInit(c: *cy.Chunk, name: []const u8, decl_t: cy.TypeId, init: ExprResult, node: *ast.Node) !LocalResult { - const var_id = try declareLocalName(c, name, decl_t, true, node); +fn declareLocalInit(c: *cy.Chunk, name: []const u8, decl_t: cy.TypeId, init: ExprResult, node: *ast.Node) !LocalResult { + const var_id = try declareLocalName(c, name, decl_t, false, true, node); const info = c.varStack.items[var_id].inner.local; var data = c.ir.getStmtDataPtr(info.declIrStart, .declareLocalInit); data.init = init.irIdx; @@ -2774,33 +2785,40 @@ fn declareLocalNameInit(c: *cy.Chunk, name: []const u8, decl_t: cy.TypeId, init: return .{ .id = var_id, .ir_id = info.declIrStart }; } -fn declareLocalName(c: *cy.Chunk, name: []const u8, declType: TypeId, hasInit: bool, node: *ast.Node) !LocalVarId { - const proc = c.proc(); - if (proc.nameToVar.get(name)) |varInfo| { - if (varInfo.blockId == c.semaBlocks.items.len - 1) { - const svar = &c.varStack.items[varInfo.varId]; - if (svar.isParentLocalAlias()) { - return c.reportErrorFmt("`{}` already references a parent local variable.", &.{v(name)}, node); - } else if (svar.type == .staticAlias) { - return c.reportErrorFmt("`{}` already references a static variable.", &.{v(name)}, node); +fn declareHiddenLocal(c: *cy.Chunk, name: []const u8, decl_t: cy.TypeId, init: ExprResult, node: *ast.Node) !LocalResult { + const var_id = try declareLocalName(c, name, decl_t, true, true, node); + const info = c.varStack.items[var_id].inner.local; + var data = c.ir.getStmtDataPtr(info.declIrStart, .declareLocalInit); + data.init = init.irIdx; + data.initType = CompactType.init(decl_t); + return .{ .id = var_id, .ir_id = info.declIrStart }; +} + +fn declareLocalName(c: *cy.Chunk, name: []const u8, declType: TypeId, hidden: bool, hasInit: bool, node: *ast.Node) !LocalVarId { + if (!hidden) { + const proc = c.proc(); + if (proc.nameToVar.get(name)) |varInfo| { + if (varInfo.blockId == c.semaBlocks.items.len - 1) { + const svar = &c.varStack.items[varInfo.varId]; + if (svar.isParentLocalAlias()) { + return c.reportErrorFmt("`{}` already references a parent local variable.", &.{v(name)}, node); + } else if (svar.type == .staticAlias) { + return c.reportErrorFmt("`{}` already references a static variable.", &.{v(name)}, node); + } else { + return c.reportErrorFmt("Variable `{}` is already declared in the block.", &.{v(name)}, node); + } } else { - return c.reportErrorFmt("Variable `{}` is already declared in the block.", &.{v(name)}, node); + // Create shadow entry for restoring the prev var. + try c.varShadowStack.append(c.alloc, .{ + .namePtr = name.ptr, + .nameLen = @intCast(name.len), + .varId = varInfo.varId, + .blockId = varInfo.blockId, + }); } - } else { - // Create shadow entry for restoring the prev var. - try c.varShadowStack.append(c.alloc, .{ - .namePtr = name.ptr, - .nameLen = @intCast(name.len), - .varId = varInfo.varId, - .blockId = varInfo.blockId, - }); } } - return try declareLocalName2(c, name, declType, false, hasInit, node); -} - -fn declareLocalName2(c: *cy.Chunk, name: []const u8, declType: TypeId, hidden: bool, hasInit: bool, node: *ast.Node) !LocalVarId { const id = try pushLocalVar(c, .local, name, declType, hidden); var svar = &c.varStack.items[id]; @@ -2848,7 +2866,7 @@ fn declareLocalName2(c: *cy.Chunk, name: []const u8, declType: TypeId, hidden: b fn declareLocal(c: *cy.Chunk, ident: *ast.Node, declType: TypeId, hasInit: bool) !LocalVarId { const name = c.ast.nodeString(ident); - return declareLocalName(c, name, declType, hasInit, ident); + return declareLocalName(c, name, declType, false, hasInit, ident); } fn localDecl(c: *cy.Chunk, node: *ast.VarDecl) !void { @@ -3586,17 +3604,25 @@ pub fn resolveSym(c: *cy.Chunk, expr: *ast.Node) anyerror!*cy.Sym { return error.TODO; // return try cte.expandTemplateOnCallExpr(c, expr.cast(.callExpr)); }, - .pointer_slice => { - const pointer_slice = expr.cast(.pointer_slice); - return try cte.expandTemplateOnCallArgs(c, c.sema.slice_tmpl, &.{ pointer_slice.elem }, expr); + .ptr_slice => { + const ptr_slice = expr.cast(.ptr_slice); + return try cte.expandTemplateOnCallArgs(c, c.sema.ptr_slice_tmpl, &.{ ptr_slice.elem }, expr); }, .expandOpt => { const expand_opt = expr.cast(.expandOpt); return try cte.expandTemplateOnCallArgs(c, c.sema.option_tmpl, &.{expand_opt.param}, expr); }, - .expand_ptr => { - const expand_ptr = expr.cast(.expand_ptr); - return try cte.expandTemplateOnCallArgs(c, c.sema.pointer_tmpl, &.{ expand_ptr.elem }, expr); + .ptr => { + const ptr = expr.cast(.ptr); + return try cte.expandTemplateOnCallArgs(c, c.sema.pointer_tmpl, &.{ ptr.elem }, expr); + }, + .ref => { + const ref = expr.cast(.ref); + return try cte.expandTemplateOnCallArgs(c, c.sema.ref_tmpl, &.{ ref.elem }, expr); + }, + .ref_slice => { + const ref_slice = expr.cast(.ref_slice); + return try cte.expandTemplateOnCallArgs(c, c.sema.ref_slice_tmpl, &.{ ref_slice.elem }, expr); }, else => { return c.reportErrorFmt("Unsupported symbol expr: `{}`", &.{v(expr.type())}, expr); @@ -4478,7 +4504,7 @@ fn semaSwitchExpr(c: *cy.Chunk, block: *ast.SwitchBlock, target: SemaExprOptions /// The choice tag is then used for the switch expr. /// Choice payload is copied to case blocks that have a capture param. fn semaSwitchChoicePrologue(c: *cy.Chunk, info: *SwitchInfo, expr: ExprResult, exprId: *ast.Node) !u32 { - const choiceVarId = try declareLocalName2(c, "choice", expr.type.id, true, true, exprId); + const choiceVarId = try declareLocalName(c, "choice", expr.type.id, true, true, exprId); const choiceVar = &c.varStack.items[choiceVarId]; info.choiceIrVarId = choiceVar.inner.local.id; const declareLoc = choiceVar.inner.local.declIrStart; @@ -5140,9 +5166,9 @@ pub const ChunkExt = struct { const ctype = CompactType.init(sym.getStaticType().?); return ExprResult.initCustom(cy.NullId, .sym, ctype, .{ .sym = sym }); }, - .expand_ptr => { - const expand_ptr = node.cast(.expand_ptr); - const sym = try cte.expandTemplateOnCallArgs(c, c.sema.pointer_tmpl, &.{ expand_ptr.elem }, node); + .ptr => { + const ptr = node.cast(.ptr); + const sym = try cte.expandTemplateOnCallArgs(c, c.sema.pointer_tmpl, &.{ ptr.elem }, node); const ctype = CompactType.init(sym.getStaticType().?); return ExprResult.initCustom(cy.NullId, .sym, ctype, .{ .sym = sym }); }, @@ -5516,14 +5542,14 @@ pub const ChunkExt = struct { .callExpr => { return c.semaCallExpr(expr); }, - .pointer_slice => { - const sym = try cte.expandTemplateOnCallArgs(c, c.sema.slice_tmpl, &.{node.cast(.pointer_slice).elem}, node); + .ptr_slice => { + const sym = try cte.expandTemplateOnCallArgs(c, c.sema.ptr_slice_tmpl, &.{node.cast(.ptr_slice).elem}, node); const type_id = sym.getStaticType().?; const irIdx = try c.ir.pushExpr(.typeSym, c.alloc, bt.MetaType, node, .{ .typeId = type_id }); return ExprResult.init(irIdx, CompactType.init(bt.MetaType)); }, - .expand_ptr => { - const sym = try cte.expandTemplateOnCallArgs(c, c.sema.pointer_tmpl, &.{node.cast(.expand_ptr).elem}, node); + .ref_slice => { + const sym = try cte.expandTemplateOnCallArgs(c, c.sema.ref_slice_tmpl, &.{node.cast(.ref_slice).elem}, node); const type_id = sym.getStaticType().?; const irIdx = try c.ir.pushExpr(.typeSym, c.alloc, bt.MetaType, node, .{ .typeId = type_id }); return ExprResult.init(irIdx, CompactType.init(bt.MetaType)); @@ -5616,6 +5642,12 @@ pub const ChunkExt = struct { .unary_expr => { return try c.semaUnExpr(expr); }, + .ref => { + return try c.semaRefOf(expr); + }, + .ptr => { + return try c.semaPtrOf(expr); + }, .deref => { const deref = node.cast(.deref); @@ -6103,6 +6135,73 @@ pub const ChunkExt = struct { return ExprResult.initDynamic(loc, bt.Any); } + pub fn semaRefOf(c: *cy.Chunk, expr: Expr) !ExprResult { + const node = expr.node.cast(.ref); + const child = try c.semaExpr(node.elem, .{ .prefer_addressable = true }); + const child_type = c.sema.getTypeSym(child.type.id); + if (child_type.type == .array_t) { + if (child.resType == .local) { + try ensureLiftedVar(c, child.data.local); + } + + // Declare array in same block to extend lifetime. + const tempv = try declareHiddenLocal(c, "$temp", child.type.id, child, node.elem); + const temp = try semaLocal(c, tempv.id, node.elem); + return semaArrayRefSlice(c, temp, expr.node); + } + + if (!child.addressable) { + return c.reportError("Expected an addressable expression.", expr.node); + } + if (child.resType == .local) { + try ensureLiftedVar(c, child.data.local); + } + + const ref_t = try getRefType(c, child.type.id); + const irIdx = try c.ir.pushExpr(.address_of, c.alloc, ref_t, expr.node, .{ + .expr = child.irIdx, + }); + return ExprResult.initStatic(irIdx, ref_t); + } + + pub fn semaArrayRefSlice(c: *cy.Chunk, arr: ExprResult, node: *ast.Node) !ExprResult{ + const array_t = c.sema.getTypeSym(arr.type.id).cast(.array_t); + const elem_t = array_t.elem_t; + const slice_t = try getRefSliceType(c, elem_t); + const ptr_t = try getPointerType(c, elem_t); + + var b: ObjectBuilder = .{ .c = c }; + try b.begin(slice_t, 2, node); + var loc = try c.ir.pushExpr(.address_of, c.alloc, ptr_t, node, .{ + .expr = arr.irIdx, + }); + b.pushArg(ExprResult.initStatic(loc, ptr_t)); + + const len = try semaInt(c, @intCast(array_t.n), node); + b.pushArg(len); + loc = b.end(); + + return ExprResult.initStatic(loc, slice_t); + } + + pub fn semaPtrOf(c: *cy.Chunk, expr: Expr) !ExprResult { + const node = expr.node.cast(.ptr); + const child = try c.semaExpr(node.elem, .{ .prefer_addressable = true }); + if (!child.addressable) { + return c.reportError("Expected an addressable expression.", expr.node); + } + + if (child.resType == .local) { + try ensureLiftedVar(c, child.data.local); + } + + const ptr_t = try getPointerType(c, child.type.id); + const irIdx = try c.ir.pushExpr(.address_of, c.alloc, ptr_t, expr.node, .{ + .expr = child.irIdx, + }); + return ExprResult.initStatic(irIdx, ptr_t); + } + pub fn semaUnExpr(c: *cy.Chunk, expr: Expr) !ExprResult { const node = expr.node.cast(.unary_expr); @@ -6136,22 +6235,6 @@ pub const ChunkExt = struct { }}); return ExprResult.initStatic(loc, bt.Boolean); }, - .address_of => { - const child = try c.semaExpr(node.child, .{ .prefer_addressable = true }); - if (!child.addressable) { - return c.reportError("Expected an addressable expression.", expr.node); - } - - if (child.resType == .local) { - try ensureLiftedVar(c, child.data.local); - } - - const ptr_t = try getPointerType(c, child.type.id); - const irIdx = try c.ir.pushExpr(.address_of, c.alloc, ptr_t, expr.node, .{ - .expr = child.irIdx, - }); - return ExprResult.initStatic(irIdx, ptr_t); - }, .bitwiseNot => { const child = try c.semaExprTarget(node.child, expr.target_t); if (child.type.isDynAny()) { @@ -6324,7 +6407,7 @@ fn semaWithInitPairs(c: *cy.Chunk, type_sym: *cy.Sym, type_id: cy.TypeId, init_n try pushBlock(c, node); { // create temp with object. - const var_id = try declareLocalName(c, "$temp", type_id, true, node); + const var_id = try declareLocalName(c, "$temp", type_id, true, true, node); const temp_ir_id = c.varStack.items[var_id].inner.local.id; const temp_ir = c.varStack.items[var_id].inner.local.declIrStart; const decl_stmt = c.ir.getStmtDataPtr(temp_ir, .declareLocalInit); @@ -6672,9 +6755,11 @@ pub const Sema = struct { funcSigMap: std.HashMapUnmanaged(FuncSigKey, FuncSigId, FuncSigKeyContext, 80), future_tmpl: *cy.sym.Template, - slice_tmpl: *cy.sym.Template, + ref_slice_tmpl: *cy.sym.Template, + ptr_slice_tmpl: *cy.sym.Template, option_tmpl: *cy.sym.Template, pointer_tmpl: *cy.sym.Template, + ref_tmpl: *cy.sym.Template, list_tmpl: *cy.sym.Template, table_type: *cy.sym.ObjectType, array_tmpl: *cy.sym.Template, @@ -6684,9 +6769,11 @@ pub const Sema = struct { .alloc = alloc, .compiler = compiler, .future_tmpl = undefined, - .slice_tmpl = undefined, + .ref_slice_tmpl = undefined, + .ptr_slice_tmpl = undefined, .option_tmpl = undefined, .pointer_tmpl = undefined, + .ref_tmpl = undefined, .list_tmpl = undefined, .table_type = undefined, .array_tmpl = undefined, @@ -7147,4 +7234,18 @@ pub fn getPointerType(c: *cy.Chunk, elem_t: cy.TypeId) !cy.TypeId { defer c.vm.release(arg); const sym = try cte.expandTemplate(c, c.sema.pointer_tmpl, &.{ arg }); return sym.getStaticType().?; +} + +pub fn getRefType(c: *cy.Chunk, elem_t: cy.TypeId) !cy.TypeId { + const arg = try c.vm.allocType(elem_t); + defer c.vm.release(arg); + const sym = try cte.expandTemplate(c, c.sema.ref_tmpl, &.{ arg }); + return sym.getStaticType().?; +} + +pub fn getRefSliceType(c: *cy.Chunk, elem_t: cy.TypeId) !cy.TypeId { + const arg = try c.vm.allocType(elem_t); + defer c.vm.release(arg); + const sym = try cte.expandTemplate(c, c.sema.ref_slice_tmpl, &.{ arg }); + return sym.getStaticType().?; } \ No newline at end of file diff --git a/src/std/os_ffi.zig b/src/std/os_ffi.zig index caea7b998..365c6bec0 100644 --- a/src/std/os_ffi.zig +++ b/src/std/os_ffi.zig @@ -811,7 +811,7 @@ fn toCyType(vm: *cy.VM, ctype: CType) !types.TypeId { .charPtr, .voidPtr => { const data = vm.getData(*cy.builtins.BuiltinsData, "builtins"); - return data.PointerVoid; + return data.PtrVoid; }, .void => return bt.Void, else => { @@ -1006,7 +1006,7 @@ pub fn ffiBindLib(vm: *cy.VM, config: BindLibConfig) !Value { const symKey = try vm.allocAstringConcat("ptrTo", typeName); const func = cy.ptrAlignCast(cy.ZHostFuncFn, funcPtr); - const funcSigId = try vm.sema.ensureFuncSigRt(&.{ bt_data.PointerVoid }, cstruct.type); + const funcSigId = try vm.sema.ensureFuncSigRt(&.{ bt_data.PtrVoid }, cstruct.type); const funcVal = try cy.heap.allocHostFunc(vm, func, 1, funcSigId, cyState, false); try table.asHeapObject().table.set(vm, symKey, funcVal); vm.release(symKey); @@ -1054,7 +1054,7 @@ pub fn ffiBindLib(vm: *cy.VM, config: BindLibConfig) !Value { const name_id = try rt.ensureNameSymExt(vm, methodName, true); const managed_name = rt.getName(vm, name_id); - const funcSigId = try vm.sema.ensureFuncSigRt(&.{ bt.Any, bt_data.PointerVoid }, cstruct.type); + const funcSigId = try vm.sema.ensureFuncSigRt(&.{ bt.Any, bt_data.PtrVoid }, cstruct.type); const group = try vm.addFuncGroup(); const func_sym = rt.FuncSymbol.initHostFunc(@ptrCast(func), true, true, 2, funcSigId); _ = try vm.addGroupFunc(group, managed_name, funcSigId, func_sym); @@ -1135,7 +1135,7 @@ fn cGetFuncPtr(vm: *cy.VM, val: Value) callconv(.C) ?*anyopaque { fn cAllocCyPointer(vm: *cy.VM, ptr: ?*anyopaque) callconv(.C) Value { const bt_data = vm.getData(*cy.builtins.BuiltinsData, "builtins"); - return cy.heap.allocPointer(vm, bt_data.PointerVoid, ptr) catch cy.fatal(); + return cy.heap.allocPointer(vm, bt_data.PtrVoid, ptr) catch cy.fatal(); } fn cAllocInt(vm: *cy.VM, val: i64) callconv(.C) Value { diff --git a/src/std/test.cy b/src/std/test.cy index f9333b7ce..0fcf77ad6 100644 --- a/src/std/test.cy +++ b/src/std/test.cy @@ -19,6 +19,17 @@ func eqList(a List[#T], b List[T]) bool: return false return true +--| Returns `true` if two slices have the same size and the elements are equal +--| as if `eq` was called on those corresponding elements. +func eqSlice(a []#T, b []T) bool: + if a.len() != b.len(): + eprint("Length mismatch: $(a.len()) != $(b.len())") + return false + for a -> a_elem, i: + if !eq(a_elem, b[i]): + return false + return true + --| Returns `true` if two numbers are near each other within epsilon 1e-5. @host func eqNear(a #T, b T) bool diff --git a/src/sym.zig b/src/sym.zig index 7182d2a00..46e95553e 100644 --- a/src/sym.zig +++ b/src/sym.zig @@ -1660,7 +1660,7 @@ pub fn writeSymName(s: *cy.Sema, w: anytype, sym: *cy.Sym, config: SymFormatConf const name = s.getTypeBaseName(arg.type.type); try w.print("*{s}", .{name}); return; - } else if (variant.root_template == s.slice_tmpl) { + } else if (variant.root_template == s.ref_slice_tmpl) { const arg = variant.args[0].asHeapObject(); const name = s.getTypeBaseName(arg.type.type); try w.print("[]{s}", .{name}); diff --git a/src/types.zig b/src/types.zig index db42eb9c5..9d63148e8 100644 --- a/src/types.zig +++ b/src/types.zig @@ -225,6 +225,12 @@ pub const SemaExt = struct { return @intCast(typeId); } + pub fn isRefType(s: *cy.Sema, id: cy.TypeId) bool { + const sym = s.getTypeSym(id); + const variant = sym.getVariant() orelse return false; + return variant.root_template == s.ref_tmpl; + } + pub fn isPointerType(s: *cy.Sema, id: cy.TypeId) bool { const sym = s.getTypeSym(id); const variant = sym.getVariant() orelse return false; diff --git a/test/core/arrays.cy b/test/core/arrays.cy index 1d0741ff4..26a629bba 100644 --- a/test/core/arrays.cy +++ b/test/core/arrays.cy @@ -3,6 +3,7 @@ use t 'test' -- TODO: Some of these tests should be moved to ref/ptr slices. var arr = [3]int{1, 2, 3} +t.eqSlice(&arr, &[3]int{1, 2, 3}) -- index operator t.eq(try arr[-1], error.OutOfBounds)