Merge branch 'master' of https://github.com/dfinity/motoko into ryan/…

…rts-outside-nix
dfinity · Mar 5, 2024 · 248f014 · 248f014
2 parents 7aaab9e + c30c898
commit 248f014
Show file tree

Hide file tree

Showing 58 changed files with 592 additions and 250 deletions.
diff --git a/Building.md b/Building.md
@@ -56,11 +56,11 @@ For more details on our CI and CI setup, see `CI.md`.
 
 ## Making releases
 
-We make frequent releases, at least weekly. The steps to make a release (say, version 0.10.1) are:
+We make frequent releases, at least weekly. The steps to make a release (say, version 0.11.2) are:
 
  * Make sure that the top section of `Changelog.md` has a title like
 
-        ## 0.10.1 (2023-09-20)
+        ## 0.11.2 (2024-02-29)
 
    with today’s date.
 
@@ -75,20 +75,20 @@ We make frequent releases, at least weekly. The steps to make a release (say, ve
 
    If not, create and merge a separate PR to update the doc (adding any new files) and goto step 0.
 
- * Define a shell variable `export MOC_MINOR=1`
+ * Define a shell variable `export MOC_MINOR=2`
 
- * Look at `git log --first-parent 0.10.$(expr $MOC_MINOR - 1)..HEAD` and check
+ * Look at `git log --first-parent 0.11.$(expr $MOC_MINOR - 1)..HEAD` and check
    that everything relevant is mentioned in the changelog section, and possibly
    clean it up a bit, curating the information for the target audience.
 
- * `git commit -am "chore: Releasing 0.10."$MOC_MINOR`
+ * `git commit -am "chore: Releasing 0.11."$MOC_MINOR`
  * Create a PR from this commit, and label it `automerge-squash`. E.g.
-   with `git push origin HEAD:$USER/0.10.$MOC_MINOR`. Mergify will
+   with `git push origin HEAD:$USER/0.11.$MOC_MINOR`. Mergify will
    merge it into `master` without additional approval, but it will take some
    time as the title (version number) enters into the `nix` dependency tracking.
  * `git switch master; git pull --rebase`. The release commit should be your `HEAD`
- * `git tag 0.10.$MOC_MINOR -m "Motoko 0.10."$MOC_MINOR`
- * `git push origin 0.10.$MOC_MINOR`
+ * `git tag 0.11.$MOC_MINOR -m "Motoko 0.11."$MOC_MINOR`
+ * `git push origin 0.11.$MOC_MINOR`
 
 Pushing the tag should cause GitHub Actions to create a “Release” on the GitHub
 project. This will fail if the changelog is not in order (in this case, fix and
@@ -102,11 +102,11 @@ branch to the `next-moc` branch.
 * Wait ca. 5min after releasing to give the CI/CD pipeline time to upload the release artifacts
 * Change into `motoko-base`
 * `git switch next-moc; git pull`
-* `git switch -c $USER/update-moc-0.10.$MOC_MINOR`
+* `git switch -c $USER/update-moc-0.11.$MOC_MINOR`
 * Update the `moc_version` env variable in `.github/workflows/{ci, package-set}.yml`
   to the new released version:
-  `perl -pi -e "s/moc_version: \"0\.10\.\\d+\"/moc_version: \"0.10.$MOC_MINOR\"/g" .github/workflows/ci.yml .github/workflows/package-set.yml`
-* `git add .github/ && git commit -m "Motoko 0.10."$MOC_MINOR`
+  `perl -pi -e "s/moc_version: \"0\.11\.\\d+\"/moc_version: \"0.11.$MOC_MINOR\"/g" .github/workflows/ci.yml .github/workflows/package-set.yml`
+* `git add .github/ && git commit -m "Motoko 0.11."$MOC_MINOR`
 * You can `git push` now
 
 Make a PR off of that branch and merge it using a _normal merge_ (not
@@ -115,8 +115,8 @@ repo by a scheduled `niv-updater-action`.
 
 Finally tag the base release (so the documentation interpreter can do the right thing):
 * `git switch master && git pull`
-* `git tag moc-0.10.$MOC_MINOR`
-* `git push origin moc-0.10.$MOC_MINOR`
+* `git tag moc-0.11.$MOC_MINOR`
+* `git push origin moc-0.11.$MOC_MINOR`
 
 If you want to update the portal documentation, typically to keep in sync with a `dfx` release, follow the instructions in https://github.com/dfinity/portal/blob/master/MAINTENANCE.md.
 

diff --git a/Changelog.md b/Changelog.md
@@ -13,9 +13,20 @@
 
     **Limitations**: recursive and mutually recursive definitions are considered used,
     even if never referenced outside the recursive definition.
-    
+
   * Remove `__get_candid_interface_tmp_hack` endpoint. Candid interface is already stored as canister metadata, this temporary endpoint is redundant, thus removed. (#4386)
 
+  * Improved capability system, introducing a synchronous (`system`) capability (#4406).
+
+    `actor` initialisation body, `pre`/`postupgrade` hooks, `async` function bodies (and
+    blocks) possess this capability. Functions (and classes) can demand it by prepending `system`
+    to the type argument list. The capability can be forwarded in calls by mentioning `<system, …>`
+    in the instantiation parameter list.
+
+    BREAKING CHANGE (Minor): A few built-in functions have been marked with demand
+    for the `system` capability. In order to call these, the full call hierarchy needs to be
+    adapted to pass the `system` capability.
+
 ## 0.10.4 (2024-01-10)
 
 * motoko (`moc`)

diff --git a/doc/md/examples/Alice.mo b/doc/md/examples/Alice.mo
@@ -5,12 +5,12 @@ actor Alice {
 
   public func test() : async () {
 
-    Cycles.add(10_000_000_000_000);
+    Cycles.add<system>(10_000_000_000_000);
     let porky = await Lib.PiggyBank(Alice.credit, 1_000_000_000);
 
     assert (0 == (await porky.getSavings()));
 
-    Cycles.add(1_000_000);
+    Cycles.add<system>(1_000_000);
     await porky.deposit();
     assert (1_000_000 == (await porky.getSavings()));
 
@@ -20,7 +20,7 @@ actor Alice {
     await porky.withdraw(500_000);
     assert (0 == (await porky.getSavings()));
 
-    Cycles.add(2_000_000_000);
+    Cycles.add<system>(2_000_000_000);
     await porky.deposit();
     let refund = Cycles.refunded();
     assert (1_000_000_000 == refund);
@@ -31,7 +31,7 @@ actor Alice {
   // Callback for accepting cycles from PiggyBank
   public func credit() : async () {
     let available = Cycles.available();
-    let accepted = Cycles.accept(available);
+    let accepted = Cycles.accept<system>(available);
     assert (accepted == available);
   }
 

diff --git a/doc/md/examples/PiggyBank.mo b/doc/md/examples/PiggyBank.mo
@@ -20,7 +20,7 @@ shared(msg) actor class PiggyBank(
     let acceptable =
       if (amount <= limit) amount
       else limit;
-    let accepted = Cycles.accept(acceptable);
+    let accepted = Cycles.accept<system>(acceptable);
     assert (accepted == acceptable);
     savings += acceptable;
   };
@@ -29,7 +29,7 @@ shared(msg) actor class PiggyBank(
     : async () {
     assert (msg.caller == owner);
     assert (amount <= savings);
-    Cycles.add(amount);
+    Cycles.add<system>(amount);
     await benefit();
     let refund = Cycles.refunded();
     savings -= amount - refund;

diff --git a/doc/md/examples/Reminder.mo b/doc/md/examples/Reminder.mo
@@ -11,9 +11,9 @@ actor Reminder {
     print("Happy New Year!");
   };
 
-  ignore setTimer(#seconds (solarYearSeconds - abs(now() / 1_000_000_000) % solarYearSeconds),
+  ignore setTimer<system>(#seconds (solarYearSeconds - abs(now() / 1_000_000_000) % solarYearSeconds),
     func () : async () {
-      ignore recurringTimer(#seconds solarYearSeconds, remind);
+      ignore recurringTimer<system>(#seconds solarYearSeconds, remind);
       await remind();
   });
 }
diff --git a/doc/md/examples/grammar.txt b/doc/md/examples/grammar.txt
@@ -52,7 +52,7 @@
 
 <typ_nobin> ::= 
     <typ_pre>
-    <func_sort_opt> ('<' <list(<typ_bind>, ',')> '>')? <typ_un> '->' <typ_nobin>
+    <func_sort_opt> <typ_params_opt> <typ_un> '->' <typ_nobin>
 
 <typ> ::= 
     <typ_nobin>
@@ -66,10 +66,19 @@
 <typ_args> ::= 
     '<' <list(<typ>, ',')> '>'
 
+<inst> ::= 
+    <empty>
+    '<' <list(<typ>, ',')> '>'
+    '<' 'system' (',' <typ>)* '>'
+
+<typ_params_opt> ::= 
+    ('<' <list(<typ_bind>, ',')> '>')?
+    '<' 'system' (',' <typ_bind>)* '>'
+
 <typ_field> ::= 
     'type' <id> ('<' <list(<typ_bind>, ',')> '>')? '=' <typ>
     'var'? <id> ':' <typ>
-    <id> ('<' <list(<typ_bind>, ',')> '>')? <typ_nullary> ':' <typ>
+    <id> <typ_params_opt> <typ_nullary> ':' <typ>
 
 <typ_tag> ::= 
     '#' <id> (':' <typ>)?
@@ -165,7 +174,7 @@
     <exp_post> '[' <exp> ']'
     <exp_post> '.'<nat>
     <exp_post> '.' <id>
-    <exp_post> ('<' <list(<typ>, ',')> '>')? <exp_nullary>
+    <exp_post> <inst> <exp_nullary>
     <exp_post> '!'
     '(' 'system' <exp_post> '.' <id> ')'
 
@@ -293,8 +302,8 @@
     'let' <pat> '=' <exp>
     'type' <id> ('<' <list(<typ_bind>, ',')> '>')? '=' <typ>
     <obj_sort> <id>? (':' <typ>)? '='? <obj_body>
-    <shared_pat_opt> 'func' <id>? ('<' <list(<typ_bind>, ',')> '>')? <pat_plain> (':' <typ>)? <func_body>
-    <shared_pat_opt> <obj_sort>? 'class' <id>? ('<' <list(<typ_bind>, ',')> '>')? <pat_plain> (':' <typ>)? <class_body>
+    <shared_pat_opt> 'func' <id>? <typ_params_opt> <pat_plain> (':' <typ>)? <func_body>
+    <shared_pat_opt> <obj_sort>? 'class' <id>? <typ_params_opt> <pat_plain> (':' <typ>)? <class_body>
 
 <dec> ::= 
     <dec_var>

diff --git a/doc/md/language-manual.md b/doc/md/language-manual.md
@@ -398,7 +398,7 @@ The syntax of a *declaration* is as follows:
   var <id> (: <typ>)? = <exp>                                            mutable
   <sort> <id>? (: <typ>)? =? <obj-body>                                  object
   <shared-pat>? func <id>? <typ-params>? <pat> (: <typ>)? =? <exp>       function
-  type <id> <typ-params>? = <typ>                                        type
+  type <id> <type-typ-params>? = <typ>                                   type
   <shared-pat>? <sort>? class                                            class
     <id>? <typ-params>? <pat> (: <typ>)? <class-body>
 
@@ -556,7 +556,7 @@ Type expressions are used to specify the types of arguments, constraints (a.k.a
 
 ``` bnf
 <typ> ::=                                     type expressions
-  <path> <typ-args>?                            constructor
+  <path> <type-typ-args>?                       constructor
   <sort>? { <typ-field>;* }                     object
   { <typ-tag>;* }                               variant
   { # }                                         empty variant
@@ -758,7 +758,7 @@ See [Stable Regions](stable-regions.md) and library [Region](./base/Region.md) f
 
 ### Constructed types
 
-`<path> <typ-args>?` is the application of a type identifier or path, either built-in (i.e. `Int`) or user defined, to zero or more type **arguments**. The type arguments must satisfy the bounds, if any, expected by the type constructor’s type parameters (see [Well-formed types](#well-formed-types)).
+`<path> <type-typ-args>?` is the application of a type identifier or path, either built-in (i.e. `Int`) or user defined, to zero or more type **arguments**. The type arguments must satisfy the bounds, if any, expected by the type constructor’s type parameters (see [Well-formed types](#well-formed-types)).
 
 Though typically a type identifier, more generally, `<path>` may be a `.`-separated sequence of actor, object or module identifiers ending in an identifier accessing a type component of a value (for example, `Acme.Collections.List`).
 
@@ -867,7 +867,7 @@ In all other positions, `( <typ> )` has the same meaning as `<typ>`.
   <id> : <typ>                                  immutable value
   var <id> : <typ>                              mutable value
   <id> <typ-params>? <typ1> : <typ2>            function value (short-hand)
-  type <id> <typ-params>? = <typ>               type component
+  type <id> <type-typ-params>? = <typ>          type component
 ```
 
 A type field specifies the name and type of a value field of an object, or the name and definition of a type component of an object. The value field names within a single object type must be distinct and have non-colliding hashes. The type component names within a single object type must also be distinct and have non-colliding hashes. Value fields and type components reside in separate name spaces and thus may have names in common.
@@ -876,7 +876,7 @@ A type field specifies the name and type of a value field of an object, or the n
 
 `var <id> : <typ>` specifies a *mutable* field, named `<id>` of type `<typ>`.
 
-`type <id> <typ-params>? = <typ>` specifies a *type* component, with field name `<id>`, abbreviating (parameterized) type `<typ>`.
+`type <id> <type-typ-params>? = <typ>` specifies a *type* component, with field name `<id>`, abbreviating (parameterized) type `<typ>`.
 
 Unlike type declarations, a type component is not, in itself, recursive (though it may abbreviate an existing recursive type).
 In particular, the name `<id>` is not bound in `<typ>` nor in any other fields of the enclosing object type. The name `<id>` only determines the label to use when accessing the definition through a record of this type (using the dot notation).
@@ -910,7 +910,24 @@ When enclosed by a non-`actor` object type, `<id> <typ-params>? <typ1> : <typ2>`
   <id>                                        unconstrained type parameter
 ```
 
-A type constructors, function value or function type may be parameterised by a vector of comma-separated, optionally constrained, type parameters.
+``` bnf
+<type-typ-params> ::=                         type parameters to type constructors
+  < typ-param,* >
+
+<typ-params> ::=                              function type parameters
+  < typ-param,* >                             type parameters
+  < system (, <typ-param>*)) >                system capability prefixed type parameters
+
+<typ-param>
+  <id> <: <typ>                               constrained type parameter
+  <id>                                        unconstrained type parameter
+
+```
+
+A type constructor may be parameterised by a vector of comma-separated, optionally constrained, type parameters.
+
+A function, class constructor or function type may be parameterised by a vector of comma-separated, optionally constrained, type parameters.
+The first of these may be the special, pseudo type parameter `system`.
 
 `<id> <: <typ>` declares a type parameter with constraint `<typ>`. Any instantiation of `<id>` must subtype `<typ>` (at that same instantiation).
 
@@ -920,11 +937,20 @@ The names of type parameters in a vector must be distinct.
 
 All type parameters declared in a vector are in scope within its bounds.
 
+The `system` pseudo-type parameter on function types indicates that a value of that type
+requires `system` capability in order to be called and may itself call functions requiring `system` capability during its execution.
+
 ### Type arguments
 
 ``` bnf
-<typ-args> ::=                                type arguments
+<type-typ-args> ::=                           type arguments to type constructors
   < <typ>,* >
+
+
+<typ-args> ::=                                type arguments to functions
+  < <typ>,* >                                   plain type arguments
+  < system (, <typ>*) >                         system capability prefixed type arguments
+
 ```
 
 Type constructors and functions may take type arguments.
@@ -935,6 +961,20 @@ For a function, the number of type arguments, when provided, must agree with the
 
 Given a vector of type arguments instantiating a vector of type parameters, each type argument must satisfy the instantiated bounds of the corresponding type parameter.
 
+In function calls, supplying the `system` pseudo type argument grants system capability to the function that requires it.
+
+System capability is available only in the following syntactic contexts:
+
+- in the body of an actor expression or actor class;
+- in the body of a (non-`query`) `shared` function, asynchronous function, `async` expression or `async*` expression;
+- in the body of a function or class that is declared with `system` pseudo type parameter;
+- in system functions `preupgrade` and `postupgrade`.
+
+No other context provides `system` capabilities, including `query` and `composite query` methods.
+
+The `<system>` type parameters of shared and asynchronous functions need not be declared.
+
+
 ### Well-formed types
 
 A type `T` is well-formed only if (recursively) its constituent types are well-formed, and:
@@ -1265,8 +1305,8 @@ The declaration `<dec>` of a `system` field must be a manifest `func` declaratio
 | `heartbeat`   | `() -> async ()`                                              | heartbeat action    |
 | `timer`       | `(Nat64 -> ()) -> async ()`                                   | timer action        |
 | `inspect`     | `{ caller : Principal; msg : <Variant>; arg : Blob } -> Bool` | message predicate   |
-| `preupgrade`  | `() -> ()`                                                    | pre upgrade action  |
-| `postupgrade` | `() -> ()`                                                    | post upgrade action |
+| `preupgrade`  | `<system>() -> ()`                                            | pre upgrade action  |
+| `postupgrade` | `<system>() -> ()`                                            | post upgrade action |
 
 -   `heartbeat`, when declared, is called on every Internet Computer subnet **heartbeat**, scheduling an asynchronous call to the `heartbeat` function. Due to its `async` return type, a heartbeat function may send messages and await results. The result of a heartbeat call, including any trap or thrown error, is ignored. The implicit context switch means that the time the heartbeat body is executed may be later than the time the heartbeat was issued by the subnet.
 
@@ -1275,8 +1315,9 @@ The declaration `<dec>` of a `system` field must be a manifest `func` declaratio
 -   `inspect`, when declared, is called as a predicate on every Internet Computer ingress message (with the exception of HTTP query calls). The return value, a `Bool`, indicates whether to accept or decline the given message. The argument type depends on the interface of the enclosing actor (see [Inspect](#inspect)).
 
 -   `preupgrade`, when declared, is called during an upgrade, immediately *before* the (current) values of the (retired) actor’s stable variables are transferred to the replacement actor.
+     Its `<system>` type parameter is implicitly assumed and need not be declared.
 
--   `postupgrade`, when declared, is called during an upgrade, immediately *after* the (replacement) actor body has initialized its fields (inheriting values of the retired actors' stable variables), and before its first message is processed.
+-   `postupgrade`, when declared, is called during an upgrade, immediately *after* the (replacement) actor body has initialized its fields (inheriting values of the retired actors' stable variables), and before its first message is processed. Its `<system>` type parameter is implicitly assumed and need not be declared.
 
 These `preupgrade` and `postupgrade` system methods provide the opportunity to save and restore in-flight data structures (e.g. caches) that are better represented using non-stable types.
 
@@ -1481,7 +1522,7 @@ Evaluation of `var <id> (: <typ>)? = <exp>` proceeds by evaluating `<exp>` to a
 
 ### Type declaration
 
-The declaration `type <id> <typ-params>? = <typ>` declares a new type constructor `<id>`, with optional type parameters `<typ-params>` and definition `<typ>`.
+The declaration `type <id> <type-typ-params>? = <typ>` declares a new type constructor `<id>`, with optional type parameters `<type-typ-params>` and definition `<typ>`.
 
 The declaration `type C< X0 <: T0, …, Xn <: Tn > = U` is well-formed provided:
 
@@ -1698,7 +1739,7 @@ The *class* declaration `<shared-pat>? <sort>? class <id>? <typ-params>? <pat> (
 
 ``` bnf
 <shared-pat>? <sort>? class <id> <typ-params>? <pat> (: <typ>)? <class-body> :=
-  type <id> <typ-params> = <sort> { <typ-field>;* };
+  type <id> <type-typ-params>? = <sort> { <typ-field>;* };
   <shared-pat>? func <id> <typ-params>? <pat> : async? <id> <typ-args> =
     async? <sort> <id_this>? <obj-body>
 ```
@@ -1707,7 +1748,7 @@ where:
 
 -   `<shared-pat>?`, when present, requires `<sort>` == `actor`, and provides access to the `caller` of an `actor` constructor, and
 
--   `<typ-args>?` is the sequence of type identifiers bound by `<typ-params>?` (if any), and
+-   `<typ-args>?` and `<type-typ-params>?` is the sequence of type identifiers bound by `<typ-params>?` (if any), and
 
 -   `<typ-field>;*` is the set of public field types inferred from `<dec-field>;*`.
 

diff --git a/nix/sources.json b/nix/sources.json
@@ -75,10 +75,10 @@
         "homepage": null,
         "owner": "dfinity",
         "repo": "motoko-base",
-        "rev": "712d0587b0c8d2958adf450bd2e1554f2b0ff258",
-        "sha256": "17ghrfgzwlz7hbapy4sq9wbckxqy4xmcj8jpqv68bng9s83zvvvn",
+        "rev": "c7774995a5e271339dff11e16474ece959cc4b60",
+        "sha256": "10zfybm940bdbx1a1z614127r8bnjrxyq3qm6d3zn27l32y248k4",
         "type": "tarball",
-        "url": "https://github.com/dfinity/motoko-base/archive/712d0587b0c8d2958adf450bd2e1554f2b0ff258.tar.gz",
+        "url": "https://github.com/dfinity/motoko-base/archive/c7774995a5e271339dff11e16474ece959cc4b60.tar.gz",
         "url_template": "https://github.com/<owner>/<repo>/archive/<rev>.tar.gz"
     },
     "motoko-matchers": {