Allow JS interpreters (and global state) to be reused (microsoft#5564)

(cherry picked from commit a0dfdfd)

# Conflicts:
#	.threading_canary
#	CHANGELOG.md
#	doc/schemas/gov_openapi.json
#	samples/apps/logging/js/app.json
#	src/enclave/enclave.h
#	src/node/rpc/member_frontend.h

.threading_canary

@@ -1 +1 @@
-THIS looks like a job for Threading Canard!y!1!..
+...

doc/audit/builtin_maps.rst

@@ -284,7 +284,7 @@ JavaScript engine version of the module cache, accessible by JavaScript endpoint
 
 ``js_runtime_options``
 ~~~~~~~~~~~~~~~~~~~~~~
-QuickJS runtime memory, accessible by Javascript endpoint function.
+QuickJS runtime options, used to configure runtimes created by CCF.
 
 **Key** Sentinel value 0, represented as a little-endian 64-bit unsigned integer.
 
@@ -294,6 +294,15 @@ QuickJS runtime memory, accessible by Javascript endpoint function.
    :project: CCF
    :members:
 
+``interpreter.flush``
+~~~~~~~~~~~~~~~~~~~~~~
+Used by transactions that set the JS application to signal to the interpreter cache system
+that existing instances need to be flushed.
+
+**Key** Sentinel value 0, represented as a little-endian 64-bit unsigned integer.
+
+**Value** Boolean, represented as JSON.
+
 ``endpoints``
 ~~~~~~~~~~~~~
 

doc/build_apps/js_app_bundle.rst

@@ -340,3 +340,59 @@ If CCF is updated and introduces a newer JavaScript engine version, then any pre
     }
 
 .. note:: The operator RPC :http:GET:`/node/js_metrics` returns the size of the bytecode and whether it is used. If it is not used, then either no bytecode is stored or it needs to be re-compiled due to a CCF update.
+
+Reusing interpreters
+~~~~~~~~~~~~~~~~~~~~
+
+By default, every request executes in a freshly-constructed JS interpreter. This provides extremely strict sandboxing - the only interaction with other requests is transactionally via the KV - and so forbids the sharing of any global state. For some applications, this may lead to unnecessarily duplicated work.
+
+For instance, if your application needs to construct a large, immutable singleton object to process a request, that construction cost will be paid in each and every request. Requests could execute significantly faster if they were able to access and reuse a previously-constructed object, rather than constructing their own. JS libraries designed for other runtimes (such as Node) may benefit from this, as they expect to have a persistent global state.
+
+CCF supports this pattern with `interpreter reuse`. Applications may opt-in to persisting an interpreter, and all of its global state, to be reused by multiple requests. This means that expensive initialisation work can be done once, and the resulting objects stashed in the global state where future requests will reuse them.
+
+Note that this removes the sandboxing protections described above. If the contents of the global state change the result of a request's execution, then the execution will no longer be reproducible from the state recorded in the ledger, since the state of the interpreter cache will not be recorded. This should be avoided - reuse should only be used to make a handler `faster`, not to `change its behaviour`.
+
+This behaviour is controlled in ``app.json``, with the ``"interpreter_reuse"`` property on each endpoint. The default behaviour, taken when the field is omitted, is to avoid any interpreter reuse, providing strict sandboxing safety. To reuse an interpreter, set ``"interpreter_reuse"`` to an object of the form ``{"key": "foo"}``, where ``foo`` is an arbitrary, app-defined string. Interpreters will be shared between endpoints where this string matches. For instance:
+
+.. code-block:: json
+
+    {
+      "endpoints": {
+        "/admin/modify": {
+          "post": {
+            "js_module": ...,
+            "interpreter_reuse": {"key": "admin_interp"}
+          }
+        },
+        "/admin/admins": {
+          "get": {
+            "js_module": ...,
+            "interpreter_reuse": {"key": "admin_interp"}
+          },
+          "post": {
+            "js_module": ...,
+            "interpreter_reuse": {"key": "admin_interp"}
+          }
+        },
+        "/sum/{a}/{b}": {
+          "get": {
+            "js_module": ...,
+            "interpreter_reuse": {"key": "sum"}
+          }
+        },
+        "/fast/and/small": {
+          "get": {
+            "js_module": ...
+            // No "interpreter_reuse" field
+          }
+        }
+      }
+    }
+
+In this example, each CCF node will store up-to 2 interpreters, and divides the endpoints into 3 classes:
+
+- Requests to ``POST /admin/modify``, ``GET /admin/admins``, and ``POST /admin/admins`` will reuse the same interpreter (keyed by the string ``"admin_interp"``).
+- Requests to ``GET /sum/{a}/{b}`` will use a separate interpreter (keyed by the string ``"sum"``).
+- Requests to ``GET /fast/and/small`` will `not reuse any interpreters`, instead getting a fresh interpreter for each incoming request.
+
+Note that ``"interpreter_reuse"`` describes when interpreters `may` be reused, but does not ensure that an interpreter `is` reused. A CCF node may decide to evict interpreters to limit memory use, or for parallelisation. Additionally, interpreters are node-local, are evicted for semantic safety whenever the JS application is modified, and only constructed on-demand for an incoming request (so the first request will see no performance benefit, since it includes the initialisation cost that later requests can skip). In short, this reuse should be seen as a best-effort optimisation - when it takes effect it will make many request patterns significantly faster, but it should not be relied upon for correctness.

doc/schemas/gov_openapi.json

@@ -165,6 +165,9 @@
           "forwarding_required": {
             "$ref": "#/components/schemas/ForwardingRequired"
           },
+          "interpreter_reuse": {
+            "$ref": "#/components/schemas/InterpreterReusePolicy"
+          },
           "js_function": {
             "$ref": "#/components/schemas/string"
           },
@@ -307,11 +310,29 @@
       "HttpMethod": {
         "type": "string"
       },
+      "InterpreterReusePolicy": {
+        "oneOf": [
+          {
+            "properties": {
+              "key": {
+                "type": "string"
+              }
+            },
+            "required": [
+              "key"
+            ],
+            "type": "object"
+          }
+        ]
+      },
       "JSRuntimeOptions": {
         "properties": {
           "log_exception_details": {
             "$ref": "#/components/schemas/boolean"
           },
+          "max_cached_interpreters": {
+            "$ref": "#/components/schemas/uint64"
+          },
           "max_execution_time_ms": {
             "$ref": "#/components/schemas/uint64"
           },
@@ -1270,7 +1291,7 @@
   "info": {
     "description": "This API is used to submit and query proposals which affect CCF's public governance tables.",
     "title": "CCF Governance API",
-    "version": "4.1.3"
+    "version": "4.1.2"
   },
   "openapi": "3.0.0",
   "paths": {
@@ -1613,6 +1634,28 @@
         }
       }
     },
+    "/gov/kv/interpreter/flush": {
+      "get": {
+        "responses": {
+          "200": {
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/boolean"
+                }
+              }
+            },
+            "description": "Default response description"
+          },
+          "default": {
+            "$ref": "#/components/responses/default"
+          }
+        },
+        "x-ccf-forwarding": {
+          "$ref": "#/components/x-ccf-forwarding/sometimes"
+        }
+      }
+    },
     "/gov/kv/js_runtime_options": {
       "get": {
         "responses": {

doc/schemas/node_openapi.json

@@ -479,6 +479,9 @@
           "bytecode_used": {
             "$ref": "#/components/schemas/boolean"
           },
+          "max_cached_interpreters": {
+            "$ref": "#/components/schemas/uint64"
+          },
           "max_execution_time": {
             "$ref": "#/components/schemas/uint64"
           },
@@ -494,7 +497,8 @@
           "bytecode_used",
           "max_heap_size",
           "max_stack_size",
-          "max_execution_time"
+          "max_execution_time",
+          "max_cached_interpreters"
         ],
         "type": "object"
       },

include/ccf/endpoint.h

@@ -83,6 +83,23 @@ namespace ccf::endpoints
      {Mode::ReadOnly, "readonly"},
      {Mode::Historical, "historical"}});
 
+  struct InterpreterReusePolicy
+  {
+    enum
+    {
+      KeyBased
+    } kind;
+
+    std::string key;
+
+    bool operator==(const InterpreterReusePolicy&) const = default;
+  };
+
+  void to_json(nlohmann::json& j, const InterpreterReusePolicy& grp);
+  void from_json(const nlohmann::json& j, InterpreterReusePolicy& grp);
+  std::string schema_name(const InterpreterReusePolicy*);
+  void fill_json_schema(nlohmann::json& schema, const InterpreterReusePolicy*);
+
   struct EndpointProperties
   {
     /// Endpoint mode
@@ -99,13 +116,23 @@ namespace ccf::endpoints
     std::string js_module;
     /// JavaScript function name
     std::string js_function;
+    /// Determines how JS interpreters may be reused between multiple calls,
+    /// sharing global state in potentially unsafe ways. The default empty value
+    /// means no reuse is permitted.
+    std::optional<InterpreterReusePolicy> interpreter_reuse = std::nullopt;
   };
 
   DECLARE_JSON_TYPE_WITH_OPTIONAL_FIELDS(EndpointProperties);
   DECLARE_JSON_REQUIRED_FIELDS(
     EndpointProperties, forwarding_required, authn_policies);
   DECLARE_JSON_OPTIONAL_FIELDS(
-    EndpointProperties, openapi, openapi_hidden, mode, js_module, js_function);
+    EndpointProperties,
+    openapi,
+    openapi_hidden,
+    mode,
+    js_module,
+    js_function,
+    interpreter_reuse);
 
   struct EndpointDefinition
   {

include/ccf/service/tables/jsengine.h

@@ -23,13 +23,18 @@ namespace ccf
     /// NOTE: this is a security risk as it may leak sensitive information,
     ///       albeit to the caller only.
     bool return_exception_details = false;
+    /// @brief how many interpreters may be cached in-memory for future reuse
+    size_t max_cached_interpreters = 10;
   };
 
   DECLARE_JSON_TYPE_WITH_OPTIONAL_FIELDS(JSRuntimeOptions)
   DECLARE_JSON_REQUIRED_FIELDS(
     JSRuntimeOptions, max_heap_bytes, max_stack_bytes, max_execution_time_ms)
   DECLARE_JSON_OPTIONAL_FIELDS(
-    JSRuntimeOptions, log_exception_details, return_exception_details);
+    JSRuntimeOptions,
+    log_exception_details,
+    return_exception_details,
+    max_cached_interpreters);
 
   using JSEngine = ServiceValue<JSRuntimeOptions>;
 

include/ccf/service/tables/modules.h

@@ -16,6 +16,7 @@ namespace ccf
   using ModulesQuickJsBytecode =
     kv::RawCopySerialisedMap<std::string, std::vector<uint8_t>>;
   using ModulesQuickJsVersion = kv::RawCopySerialisedValue<std::string>;
+  using InterpreterFlush = ServiceValue<bool>;
 
   namespace Tables
   {
@@ -24,5 +25,7 @@ namespace ccf
       "public:ccf.gov.modules_quickjs_bytecode";
     static constexpr auto MODULES_QUICKJS_VERSION =
       "public:ccf.gov.modules_quickjs_version";
+    static constexpr auto INTERPRETER_FLUSH =
+      "public:ccf.gov.interpreter.flush";
   }
 }