Analyzer-Wide Common Semantics API (#22)

work/book/src/SUMMARY.md

@@ -68,6 +68,7 @@
     - [Granularity](semantics/granularity.md)
     - [The Analyzer](semantics/the-analyzer.md)
     - [Tasks Management](semantics/tasks-management.md)
+    - [Multi-File Analysis](semantics/multi-file-analysis.md)
     - [Language Server Design](semantics/language-server-design.md)
     - [Configuration Issues](semantics/configuration-issues.md)
     - [Code Diagnostics](semantics/code-diagnostics.md)

work/book/src/semantics/multi-file-analysis.md

@@ -0,0 +1,173 @@
+<!------------------------------------------------------------------------------
+  This file is a part of the "Lady Deirdre" work,
+  a compiler front-end foundation technology.
+
+  This work is proprietary software with source-available code.
+
+  To copy, use, distribute, and contribute to this work, you must agree to
+  the terms of the General License Agreement:
+
+  https://github.com/Eliah-Lakhin/lady-deirdre/blob/master/EULA.md.
+
+  The agreement grants you a Commercial-Limited License that gives you
+  the right to use my work in non-commercial and limited commercial products
+  with a total gross revenue cap. To remove this commercial limit for one of
+  your products, you must acquire an Unrestricted Commercial License.
+
+  If you contribute to the source code, documentation, or related materials
+  of this work, you must assign these changes to me. Contributions are
+  governed by the "Derivative Work" section of the General License
+  Agreement.
+
+  Copying the work in parts is strictly forbidden, except as permitted under
+  the terms of the General License Agreement.
+
+  If you do not or cannot agree to the terms of this Agreement,
+  do not use this work.
+
+  This work is provided "as is" without any warranties, express or implied,
+  except to the extent that such disclaimers are held to be legally invalid.
+
+  Copyright (c) 2024 Ilya Lakhin (Илья Александрович Лахин).
+  All rights reserved.
+------------------------------------------------------------------------------->
+
+# Multi-File Analysis
+
+A compilation project usually consists of multiple compilation units that are
+semantically connected to each other.
+
+For example, a Java file may declare a class with signatures that reference
+classes declared in other files within the same Java package.
+
+To establish semantic relationships between these compilation units, you can
+define a special analyzer-wide feature object.
+
+From the [Shared Semantics](todo) example:
+
+```rust,noplayground
+#[derive(Node)]
+
+// Defines a semantic feature that is shared across all documents in the Analyzer.
+#[semantics(CommonSemantics)]
+
+pub enum SharedSemanticsNode {
+    // ...
+}
+
+#[derive(Feature)]
+#[node(SharedSemanticsNode)]
+pub struct CommonSemantics {
+    pub modules: Slot<SharedSemanticsNode, HashMap<String, Id>>,
+}
+```
+
+## Common Semantics
+
+The common semantics feature is a typical feature object, except that it is not
+bound to any specific node within a compilation unit and is instantiated during
+the creation of the Analyzer.
+
+This feature is not tied to any syntax tree scope. Therefore, its members will
+not be directly invalidated during the editing of the Analyzer's documents.
+
+However, the members of this feature are part of the semantic graph and are
+subject to the normal rules of the semantic graph, such as the prohibition of
+cycles between computable functions.
+
+Common semantic features typically include:
+
+- Analyzer-wide reducing attributes, such as an attribute that collects all
+  syntax and semantic issues detected across all managed documents.
+- External configuration metadata specified via the system of
+  [Slots](https://docs.rs/lady-deirdre/2.0.1/lady_deirdre/analysis/struct.Slot.html).
+  For instance, a map between file names and their document IDs within the
+  Analyzer (as in the example above).
+
+You can access common semantics both inside and outside of computable
+functions. Inside a computable function, you can access common semantics
+using the [AttrContext::common](https://docs.rs/lady-deirdre/2.0.1/lady_deirdre/analysis/struct.AttrContext.html#method.common)
+method. To access the semantics outside, you would use the
+[AbstractTask::common](https://docs.rs/lady-deirdre/2.0.1/lady_deirdre/analysis/trait.AbstractTask.html#method.common)
+method.
+
+```rust,noplayground
+#[derive(Clone, PartialEq, Eq)]
+pub enum KeyResolution {
+    Unresolved,
+    Recusrive,
+    Number(usize),
+}
+
+impl Computable for KeyResolution {
+    type Node = SharedSemanticsNode;
+
+    fn compute<H: TaskHandle, S: SyncBuildHasher>(
+        context: &mut AttrContext<Self::Node, H, S>,
+    ) -> AnalysisResult<Self> {
+        // ...
+
+        // Reading the common semantics inside the computable function.
+        let modules = context.common().modules.read(context).unwrap_abnormal()?;
+        
+        // ...
+    }
+}
+
+let handle = TriggerHandle::new();
+let mut task = analyzer.mutate(&handle, 1).unwrap();
+
+let doc_id = task.add_mutable_doc("x = 10; y = module_2::b; z = module_2::c;");
+doc_id.set_name("module_1");
+
+// Modifying the Slot value of the common semantics outside.
+task.common()
+    .modules
+    .mutate(&task, |modules| {
+        let _ = modules.insert(String::from("module_1"), doc_id);
+
+        true
+    })
+    .unwrap();
+```
+
+## Slots
+
+The primary purpose of a [Slot](https://docs.rs/lady-deirdre/2.0.1/lady_deirdre/analysis/struct.Slot.html)
+is to provide a convenient mechanism for injecting configuration metadata
+external to the Analyzer into the semantic graph. For instance, mapping between
+file system names and the Analyzer's document IDs can be injected through a
+common semantics Slot.
+
+Slot is a special feature of the semantic graph that is quite similar to
+attributes, except that a Slot does not have an associated computable function.
+Instead, Slots have associated values of a specified type (the second generic
+argument of the `Slot<Node, ValueType>` signature).
+
+You can [snapshot](https://docs.rs/lady-deirdre/2.0.1/lady_deirdre/analysis/struct.Slot.html#method.snapshot)
+the current Slot value outside of computable functions, and you can
+[read](https://docs.rs/lady-deirdre/2.0.1/lady_deirdre/analysis/struct.Slot.html#method.read)
+Slot values within the computable functions of attributes, thereby subscribing
+those attributes to changes in the Slot, much like with normal attributes.
+
+By default, Slot values are set to the `Default` of the value type. You can
+modify the content of the Slot value using the
+[Slot::mutate](https://docs.rs/lady-deirdre/2.0.1/lady_deirdre/analysis/struct.Slot.html#method.mutate)
+method with a mutable (or exclusive) task.
+
+```rust,noplayground
+task.common()
+    .modules
+    // The `task` is a MutationTask or an ExclusiveTask.
+    //
+    // The provided callback accepts a mutable reference to the current
+    // value of the Slot, and returns a boolean flag indicating whether the
+    // value has changed.
+    .mutate(&task, |modules| {
+        let _ = modules.insert(String::from("module_1"), doc_id);
+
+        // Indicates that the `modules` content has been changed.
+        true
+    })
+    .unwrap();
+```

work/crates/derive/src/feature/output.rs

@@ -165,6 +165,11 @@ impl ToTokens for FeatureInput {
                     &#core::analysis::NIL_ATTR_REF
                 }
 
+                #[inline(always)]
+                fn slot_ref(&self) -> &#core::analysis::SlotRef {
+                    &#core::analysis::NIL_SLOT_REF
+                }
+
                 fn feature(&self, key: #core::syntax::Key)
                     -> #core::analysis::AnalysisResult<&dyn #core::analysis::AbstractFeature>
                 {

work/crates/derive/src/lib.rs

@@ -390,6 +390,16 @@ pub fn token(input: proc_macro::TokenStream) -> proc_macro::TokenStream {
 ///
 /// // Optional.
 /// //
+/// // Specifies the semantic entry-point for the common semantics shared across
+/// // all documents in the Analyzer.
+/// //
+/// // The type of this field must implement the `Feature` trait.
+/// //
+/// // If omitted, the default common semantics will be `VoidFeature<MyNode>`
+/// #[semantics(<common semantics type>)]
+///
+/// // Optional.
+/// //
 /// // Defines the expression that will be automatically parsed zero or more
 /// // times between every consumed token in the node's parse rules.
 /// //

work/crates/derive/src/node/inheritance.rs

@@ -391,6 +391,24 @@ impl Inheritance {
         Some(quote_spanned!(span=> Self::#ident { #field_ident: _0, .. } => #body,))
     }
 
+    pub(super) fn compile_slot_ref(&self) -> Option<TokenStream> {
+        let (field_ident, field_ty) = self.semantics.as_ref()?;
+
+        let body = {
+            let span = field_ty.span();
+            let core = span.face_core();
+
+            quote_spanned!(span=>
+                <#field_ty as #core::analysis::AbstractFeature>::slot_ref(_0)
+            )
+        };
+
+        let ident = &self.ident;
+        let span = ident.span();
+
+        Some(quote_spanned!(span=> Self::#ident { #field_ident: _0, .. } => #body,))
+    }
+
     pub(super) fn compile_feature_getter(&self) -> Option<TokenStream> {
         let (field_ident, field_ty) = self.semantics.as_ref()?;
 

work/crates/derive/src/node/input.rs

@@ -73,6 +73,7 @@ pub struct NodeInput {
     pub(super) generics: ParserGenerics,
     pub(super) token: Type,
     pub(super) classifier: Option<Type>,
+    pub(super) common: Option<Type>,
     pub(super) trivia: Option<Rule>,
     pub(super) recovery: Option<Recovery>,
     pub(crate) dump: Dump,
@@ -131,6 +132,7 @@ impl TryFrom<DeriveInput> for NodeInput {
 
         let mut token = None;
         let mut classifier = None;
+        let mut common = None;
         let mut trivia = None;
         let mut recovery = None;
         let mut dump = Dump::None;
@@ -165,6 +167,14 @@ impl TryFrom<DeriveInput> for NodeInput {
                     classifier = Some(attr.parse_args::<Type>()?);
                 }
 
+                "semantics" => {
+                    if common.is_some() {
+                        return Err(error!(span, "Duplicate Semantics attribute.",));
+                    }
+
+                    common = Some(attr.parse_args::<Type>()?);
+                }
+
                 "trivia" => {
                     if trivia.is_some() {
                         return Err(error!(span, "Duplicate Trivia attribute.",));
@@ -562,6 +572,7 @@ impl TryFrom<DeriveInput> for NodeInput {
             generics,
             token,
             classifier,
+            common,
             trivia,
             recovery,
             dump,

work/crates/derive/src/node/output.rs

@@ -166,6 +166,7 @@ impl NodeInput {
         let capacity = self.variants.len();
 
         let mut attr_ref = Vec::with_capacity(capacity);
+        let mut slot_ref = Vec::with_capacity(capacity);
         let mut feature_getter = Vec::with_capacity(capacity);
         let mut feature_keys = Vec::with_capacity(capacity);
 
@@ -175,6 +176,7 @@ impl NodeInput {
             }
 
             attr_ref.push(variant.inheritance.compile_attr_ref());
+            slot_ref.push(variant.inheritance.compile_slot_ref());
             feature_getter.push(variant.inheritance.compile_feature_getter());
             feature_keys.push(variant.inheritance.compile_feature_keys());
         }
@@ -192,6 +194,15 @@ impl NodeInput {
                     }
                 }
 
+                fn slot_ref(&self) -> &#core::analysis::SlotRef {
+                    match self {
+                        #( #slot_ref )*
+
+                        #[allow(unreachable_patterns)]
+                        _ => &#core::analysis::NIL_SLOT_REF,
+                    }
+                }
+
                 #[allow(unused_variables)]
                 fn feature(&self, key: #core::syntax::Key)
                     -> #core::analysis::AnalysisResult<&dyn #core::analysis::AbstractFeature>
@@ -228,6 +239,11 @@ impl NodeInput {
             None => quote_spanned!(span=> #core::analysis::VoidClassifier::<Self>),
         };
 
+        let common = match &self.common {
+            Some(ty) => ty.to_token_stream(),
+            None => quote_spanned!(span=> #core::analysis::VoidFeature::<Self>),
+        };
+
         let (impl_generics, type_generics, where_clause) = self.generics.ty.split_for_impl();
 
         let capacity = self.variants.len();
@@ -264,6 +280,8 @@ impl NodeInput {
             {
                 type Classifier = #classifier;
 
+                type CommonSemantics = #common;
+
                 #[allow(unused_variables)]
                 fn init<
                     H: #core::analysis::TaskHandle,

work/crates/examples/readme.md

@@ -37,8 +37,8 @@
 This crate contains examples showcasing the core features of Lady Deirdre.
 
 The source code of each example is accompanied by detailed explanations and
-comments in the [User Guide](https://lady-deirdre.lakhin.com/). Therefore, it is recommended to explore them
-alongside the corresponding chapters of the guide.
+comments in the [User Guide](https://lady-deirdre.lakhin.com/). Therefore, it is
+recommended to explore them alongside the corresponding chapters of the guide.
 
 Each example is located in its own crate module within the "src" directory.
 The root "mod.rs" file of each module includes runnable tests that demonstrate
@@ -76,6 +76,18 @@ specific features of the example.
 
   Relevant User Guide chapter: [Semantics](https://lady-deirdre.lakhin.com/semantics/semantics.html).
 
+- [Shared Semantics](https://github.com/Eliah-Lakhin/lady-deirdre/tree/master/work/crates/examples/src/shared_semantics).
+
+  Illustrates how to organize cross-file semantic connections.
+
+  The source code of the files contains a set of key-value pairs, where the key
+  is any identifier, and the value is either a numeric value or a reference to
+  another identifier within the same file or a different one. This example
+  demonstrates resolving key values through a system of references down to their
+  numeric values.
+
+  Relevant User Guide chapter: [Multi-File Analysis](https://lady-deirdre.lakhin.com/semantics/multi-file-analysis.html).
+
 - [JSON Formatter](https://github.com/Eliah-Lakhin/lady-deirdre/tree/master/work/crates/examples/src/json_formatter).
 
   Shows how to use the source code formatter tools of Lady Deirdre to implement

work/crates/examples/src/lib.rs

@@ -40,3 +40,4 @@ pub mod expr_parser;
 pub mod json_formatter;
 pub mod json_grammar;
 pub mod json_highlight;
+pub mod shared_semantics;