BUGFIX: Refactor node tree using Query DTOs

[grebaldi]

fixes: #54

TLDR; This PR replaces the mechanism through which Sitegeist.Archaeopteryx builds its own node tree with a custom-made backend API. The new approach has allowed to move a lot of responsibility from the client application to the server, which is much better suited to answer domain-specific questions. @nezaniel and I have designed this with Neos.Neos.Ui in mind. If the idea works for Archaeopteryx, we may be able to move the approach into the Neos UI as well.

General considerations

The entire approach is pretty much modeled after the ideas outlined in neos/neos-ui#3331. However, the implementation in this PR is more Neos/Flow-idiomatic and requires less reflection. The overall idea is to move responsibility away from the client application to the server by creating a thin API layer, that maps directly to UI concepts.

This PR introduces multiple query data transfer objects (DTOs) with co-located query handlers. Each query has its own namespace beneath Application. Together they form the newly introduced Application layer.

*Query DTOs are final readonly classes with a fromArray static factory method. Each query DTO is accompanied by a *QueryHandler class, that is #[Flow\Scope('singleton')] and performs the actual operation. Each query DTO is also accompanied by a *QueryResult DTO class, that is also final readonly and additionally implements \JsonSerializable. The *QueryResult class is always the return type of the respective *QueryHandler->handle method.

There are some supporting DTO classes (like TreeNode or Breadcrumb). Those classes are usually co-located with the *QueryResult- or *Query DTO that are using them. In case multiple queries or query results need the same DTO, those DTOs have been moved into the Application\Shared namespace.

Each query has its own Controller. For this, a little bit of framework code has been introduced to reduce boilerplate. But in principle, the controllers do nothing more but to implement Flow's ControllerInterface and run their logic directly in processRequest (so: no actions or any other MVC concepts). All query controllers have very limited responsibility: Deserialize the query, invoke the query handler, serialize and return the query result. Everything else is responsibility of the query handler.

Also, all query controllers are folded into the Neos backend authentication. Neos.Neos:AbstractEditor is given access by default.

The Queries

Query GetTree

GetTree is used to render the node tree

Property Name	Type	Description
workspaceName	string	The name of the workspace the user is currently looking at.
dimensionValues	array	The exact combination of dimension values the user is currently looking at.
startingPoint	NodePath	The starting point for fetching the tree. (This is reflected by startingPoint in the editor configuration)
loadingDepth	int	The maximum traversal depth for fetching the tree. (This is reflected by loadingDepth in the editor configuration)
baseNodeTypeFilter	string	A filter string to match the type of every node in the tree against. Nodes with a type that does not match this filter never show up in the tree. (This is reflected by baseNodeType in the editor configuration)
linkableNodeTypes	NodeTypeNames	A list of node type names to mark all nodes that are allowed to be used as link targets. (This is reflected by allowedNodeTypes in the editor configuration)
narrowNodeTypeFilter	string	An additional node type filter selected by the user. If this is set, every node with a type that matches this filter will be retrieved and the tree will be build from the bottom-up.
searchTerm	string	A search term to find nodes by their property contents. If this is set, every node with a type that matches the search term will be retrieved and the tree will be build from the bottom-up.
selectedNodeId (optional)	NodeAggregateIdentifier	The identifier of the node that is currently selected. If this is set, the selected node (if it exists) will be guaranteed to be part of the resulting tree. If it resides deeper in the tree than loadingDepth would allow, it's entire branch will be build from the bottom-up

The GetTree query returns a hierarchy of TreeNode DTOs, which contain all the information the UI needs to render a tree. The tree is rendered with the node at startingPoint at its root.

By default, the depth of the rendered tree will be limited by loadingDepth. Descendants that are excluded thusly, can later be loaded using the GetChildrenForTreeNode query.

If searchTerm is set, the query will perform a search instead and return a tree containing every node that matches the searchTerm plus its ancestors (until startingPoint).

The query is informed by the editor configuration, by what is currently set in the Neos UI document tree (initially) and by user input.

If a node is selected whose depth in the tree exceeds loadingDepth and the Archaeopteryx dialog is opened after that, the tree will be loaded down to the depth of the selected node, including all ancestors and (ancestor-)siblings up to startingPoint.

EXAMPLE

Query:

https://my-neos-dev-instance.ddev.site/sitegeist/archaeopteryx/get-tree?workspaceName=user-admin&dimensionValues%5Blanguage%5D%5B%5D=en_US&dimensionValues%5BtargetGroup%5D%5B%5D=private&startingPoint=%2Fsites%2Fvendor-site&loadingDepth=1&baseNodeTypeFilter=Vendor.Site%3ADocument&narrowNodeTypeFilter=&searchTerm=&selectedNodeId=60c3de73-bb08-41fc-b73a-0c252cdf41d6

Result:

{
    "success": {
        "root": {
            "nodeAggregateIdentifier": "7d1a814b-f19b-4801-8621-eb8889fe638b",
            "icon": "globe",
            "label": "Example Site",
            "nodeTypeLabel": "Home Page",
            "isMatchedByFilter": true,
            "isLinkable": true,
            "isDisabled": false,
            "isHiddenInMenu": false,
            "hasScheduledDisabledState": false,
            "hasUnloadedChildren": false,
            "children": [
                {
                    "nodeAggregateIdentifier": "036b0e4c-e931-56b9-db53-7b4a9c5ded26",
                    "icon": "exclamation-triangle",
                    "label": "404"
                    // ...
                },
                {
                    "nodeAggregateIdentifier": "1e2948de-1e73-4688-9ae7-29a932e43eed",
                    "icon": "file",
                    "label": "About us"
                    // ...
                }
                // ...
            ]
        }
    }
}

baseNodeTypeFilter vs. narrowNodeTypeFilter vs. allowedNodeTypes (???)

I found it a bit confusing to differentiate between all those filters in relation to the tree.

There is a thing called baseNodeTypeFilter, which is configured via the editor configuration (analogous to how that's done with the document and content trees in Neos UI). The configuration key just says baseNodeType, but it is actually a filter string. This filter determines which node types can show up in the tree. So, any node type that doesn't match this filter will never show up in the tree, regardless of any other configuration or user selection.

Then there's a thing, that I called narrowNodeTypeFilter in this PR. This is the user-selected node type filter. If a user selects a filter option, the GetTreeQuery will actually perform a search and return all nodes that match the node type filter plus all of their ancestors.

There's also the configuration option allowedNodeTypes. This has nothing to do with filters at all. This option determines which node types are allowed to be linked (introduced in #11).

DTO TreeNode

The various states a tree node can be in

Property Name	Type	Description
nodeAggregateIdentifier	NodeAggregateIdentifier	The ID of the (CR) Node aggregate that is represented by this tree node
icon	string	The icon for the tree node (by default taken from the node type's UI configuration)
label	string	The label for the tree node (by default this is just the (CR) node label)
nodeTypeLabel	string	The node type label (This is needed for the title attribute of the tree node icon)
isMatchedByFilter	bool	Flag that tells us whether this tree node matches the filter (searchTerm and/or narrowNodeTypeFilter) given in the query. If this is false the tree node will appear with reduced opacity
isLinkable	bool	Flag that tells us whether this tree node is allowed to be linked to (as per allowedNodeTypes configuration)
isDisabled	bool	Flag that tells us whether the node is disabled (or pre-9.0: "hidden"). If this is true, a little red icon will mark this tree node as disabled
isHiddenInMenu	bool	Flag that tells us whether the node is hidden in menus (or pre-9.0: "hidden in index"). If this is true the tree node will appear with reduced opacity
hasScheduledDisabledState	bool	Flag that tells us whether this node is scheduled to be disabled (determined by hiddenBeforeDateTime and hiddenAfterDateTime). If this is true, a little clock icon will mark this tree node as scheduled to be disabled
hasUnloadedChildren	bool	Flag that indicates whether this node has children that are not included in this result set because of loadingDepth
children	TreeNodes	The children of this tree node. If the depth of this node equals loadingDepth, this array will be empty and hasUnloadedChildren will be true

Query GetChildrenForTreeNode

Peek.2024-05-15.15-04.GetChildrenForTreeNode.mp4

Loading children that have been excluded due to loadingDepth

Property Name	Type	Description
workspaceName	string	The name of the workspace the user is currently looking at.
dimensionValues	array	The exact combination of dimension values the user is currently looking at.
treeNodeId	NodeAggregateIdentifier	The ID of the (CR) node aggregate whose children shall be retrieved
nodeTypeFilter	string	A node type filter to match the children against. Children that do not match this filter will not be loaded.
linkableNodeTypes	NodeTypeNames	A list of node type names to mark all nodes that are allowed to be used as link targets. (This is reflected by allowedNodeTypes in the editor configuration)

This query loads a single level of tree nodes for the parent identified by treeNodeId. This is used to load children that have been initially excluded due to loadingDepth in the GetTree query. For plausible results, the workspaceName, dimensionValues and linkableNodeTypes properties should be the same as with the initial GetTree query.

EXAMPLE

Query:

https://my-neos-dev-instance.ddev.site/sitegeist/archaeopteryx/get-children-for-tree-node?workspaceName=user-admin&dimensionValues%5Blanguage%5D%5B%5D=en_US&dimensionValues%5BtargetGroup%5D%5B%5D=private&treeNodeId=1e2948de-1e73-4688-9ae7-29a932e43eed&nodeTypeFilter=Vendor.Site%3ADocument

Result:

{
    "success": {
        "children": [
            {
                "nodeAggregateIdentifier": "0ba7e951-2e2c-4447-9ac9-6f84995e219d",
                "icon": "map",
                "label": "Where to find us",
                "nodeTypeLabel": "Map",
                "isMatchedByFilter": true,
                "isLinkable": true,
                "isDisabled": false,
                "isHiddenInMenu": false,
                "hasScheduledDisabledState": false,
                "hasUnloadedChildren": true,
                "children": []
            },
            // ...
        ]
    }
}

Query GetNodeSummary

The node summary used for Archaeopteryx' main dialog

Property Name	Type	Description
workspaceName	string	The name of the workspace the user is currently looking at.
dimensionValues	array	The exact combination of dimension values the user is currently looking at.
nodeId	NodeAggregateIdentifier	The ID of the (CR) node aggregate that shall be summarized

When a link is selected, Archaeopteryx displays a panel in its main dialog that summarizes the link. The GetNodeSummary query informs this panel.

EXAMPLE

Query:

https://my-neos-dev-instance.ddev.site/sitegeist/archaeopteryx/get-node-summary?workspaceName=user-admin&dimensionValues%5Blanguage%5D%5B%5D=en_US&dimensionValues%5BtargetGroup%5D%5B%5D=private&nodeId=0ba7e951-2e2c-4447-9ac9-6f84995e219d

Result:

{
    "success": {
        "icon": "map",
        "label": "Where to find us",
        "uri": "node://0ba7e951-2e2c-4447-9ac9-6f84995e219d",
        "breadcrumbs": [
            {
                "icon": "globe",
                "label": "Example Site"
            },
            {
                "icon": "file",
                "label": "About us"
            },
            {
                "icon": "map",
                "label": "Where to find us"
            }
        ]
    }
}

Query GetNodeTypeFilterOptions

The node type filter options as displayed in the select box in Archaeopteryx' main dialog

Property Name	Type	Description
baseNodeTypeFilter	string	If no nodeTypePresets are configured, the resulting options will consist of all non-abstract node types that match this filter string

The result of the GetNodeTypeFilterOptions query informs the node type filter select box in Archaeopteryx' main dialog.

Analogous to Neos.Neos.Ui, it will use nodeTreePresets to calculate the filter options, if any are configured. If not, the filter options will simply be a list of all non-abstract node types that match the given baseNodeTypeFilter.

EXAMPLE

Query:

https://my-neos-dev-instance.ddev.site/sitegeist/archaeopteryx/get-node-type-filter-options?baseNodeTypeFilter=Vendor.Site%3ADocument

Result:

{
  "success": {
    "options": [
      {
        "value": "Vendor.Site:Document,!Vendor.Site:Mixin.Type.Foo",
        "icon": "filter",
        "label": "Not the Foos"
      },
      {
        "value": "Vendor.Site:Document,!Vendor.Site:Mixin.Type.Bar",
        "icon": "filter",
        "label": "Not the Bars"
      }
    ]
  }
}

[mhsdesign]

Thanks a lot for your effort. Do we want this as a minor eg 1.5 or as a full major 2.0?

[grebaldi]

As a bugfix, this would technically be a patch release. There's no breaking change API-wise in here. It's just a lot of code.

If this feels uncomfortable, I'd go for a minor release.