Is there a better way to organise persistent data?

[chris-steele]

Feature description

Adapt content is hierarchical, but currently all models are stored as a flat list of documents. If we want to locate part of this hierarchy, for example, a page and its descendant content, we have to do this in the application tier. This means loading the entire course from the database and examining the documents.

An example use case is in the client: to edit content the user will always first request a single page. Currently the entire course is retrieved and transmitted over the network. The client then processes the documents to locate those of interest. It would be faster and more efficient if only the required documents were retrieved.

Possible solution could use https://www.mongodb.com/docs/manual/tutorial/model-tree-structures-with-ancestors-array/

Ref #39, #40 and adapt-security/adapt-authoring-ui#322

Can you work on this feature?

• I can contribute

[taylortom]

Sounds like we need a new API/endpoint for retrieving hierarchies.
Can you summarise your proposed solution?

[chris-steele]

I'm on it @taylortom - will post back on here with an update shortly

[chris-steele]

Proposed change

The user sees one of three different views when editing:

• course (pages/menus)
• page
• element (a single component, block, article or content object)

Only load the data necessary for the view:

• course requires only models of type contentobject
• page requires a single contentobject and its descendants
• element requires only one model

Proposed method

To support this for each model record:

• ordered array of ancestors (using friendly ID format)
• time the model or its subtree changed

This is complimentary to existing data:

• keep existing use of _parentId
• use subtreeUpdatedAt and leave updatedAt unchanged

As at present cache content on the client:

• when the client revisits a view pass the timestamp of the [course/page/element] to the server
• server has only one model to read
• only return content if subtreeUpdatedAt is newer
• periodically run a purge of items from content cache (remove e.g. if not accessed after time [t])

Reasons

• currently if one model changes then the whole course is reloaded by the client
• provide faster navigation and reduced server load
• fix the issue with deletion not being detected

Potential for future

• parent is first item in array of ancestors so _parentId could be retired
• provides an efficient way to notify a user of concurrent edits:
  • client asks server to check timestamp of one model (page or element)
  • client can do this periodically and provide indication if content has been changed by another user