Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[das/#106] Change patterns and templates indexes to store handles only rather than handles+targets #219

Merged
merged 5 commits into from
Sep 24, 2024

Conversation

andre-senna
Copy link
Contributor

@andre-senna andre-senna commented Sep 23, 2024

We use 5 indexes in AtomDB. Names, incoming set, outgoing set, type templates and patterns.

All of them are hash tables mapping from handles to values. Values are different deppending on the type of the index. The former 3 indexes map handle->List[Handle] (list of handles) while the later 2 map handle->List[Tuple[Handle, [Tuple[Handle]]]] (list of tuples (handle, targets) with the handle and the targets pod the mapped links.

Mapped link targets are stored in the last two indexes for the sake of performance in the pattern matcher query algorithm.

In the scope of the integration of the cache in the query engines, this optimization is no longer required, thus we want to simplify the API and store handle->List[Handle] for ALL indexes. This is what this PR is about.

This PR changes the AtomDB to consider a mapping of Handle->List[Handle] in the templates and in the pattern indexes.

This change is coupled with changes in the meta parser and the query engines in order to keep compliance.

SIDE EFFECT CHANGE: as a side effect of these changes, ORDERED_LINKS references (as well as is_ordered() calls) have been removed.

@andre-senna andre-senna self-assigned this Sep 23, 2024
angeloprobst
angeloprobst previously approved these changes Sep 23, 2024
@angeloprobst angeloprobst dismissed their stale review September 23, 2024 19:08

found some issues

Copy link
Collaborator

@angeloprobst angeloprobst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, although you forgot to update these type annotations in the Database dataclass:

image

@angeloprobst
Copy link
Collaborator

Also, it seems that outgoing_set could be annotated as dict[str, set[str]].
Currently it is:

    outgoing_set: dict[str, Any] = dc_field(default_factory=dict)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants