Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provenance runtime/memory optimisations #121

Open
alexdunnjpl opened this issue Apr 15, 2024 · 0 comments
Open

Provenance runtime/memory optimisations #121

alexdunnjpl opened this issue Apr 15, 2024 · 0 comments
Labels
B15.0 i&t.skip Skip I&T of this task/ticket task

Comments

@alexdunnjpl
Copy link
Contributor

alexdunnjpl commented Apr 15, 2024

Per #108 (comment)

It should be feasible to block provenance nonaggregate in such a way that a balance between redundant work and unworkable query volumes is achieved. Candidates:

  • given that no renamed-product support currently exists, utilize skips such that non-up-to-date products' LIDs are enumerated, then nonaggregates are skipped if they have a LID not in that enumeration.
  • one of the core ideals is to query only products having LIDs which are in the set of updated LIDs. An OpenSearch query pattern of "if lid in el1, el2, ... eln" is equivalent in performance to making n individual queries, so is infeasible, given the large quantity of nonaggregate LIDs in existence. But by enumerating the collections containing the affected non-aggregate products and then blocking non-aggregate queries by membership identity, it may be possible to perform efficiently. WIP thought.
@alexdunnjpl alexdunnjpl added B15.0 i&t.skip Skip I&T of this task/ticket task labels Apr 15, 2024
@github-project-automation github-project-automation bot moved this to Release Backlog in B15.0 Apr 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
B15.0 i&t.skip Skip I&T of this task/ticket task
Projects
No open projects
Status: Release Backlog
Development

No branches or pull requests

1 participant