Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revisit the code that populates the search results table #117

Open
danielballan opened this issue Mar 16, 2021 · 1 comment
Open

Revisit the code that populates the search results table #117

danielballan opened this issue Mar 16, 2021 · 1 comment

Comments

@danielballan
Copy link
Member

It works well sometimes, but other times it is comically slow, and it's hard to detect a pattern. I suggest we start by refactoring the worker to make it update a row at a time rather than a cell at a time and see how far that gets us.

def _load_data(get_data, indexes):
"Load a batch of data. This is run in a threadpool."
for index in indexes:
row, column = index.row(), index.column()
try:
item = get_data(row, column)
except Exception:
logger.exception("Error while loading search results")
continue
yield index, item

@cjtitus
Copy link
Contributor

cjtitus commented Jan 30, 2024

Part of the explanation for the slowness of cell-based updates is probably bluesky/tiled#645. Row-based updates will be faster, but still subject to really inefficient paging until that is fixed.

Once the issue is fixed, row-based updates will be ok, but still subject to latency on a per-row basis. I have code that does efficient chunk-based updates, which work well, though they will also get faster once bluesky/tiled#645 is fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants