Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for bulk addition of supplemental attributes #382

Open
daniel-thom opened this issue Jun 24, 2024 · 1 comment
Open

Support for bulk addition of supplemental attributes #382

daniel-thom opened this issue Jun 24, 2024 · 1 comment
Labels
performance issues that impact computational performance

Comments

@daniel-thom
Copy link
Contributor

We currently only support adding supplemental attributes one at a time. This is likely to be slow if there are thousands of attributes because each addition causes one row to be added to an SQLite table. We support bulk addition of time series for the same reason and could do the same for supplemental attributes.

@jd-lara jd-lara added the performance issues that impact computational performance label Jun 24, 2024
@daniel-thom
Copy link
Contributor Author

I made a prototype. Performance is ~3x faster with bulk additions. My benchmark adds 500k supplemental attributes to components.

  • add 1 at a time: 24.378 seconds
  • add 1000 at a time: 7.7 seconds

The code is straightforward and I'm not worried about that. There is a cost in user complexity for the bulk additions. The overall time involved here is not huge. Is it sufficient to justify the complexity (if users notice the docstrings)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance issues that impact computational performance
Projects
None yet
Development

No branches or pull requests

2 participants