Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rework how materialized views are updated #3

Open
natebrennand opened this issue Nov 9, 2014 · 2 comments
Open

rework how materialized views are updated #3

natebrennand opened this issue Nov 9, 2014 · 2 comments

Comments

@natebrennand
Copy link
Member

just noticed that Postgres actually lacks incremental updating of materialized views. this means that the materialized view tables are locked for the ~50 seconds that it takes to update them. this isn't really acceptable since it would just cause our queries to them to timeout.


proposed resolution:

  1. remake materialized views w/ different names then rename them
    • would be done in a PG transaction (means every step works or no steps work)
    • would reduce the update time
    • still takes ~50 seconds
  2. how many indexes can we add?
    • experiment and see if we can index enough things to make the materialized view creation faster
  3. try a different database?
    • MySQL doesn't seem to have this from my brief googling (possibly with this plugin)
    • Oracle actually does have them (but there's licensing fees I think?)

CC @adicu/density @rbtying

@natebrennand
Copy link
Member Author

to be considered:

views could be built from each other

hourly <- whole table
day <- hourly
week <- day
month <- week (probably won't be split properly, use day instead)?

This wouldn't fix the whole issue, but should make it significantly faster.

@natebrennand
Copy link
Member Author

This still seems like a poor man's incremental update but it would accomplish what we need.

https://wiki.postgresql.org/wiki/What's_new_in_PostgreSQL_9.4#REFRESH_MATERIALIZED_VIEW_CONCURRENTLY

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant