This repository has been archived by the owner on Aug 9, 2024. It is now read-only.
perf: move heavy queries to prepared statements #78
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Hey! Thanks for the contribution - we will look over it soon. Are the nix files required? If not, could you remove them from the PR? |
|
Yeah I can remove those, the nix files are for a dev environment. I think I also may have left a stray console.log somewhere, will clean that up too. |
|
hey! any updates? |
|
Currently away from my computer, but you all should have push perms on this PR so feel free to checkout, delete nix files, and push. The GitHub Mobile app doesn't let me delete files, and I don't have my signing key set up on phone. |
|
I'll check this tomorrow |
|
Hey, sorry for not checking it out yesterday. This pull request has some conflicts on |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I noticed that many of the feed queries end up being huge SQL statements -- nearly 4,600 characters for the main feed query. Every time someone visited the site, hit the feed API, etc, this query would be re-sent to the postgres server and the server would have to re-parse the query, re-plan and re-analyze the query, re-build the query tree, and execute.
From the postgres docs:
and I'd say that definitely applies to the feed queries which have multiple subqueries, joins, etc etc.
This PR addresses this by moving all feed queries and a few other longish queries into prepared statements. I did some very preliminary testing with this, and it all seemed to work fine, but I'd recommend running some more extensive testing before pushing to prod.