Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only reindex documents if declarations are being modified #2942

Conversation

vinistock
Copy link
Member

@vinistock vinistock commented Nov 29, 2024

Motivation

This PR tries to limit the amount of times we pay the price of reindexing the current document being modified by analyzing the context behind the last edit made to it.

Implementation

The idea of this approach is to remember the position of the last edit made to a document and the nature of the edit (insert, replace or delete).

Then, depending on the type of edit and what nodes exist under the position where the edit was made, we decide if reindexing is relevant or not.

To determine that, we look at the node under the cursor and, if it matches a node we use for indexing in the DeclarationListener, then we consider that we need to reindex.

It's not perfect, so let me know if you have other ideas on how to determine this in a way that is:

  1. Fast
  2. Accurate

Necessarily in this order because the whole point is to reduce the performance impact of having to reindex. If detecting whether we need to do it or not is equally as slow, then there's no point in doing it.

Automated Tests

Added some tests, but we should probably add more.

Copy link
Member Author

vinistock commented Nov 29, 2024

This stack of pull requests is managed by Graphite. Learn more about stacking.

@vinistock vinistock added enhancement New feature or request server This pull request should be included in the server gem's release notes labels Nov 29, 2024 — with Graphite App
@vinistock vinistock marked this pull request as ready for review November 29, 2024 21:41
@vinistock vinistock requested a review from a team as a code owner November 29, 2024 21:41
@vinistock vinistock force-pushed the 11-29-index_documents_on_modification branch from 41cc250 to 826bd1a Compare November 29, 2024 21:51
@vinistock vinistock force-pushed the 11-29-only_reindex_documents_if_declarations_are_being_modified branch 2 times, most recently from c103d7c to ea92bc7 Compare November 29, 2024 21:59
@vinistock vinistock force-pushed the 11-29-index_documents_on_modification branch 2 times, most recently from 2251c9d to e5b8c20 Compare December 10, 2024 19:55
@vinistock vinistock force-pushed the 11-29-only_reindex_documents_if_declarations_are_being_modified branch from ea92bc7 to f299a65 Compare December 10, 2024 19:55
@vinistock vinistock requested a review from andyw8 December 10, 2024 19:57
@vinistock vinistock force-pushed the 11-29-index_documents_on_modification branch from e5b8c20 to 41fc7d4 Compare January 2, 2025 19:48
@vinistock vinistock force-pushed the 11-29-only_reindex_documents_if_declarations_are_being_modified branch from f299a65 to 6fa7091 Compare January 2, 2025 19:48
@vinistock vinistock force-pushed the 11-29-index_documents_on_modification branch from 41fc7d4 to d03faf8 Compare January 7, 2025 14:29
@vinistock vinistock force-pushed the 11-29-only_reindex_documents_if_declarations_are_being_modified branch from 6fa7091 to bfdeee1 Compare January 7, 2025 14:29
@vinistock vinistock changed the base branch from 11-29-index_documents_on_modification to graphite-base/2942 January 7, 2025 14:54
@vinistock vinistock force-pushed the 11-29-only_reindex_documents_if_declarations_are_being_modified branch from bfdeee1 to 9814d66 Compare January 7, 2025 14:55
@vinistock vinistock changed the base branch from graphite-base/2942 to main January 7, 2025 14:55
@vinistock vinistock force-pushed the 11-29-only_reindex_documents_if_declarations_are_being_modified branch from 9814d66 to 5f29c15 Compare January 7, 2025 14:55
@andyw8
Copy link
Contributor

andyw8 commented Jan 8, 2025

Just thinking out loud, but might a possible alternate approach be:

  • grep the file for lines matching any of the METHODS_THAT_CHANGE_DECLARATIONS to create a checksum
  • when saving recalculate the checksum
  • if the checksum changes, re-index

Since it's not dealing with the AST it would be faster, but it may result in more unnecessary re-indexing. Also it wouldn't correctly handle multi-line declarations.

@vinistock
Copy link
Member Author

grep the file for lines matching any of the METHODS_THAT_CHANGE_DECLARATIONS to create a checksum

The issue is that this would be quite error prone. We'd be essentially trying to parse a subset of Ruby with regexes. Imagine cases like these

do_something(class: "")

my_object.end

{ def: 123 }

include = 123

And most of the bugs would be silent because the LSP would simply just not reindex when it's suppose to, which would make them hard to detect.

Copy link
Contributor

@andyw8 andyw8 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the early iterations, it might be useful to have some logging to indicate when indexing is being skipped.

test/ruby_document_test.rb Outdated Show resolved Hide resolved
@vinistock vinistock force-pushed the 11-29-only_reindex_documents_if_declarations_are_being_modified branch from 5f29c15 to 34510e6 Compare January 8, 2025 18:32
@vinistock
Copy link
Member Author

For the early iterations, it might be useful to have some logging to indicate when indexing is being skipped.

Good idea, but I did the opposite. Showing when we detect that a reindex is needed instead.

@vinistock vinistock force-pushed the 11-29-only_reindex_documents_if_declarations_are_being_modified branch from 34510e6 to 5cb18b5 Compare January 8, 2025 18:34
@vinistock vinistock merged commit f8f3548 into main Jan 8, 2025
43 checks passed
Copy link
Member Author

Merge activity

  • Jan 8, 2:00 PM EST: A user merged this pull request with Graphite.

@vinistock vinistock deleted the 11-29-only_reindex_documents_if_declarations_are_being_modified branch January 8, 2025 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request server This pull request should be included in the server gem's release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants