Overhaul TokenStream ICloseable/IDisposable Patterns, #271 #1058
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overhaul TokenStream ICloseable/IDisposable Patterns
Fixes #271
Description
This pull request focuses on refactoring the codebase by replacing the
Dispose
method with the newICloseable
interface'sClose
method in all provided TokenStream implementations to make them reusable. None of the implementations in our codebase need a non-reusable cleanup method, however, in discussion in #271, it was determined that there are possible use cases in which a custom TokenStream implementation might need to have their instance Disposed when the owning Analyzer (with stored values set by a ReuseStrategy) is disposed, so we have retained the IDisposable pattern as well. This PR also ensures that those values are disposed by way of DisposableThreadLocal and TokenStreamComponents disposing of their values.Remaining draft PR TODOs:
PrefixTreeStrategy.CellTokenStream
reusableMemoryIndex.TokenStreamAnonymousClass<T>
reusable