Multi-threaded highlighting #64
Labels
Comments
jbaiter
added a commit
that referenced
this issue
Feb 20, 2020
This is a relatively hacky way to implement the issues raised in #64. This PR adds a utility class to concurrently "warm" the OS page cache with files that will be used for highlighting. This should significantly reduce the I/O latency during the sequential highlighting process, especially when using a network storage layer or a RAID system. The idea is that a lot of storage layers can benefit from parallel I/O. Unfortunately, snippet generation with the current UHighlighter approach is strongly sequential, which means we give away a lot of potential performance, since we're limited by the I/O latency of the underlying storage layer. By pre-reading the data we might need in a concurrent way, we pre-populate the operating system's page cache, so any I/O performed by the snippet generation process further down the line should only hit the page cache and not incur as much of a latency hit. The class also provides a way to cancel the pre-loading of a given source pointer. This is called at the beginning of the snippet generation process, since at that point any background I/O on the target files will only add to the latency we might experience anyway.
jbaiter
added a commit
that referenced
this issue
Feb 20, 2020
This is a relatively hacky way to implement the issues raised in #64. This PR adds a utility class to concurrently "warm" the OS page cache with files that will be used for highlighting. This should significantly reduce the I/O latency during the sequential highlighting process, especially when using a network storage layer or a RAID system. The idea is that a lot of storage layers can benefit from parallel I/O. Unfortunately, snippet generation with the current UHighlighter approach is strongly sequential, which means we give away a lot of potential performance, since we're limited by the I/O latency of the underlying storage layer. By pre-reading the data we might need in a concurrent way, we pre-populate the operating system's page cache, so any I/O performed by the snippet generation process further down the line should only hit the page cache and not incur as much of a latency hit. The class also provides a way to cancel the pre-loading of a given source pointer. This is called at the beginning of the snippet generation process, since at that point any background I/O on the target files will only add to the latency we might experience anyway.
bitzl
pushed a commit
that referenced
this issue
Feb 20, 2020
This is a relatively hacky way to implement the issues raised in #64. This PR adds a utility class to concurrently "warm" the OS page cache with files that will be used for highlighting. This should significantly reduce the I/O latency during the sequential highlighting process, especially when using a network storage layer or a RAID system. The idea is that a lot of storage layers can benefit from parallel I/O. Unfortunately, snippet generation with the current UHighlighter approach is strongly sequential, which means we give away a lot of potential performance, since we're limited by the I/O latency of the underlying storage layer. By pre-reading the data we might need in a concurrent way, we pre-populate the operating system's page cache, so any I/O performed by the snippet generation process further down the line should only hit the page cache and not incur as much of a latency hit. The class also provides a way to cancel the pre-loading of a given source pointer. This is called at the beginning of the snippet generation process, since at that point any background I/O on the target files will only add to the latency we might experience anyway.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Currently every
(doc, field, matchOffset)combination is highlighted sequentially. Since highlighting is highly I/O-bound, it would be great if this could be parallelized at the doc- or field-level so we can take advantage of Storage-Layers that allow concurrent access (see e.g. #49).This work should probably also involve a refactor that moves away from subclassing the
uhighlight.FieldHighlightertype hierarchy and replaces it with something that is better suited to our use case. Specifically we should look at determining if there's a better way to determine passage boundaries than the currentBreakIteratorapproach.The text was updated successfully, but these errors were encountered: