Elastic Rerank model landing page #2884
Conversation
leemthompo
added
backport-8.17
|
A documentation preview will be available soon. Request a new doc build by commenting
If your PR continues to fail for an unknown reason, the doc build pipeline may be broken. Elastic employees can check the pipeline status here. |
leemthompo
changed the title
Co-authored-by: István Zoltán Szabó <[email protected]>
|
|
||
| It's important to note that if you rerank to depth `n` then you will need to run `n` inferences per query. This will include the document text and will therefore be significantly more expensive than inference for query embeddings. Hardware can be scaled to run these inferences in parallel, but we would recommend shallow reranking for CPU inference: no more than top-30 results. You may find that the preview version is cost prohibitive for high query rates and low query latency requirements. We plan to address performance issues for GA. | ||
|
|
||
| // // Is air-gapped deployment supported? |
There was a problem hiding this comment.
Yes air gapped download is supported, the same instructions as ELSER apply just change the model id
https://www.elastic.co/guide/en/machine-learning/master/ml-nlp-elser.html#air-gapped-install
There was a problem hiding this comment.
ah ok cool thanks
There was a problem hiding this comment.
@davidkyle are those model artifact files already available?
found 'em :)
There was a problem hiding this comment.
Looks like we just have cross-platform version initially?
There was a problem hiding this comment.
Yes, there is no platform specific model for rerank
Co-authored-by: David Kyle <[email protected]>
|
@davidkyle I updated with air-gapped instructions, copying the ELSER instructions but removing the trained model UI UI instructions and just replacing with "create inference endpoint". |
| + | ||
| -- | ||
| ``` | ||
| xpack.ml.model_repository: file://${path.home}/config/models/` |
There was a problem hiding this comment.
theres an extra backtick at the end of this line
There was a problem hiding this comment.
nice catch, fixing!
| + | ||
| When using the {ref}/semantic-text.html[`semantic_text` field type], text is divided into chunks. By default, each chunk contains 250 words (approximately 400 tokens). Be cautious when increasing the chunk size - if the combined length of your query and chunk text exceeds 512 tokens, the model won't have access to the full content. | ||
| + | ||
| When the combined inputs exceed the 512 token limit, a balanced truncation strategy is used. If both the query and input text are longer than 255 tokens each then both are truncated, otherwise the longest is truncated. |
There was a problem hiding this comment.
This is very clear phrasing!
There was a problem hiding this comment.
That's because it's 98% @tveasey and @davidkyle 😉
(cherry picked from commit 65aa83a)
(cherry picked from commit 65aa83a)
(cherry picked from commit 65aa83a) Co-authored-by: Liam Thompson <[email protected]>
(cherry picked from commit 65aa83a) Co-authored-by: Liam Thompson <[email protected]>
URL preview 🔭 👁️
TODO