Skip to content

Commit

Permalink
[Doc] Reorganize online pooling APIs (vllm-project#11172)
Browse files Browse the repository at this point in the history
Signed-off-by: DarkLight1337 <[email protected]>
  • Loading branch information
DarkLight1337 authored Dec 13, 2024
1 parent 238c0d9 commit 0920ab9
Show file tree
Hide file tree
Showing 10 changed files with 368 additions and 288 deletions.
14 changes: 7 additions & 7 deletions docs/source/models/pooling_models.rst
Original file line number Diff line number Diff line change
Expand Up @@ -50,10 +50,10 @@ It returns the extracted hidden states directly, which is useful for reward mode
.. code-block:: python
llm = LLM(model="Qwen/Qwen2.5-Math-RM-72B", task="reward")
output, = llm.encode("Hello, my name is")
(output,) = llm.encode("Hello, my name is")
data = output.outputs.data
print(f"Prompt: {prompt!r} | Data: {data!r}")
print(f"Data: {data!r}")
``LLM.embed``
^^^^^^^^^^^^^
Expand All @@ -64,7 +64,7 @@ It is primarily designed for embedding models.
.. code-block:: python
llm = LLM(model="intfloat/e5-mistral-7b-instruct", task="embed")
output, = llm.embed("Hello, my name is")
(output,) = llm.embed("Hello, my name is")
embeds = output.outputs.embedding
print(f"Embeddings: {embeds!r} (size={len(embeds)})")
Expand All @@ -80,7 +80,7 @@ It is primarily designed for classification models.
.. code-block:: python
llm = LLM(model="jason9693/Qwen2.5-1.5B-apeach", task="classify")
output, = llm.classify("Hello, my name is")
(output,) = llm.classify("Hello, my name is")
probs = output.outputs.probs
print(f"Class Probabilities: {probs!r} (size={len(probs)})")
Expand All @@ -102,8 +102,8 @@ These types of models serve as rerankers between candidate query-document pairs
.. code-block:: python
llm = LLM(model="BAAI/bge-reranker-v2-m3", task="score")
output, = llm.score("What is the capital of France?",
"The capital of Brazil is Brasilia.")
(output,) = llm.score("What is the capital of France?",
"The capital of Brazil is Brasilia.")
score = output.outputs.score
print(f"Score: {score}")
Expand All @@ -119,7 +119,7 @@ Please click on the above link for more details on how to launch the server.
Embeddings API
^^^^^^^^^^^^^^

Our Embeddings API is similar to ``LLM.encode``, accepting both text and :ref:`multi-modal inputs <multimodal_inputs>`.
Our Embeddings API is similar to ``LLM.embed``, accepting both text and :ref:`multi-modal inputs <multimodal_inputs>`.

The text-only API is compatible with `OpenAI Embeddings API <https://platform.openai.com/docs/api-reference/embeddings>`__
so that you can use OpenAI client to interact with it.
Expand Down
Loading

0 comments on commit 0920ab9

Please sign in to comment.