add docs

docs/docs/llms/large-language-models.mdx

@@ -0,0 +1,69 @@
+---
+id: large-language-models
+sidebar_label: LLMs in Rasa
+title: Using LLMs with Rasa
+className: hide
+abstract:
+---
+
+import RasaProLabel from "@theme/RasaProLabel";
+import RasaLabsLabel from "@theme/RasaLabsLabel";
+import RasaLabsBanner from "@theme/RasaLabsBanner";
+
+<RasaProLabel />
+
+<RasaLabsLabel />
+
+<RasaLabsBanner version="3.7.0b1" />
+
+As part of a beta release, we have released multiple components 
+which make use of the latest generation of Large Language Models (LLMs).
+This document offers an overview of what you can do with them.
+We encourage you to experiment with these components and share your findings with us.
+We are working on some larger changes to the platform that leverage LLMs natively.
+Please reach out to us if you'd like to learn more about upcoming changes.
+
+
+## LLMs can do more than just NLU
+
+The recent advances in large language models (LLMs) have opened up new
+possibilities for conversational AI. LLMs are pretrained models that can be
+used to perform a variety of tasks, including intent classification,
+dialogue handling, and natural language generation (NLG). The components described
+here all use in-context learning. In other words, instructions and examples are
+provided in a prompt which are sent to a general-purpose LLM. They do not require
+fine-tuning of large models.
+
+### Plug & Play LLMs of your choice
+
+Just like our NLU pipeline, the LLM components here can be configured to use different
+LLMs. There is no one-size-fits-all best model, and new models are being released every
+week. We encourage you to try out different models and evaluate their performance on 
+different languages in terms of fluency, accuracy, and latency.
+
+### An adjustable risk profile
+
+The potential and risks of LLMs vary per use case. For customer-facing use cases, 
+you may not ever want to send generated text to your users. Rasa gives you full 
+control over where and when you want to make use of LLMs. You can use LLMs for NLU and
+dialogue, and still only send messages that were authored by a human. 
+You can also allow an LLM to rephrase your existing messages to account for context.
+
+It's essential that your system provides full
+control over these processes. Understanding how LLMs and other components
+behave and have the power to override any decision.
+
+## Where to go from here
+
+This section of the documentation guides you through the diverse ways you can
+integrate LLMs into Rasa. We will delve into the following topics:
+
+1. [Setting up LLMs](./llm-setup.mdx)
+2. [Intentless Policy](./llm-intentless.mdx)
+4. [LLM Intent Classification](./llm-intent.mdx)
+5. [Response Rephrasing](./llm-nlg.mdx)
+
+Each link will direct you to a detailed guide on the respective topic, offering
+further depth and information about using LLMs with Rasa. By the end of this
+series, you'll be equipped to effectively use LLMs to augment your Rasa
+applications.

docs/docs/llms/llm-custom.mdx

@@ -0,0 +1,235 @@
+---
+id: llm-custom
+sidebar_label: Customizing LLM Components
+title: Customizing LLM based Components
+abstract:
+---
+
+import RasaProLabel from "@theme/RasaProLabel";
+import RasaLabsLabel from "@theme/RasaLabsLabel";
+import RasaLabsBanner from "@theme/RasaLabsBanner";
+
+<RasaProLabel />
+
+<RasaLabsLabel />
+
+<RasaLabsBanner version="3.7.0b1" />
+
+The LLM components can be extended and modified with custom versions. This
+allows you to customize the behavior of the LLM components to your needs and
+experiment with different algorithms.
+
+## Customizing a component
+
+The LLM components are implemented as a set of classes that can be extended
+and modified. The following example shows how to extend the 
+`LLMIntentClassifier` component to add a custom behavior.
+
+For example, we can change the logic that selects the intent labels that are 
+included in the prompt to the LLM model. By default, we only include a selection
+of the available intents in the prompt. But we can also include all available
+intents in the prompt. This can be done by extending the `LLMIntentClassifier`
+class and overriding the `select_intent_examples` method:
+
+```python
+from rasa_plus.ml import LLMIntentClassifier
+
+class CustomLLMIntentClassifier(LLMIntentClassifier):
+    def select_intent_examples(
+        self, message: Message, few_shot_examples: List[Document]
+    ) -> List[str]:
+        """Selects the intent examples to use for the LLM training.
+
+        Args:
+            message: The message to classify.
+            few_shot_examples: The few shot examples to use for the LLM training.
+
+        Returns:
+            The list of intent examples to use for the LLM training.
+        """
+
+        # use all available intents for the LLM prompt
+        return list(self.available_intents)
+```
+
+The custom component can then be used in the Rasa configuration file:
+
+```yaml title="config.yml"
+pipeline:
+  - name: CustomLLMIntentClassifier
+    # ...
+```
+
+To reference a component in the Rasa configuration file, you need to use the
+full name of the component class. The full name of the component class is
+`<module>.<class>`.
+
+All components are well documented in their source code. The code can 
+be found in your local installation of the `rasa_plus` python package. 
+
+## Common functions to be overridden
+Below is a list of functions that could be overwritten to customize the LLM
+components:
+
+### LLMIntentClassifier
+
+#### select_intent_examples
+
+Selects the intent examples to use for the LLM prompt. The selected intent 
+labels are included in the generation prompt. By default, only the intent
+labels that are used in the few shot examples are included in the prompt.
+
+```python
+    def select_intent_examples(
+        self, message: Message, few_shot_examples: List[Document]
+    ) -> List[str]:
+        """Returns the intents that are used in the classification prompt.
+
+        The intents are included in the prompt to help the LLM to generate the
+        correct intent. The selected intents can be based on the message or on
+        the few shot examples which are also included in the prompt.
+
+        Including all intents can lead to a very long prompt which will lead
+        to higher costs and longer response times. In addition, the LLM might
+        not be able to generate the correct intent if there are too many intents
+        in the prompt as we can't include an example for every intent. The
+        classification would in this case just be based on the intent name.
+
+        Args:
+            message: The message to classify.
+            few_shot_examples: The few shot examples that can be used in the prompt.
+
+
+        Returns:
+        The intents that are used in the classification prompt.
+        """
+```
+
+#### closest_intent_from_training_data
+The LLM generates an intent label which 
+might not always be part of the domain. This function can be used to map the
+generated intent label to an intent label that is part of the domain.
+
+The default implementation embedds the generated intent label and all intent
+labels from the domain and returns the closest intent label from the domain.
+
+```python
+    def closest_intent_from_training_data(self, generated_intent: str) -> Optional[str]:
+        """Returns the closest intent from the training data.
+
+        Args:
+            generated_intent: the intent that was generated by the LLM
+
+        Returns:
+            the closest intent from the training data.
+        """
+```
+
+#### select_few_shot_examples
+
+Selects the NLU training examples that are included in the LLM prompt. The
+selected examples are included in the prompt to help the LLM to generate the
+correct intent. By default, the most similar training examples are selected. 
+The selection is based on the message that should be classified. The most
+similar examples are selected by embedding the incoming message, all training
+examples and doing a similarity search.
+
+```python
+    def select_few_shot_examples(self, message: Message) -> List[Document]:
+        """Selects the few shot examples that should be used for the LLM prompt.
+
+        The examples are included in the classification prompt to help the LLM
+        to generate the correct intent. Since only a few examples are included
+        in the prompt, we need to select the most relevant ones.
+
+        Args:
+            message: the message to find the closest examples for
+
+        Returns:
+            the closest examples from the embedded training data
+        """
+```
+
+### LLMResponseRephraser
+
+#### rephrase
+
+Rephrases the response generated by the LLM. The default implementation
+rephrases the response by prompting an LLM to generate a response based on the
+incoming message and the generated response. The generated response is then
+replaced with the generated response.
+
+```python
+    def rephrase(
+        self,
+        response: Dict[str, Any],
+        tracker: DialogueStateTracker,
+    ) -> Dict[str, Any]:
+        """Predicts a variation of the response.
+
+        Args:
+            response: The response to rephrase.
+            tracker: The tracker to use for the prediction.
+            model_name: The name of the model to use for the prediction.
+
+        Returns:
+            The response with the rephrased text.
+        """
+```
+
+### IntentlessPolicy
+
+#### select_response_examples
+
+Samples responses that fit the current conversation. The default implementation
+samples responses from the domain that fit the current conversation.
+The selection is based on the conversation history, the history will be 
+embedded and the most similar responses will be selected.
+
+```python
+    def select_response_examples(
+        self,
+        history: str,
+        number_of_samples: int,
+        max_number_of_tokens: int,
+    ) -> List[str]:
+        """Samples responses that fit the current conversation.
+
+        Args:
+            history: The conversation history.
+            policy_model: The policy model.
+            number_of_samples: The number of samples to return.
+            max_number_of_tokens: Maximum number of tokens for responses.
+
+        Returns:
+            The sampled conversation in order of score decrease.
+        """
+```
+
+#### select_few_shot_conversations
+
+Samples conversations from the training data. The default implementation
+samples conversations from the training data that fit the current conversation.
+The selection is based on the conversation history, the history will be
+embedded and the most similar conversations will be selected.
+
+```python
+    def select_few_shot_conversations(
+        self,
+        history: str,
+        number_of_samples: int,
+        max_number_of_tokens: int,
+    ) -> List[str]:
+        """Samples conversations from the given conversation samples.
+
+        Excludes conversations without AI replies
+
+        Args:
+            history: The conversation history.
+            number_of_samples: The number of samples to return.
+            max_number_of_tokens: Maximum number of tokens for conversations.
+
+        Returns:
+            The sampled conversation ordered by similarity decrease.
+        """
+```