intent-recommendations.md

copyright	lastupdated	subcollection
years 2018, 2021	2021-03-01	assistant

{:shortdesc: .shortdesc} {:new_window: target="_blank"} {:external: target="_blank" .external} {:deprecated: .deprecated} {:important: .important} {:note: .note} {:tip: .tip} {:pre: .pre} {:codeblock: .codeblock} {:screen: .screen} {:javascript: .ph data-hd-programlang='javascript'} {:java: .ph data-hd-programlang='java'} {:python: .ph data-hd-programlang='python'} {:swift: .ph data-hd-programlang='swift'} {:video: .video}

Getting help with intents

{: #intent-recommendations}

Have Watson analyze your chat transcript log data to find the most common and frequently expressed customer needs related to your business. Watson can then recommend intents and intent user examples that you can use to train your assistant so it can recognize the same and similar requests in the future. {: shortdesc}

This feature is available to users of paid plans only. {: note}

Watson can recommend some initial intents for you to start with. If you already created some intents, Watson can analyze your logs and compare its findings with your existing intents. Based on what it learns, Watson can identify gaps in your training data and suggest new intents to fill them.

Get help getting started

{: #intent-recommendations-get-started}

Customer needs are represented in {{site.data.keyword.conversationshort}} as intents. If you have no intents, you can get started faster by getting Watson to help you. Upload a file that contains the utterances that customers use to articulate requests when they interact with your customer support personnel. You might be able to extract these utterances from call center transcripts, for example. Upload the utterances as a CSV file to {{site.data.keyword.conversationshort}} so that Watson can analyze the data. Based on the insights it uncovers, Watson recommends a base set of intents that you can build to cover the most common needs of your customers.

The following video provides a 3-minute overview of intent and intent user example recommendations.

{: video output="iframe" id="youtubeplayer" frameborder="0" width="560" height="315" webkitallowfullscreen mozallowfullscreen allowfullscreen}

To read a transcript of the video, open the video on YouTube.com, click the More actions icon, and then choose Open transcript.

To learn more about how intent recommendations can help you build a useful bot faster, read this blog post{: external}.

Keep your training data current

{: #intent-recommendations-stay-current}

As the subjects that your customers want to discuss change, you can use the intent and intent user example recommendations features to help keep your intents relevant over time.

Not only can Watson analyze uploaded support site chat log transcripts, but it can process the logs that are generated by a deployed assistant as it interacts with your customers.

Mine your existing data to do one of the following things:

• Get intent recommendations
• Get intent user example recommendations

For more information about the language support for this feature, see Supported languages.

Choosing an intent recommendations data source

{: #intent-recommendations-data-resources}

Intent recommendations are based on analysis of real-world user utterances, which you can provide in uploaded CSV files or by connecting to the log of a deployed assistant. The first time that you use intent recommendations, you must indicate the source from which you want Watson to derive its recommendations.

1. Open your dialog skill.

2. From the skill menu, click Intents.

3. Click Intent recommendations.

4. Choose the type of recommendation data source you want to use.

   • Live assistant log: The log from an assistant that is deployed and is actively interacting with customers
   • CSV log files: A list of utterances that is extracted from support center chat transcripts.

   You must choose a recommendation source type; you cannot use both.

The recommendation source data that you add is used to derive both intent and intent user example recommendations.

Getting recommendations from an assistant log

{: #intent-recommendations-live-assistant-add}

Use user conversations that took place between your assistant and your customers as the source for intent recommendations.

To use assistant chat logs as the source for your intent and intent user example recommendations, the assistant must meet these requirements:

• The assistant must be live. Meaning it is deployed in a production environment, and is interacting with customers.

• The application with which you deploy the assistant must use the v2 /message API. The log of an assistant that receives user input through the v1 /message API cannot be used as a recommendation source.

  All of the built-in integrations use the v2 API. See Adding integrations. {: tip}

• The assistant must have exchanged messages with a customer within the log retention time period. Logs are stored for a number of days, the total of which is determined by your service plan type. If no customer interactions take place within the retention period, then there is no log data available to use as a recommendation source. For more information about log limits, see Log limits.

• The assistant log must contain at least 3,000 messages.

{{site.data.keyword.Bluemix_dedicated_notm}}: You cannot get intent recommendations from logs. You can see the option to link to a connected log, but the data source always indicates that there are 0 logs available. To use an assistant log as the source for your intent recommendations, switch to an Enterprise plan. Alternatively, you can add the user utterances from the logs to a CSV file, upload it, and choose the logs CSV file as the data source.

Getting recommendations from a CSV log file

{: #intent-recommendations-log-files-add}

To share your support center chat log transcript data with Watson, you must upload the data as a CSV file in the correct format. Create a comma-separated value (CSV) file with one sentence that represents a customer utterance per line. Ideally, the utterances are short phrases that are extracted from your call center transcripts that contain real-world customer questions and requests.

Each user example source file can be a maximum size of 20 MB.

Follow these additional guidelines:

• Remove any sensitive data from the utterances that you include in the file.

  Sensitive data includes any information about an identifiable natural person such as names, email addresses, and customer IDs, and regulated data such as protected health information. {: important}

• Utterances that are shorter than 3 words or longer than 20 words are automatically filtered out during import.

  Short and long utterances do not make good intent user examples. If your log transcript data has many utterances that are longer than 20 words, consider first splitting the long utterances into multiple sentences, and then add one sentence per line in the CSV file.

• The file must contain at least 100 utterances. A file with 500 or more utterances will give you better results. Provide as much data as you can within the 20 MB file limit.

• Do not submit duplicate utterances. Duplicating data within the CSV does not improve results.

• If an utterance contains a comma, surround the utterance in quotation marks.

• The CSV must include only one column.

• Remove everything that is not a customer-generated utterance, including any human agent responses or notes.

Here's some sample CSV file content:

What happens to my coverage if I trade in my car?
i'd like to buy a house.
How do I add a dependent to my plan?
"first, i want to know if i am already registered."

Any files you upload are shared with the other users who have access to the skill. The utterances from all the files that were uploaded to this skill are mined when you ask for both intent recommendations and intent user example recommendations.

Getting intent recommendations

{: #intent-recommendations-get-intent-recommendations-task}

Before you begin, you must choose an intent recommendations data source.

After the data is provided, Watson evaluates the user utterances and identifies common problem areas that customers mention frequently. {{site.data.keyword.conversationshort}} then displays a set of discrete groups of related candidate user examples that capture the trending customer needs. The groups are so granular in scope that you might choose to add separately-grouped candidate user examples to the same intent. You can review each recommended intent and the corresponding user examples to choose the ones you want to add to your training data.

To get intent recommendations, complete the following steps:

1. Open your dialog skill.

2. From the skill menu, click Intents.

3. Click Intent recommendations.

   Give Watson time to analyze your data and group the utterances.

4. Review the groups of candidate intents that are recommended by Watson.

   If your recommendation source has over 500,000 user utterances, then Watson samples the data and derives recommendations from the sampled subset.

   The top intent recommendations are selected and ranked as follows:

   • Utterances are grouped together based on their similarity. Candidate groups with a higher number of cohesive utterances are ranked higher.
   • Candidate groups with utterances that are distinct from existing intent user examples are ranked higher. Groups that are similar to existing intents in the skill are ranked lower. Uniqueness suggests that new training data can address customer needs that are not being met currently.

5. Click a candidate intent to see its associated user examples.

   Watson chooses a maximum of 20 user examples for each candidate intent it recommends. It deduplicates utterances from the source, and limits the overall number of utterances to a set of top user examples so it's easier for you to understand and work with them. It chooses the top user examples by identifying the most representative utterances. It then selects utterances that are distinct from the top utterances to add variety. The list is ordered based on the length of the user examples, with the shorter examples listed higher.

6. Select any utterances that you want to add to your training data. You can click the User utterance checkbox to select all of the utterances.

   Do one of the following things next:

   • Change the intent name.

     Watson creates a name for each candidate intent based on common terms in the corresponding utterances. You can change the intent name before you add it. A name that begins with an underscore (_), such as _not_able_to_login, signifies that the name is not descriptive of the grouped utterances as a whole.

   • To add the recommended intent with the selected utterances as user examples, click Create new intent.

   • To add the selected utterances from the recommended intent to one of your existing intents as user examples instead, click Add to existing intent, choose the intent, and then click Add.

The intents and intent user examples that you add in this way do count toward your intent and intent user example totals for which there are limits per plan. See Intent limits for more details.

As the subjects that your customers want to discuss change, you can use the intent user example recommendations feature to help keep your intents up-to-date and relevant over time.

Getting intent user example recommendations

{: #intent-recommendations-get-example-recommendations-task}

For any intent that is already part of your training data, you can find and add new user examples by asking Watson for intent user example recommendations.

Before you begin, you must choose an intent recommendations data source. Even though you request user example recommendations on a per-intent basis, the same data source is used to find user examples that are appropriate for every intent. Watson knows which intent you are working on and finds suitable examples to recommend for that specific intent.

1. Follow the steps in Creating intents to create an intent.

2. Add at least 5 user examples that illustrate the full range of typical utterances that you anticipate customers might say to trigger this intent.

   These seed user examples teach Watson about the kinds of utterances to look for in the recommendation source data you provided.

3. Click Show recommendations.

   Recommended user examples are displayed in the side pane.

   If no recommendations are made, then the connected recommendation source does not contain examples that are suitable for this intent.

4. If the current recommendation source cannot provide useful recommendations for any of your intents, you can try a different set of utterances by changing the recommendation source that you're using. See Managing recommendation sources for more details.

5. After Watson shows you recommendations, select the utterances that you want to add as user examples for this intent, and then click Add. Or click Next set to review more utterances.

6. If you want to search the content of the recommendation source for user examples yourself, click the Search Logs tab. Enter a keyword on which to base the search, and then press Enter.

   Follow these search query syntax guidelines:

   • Boolean operators (such as AND and OR) are supported.

   • Add quoted text to search for an exact text match ("thisstringmustbepresent").

   • You can use regular expressions, such as *ly to find all terms that end with ly.

   • The following characters are used as regular expression operators:

     + - = && || > < ! ( ) { } [ ] ^ " ~ * ? : \ /

     If you want to include one in a search term without it being processed as an operator, you must prefix it with a backslash (\).

The user examples that you add in this way do count toward your intent user example totals for which there are limits per plan. See Intent limits for more details.

Managing recommendation sources

{: #intent-recommendations-manage-sources}

You can add or delete CSV files or change the assistant from which you mine conversation logs.

1. Open your dialog skill.

2. From the skill menu, click Intents.

3. Click Recommendation sources.