Skip to content

Commit

Permalink
Update docs about rasa data validate flows checks (#12916)
Browse files Browse the repository at this point in the history
* update docs about rasa data validate flows

* address review comments

* typo ci update, add more context
  • Loading branch information
ancalita authored Oct 12, 2023
1 parent 818bfad commit 2571266
Show file tree
Hide file tree
Showing 2 changed files with 23 additions and 2 deletions.
1 change: 1 addition & 0 deletions .typo-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -217,5 +217,6 @@ excluded_words:
- HashiCorp
- sqltrackerstore
- wsgi
- pypred

spellcheck_filenames: false
24 changes: 22 additions & 2 deletions docs/docs/command-line-interface.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ The following commands are relevant to all assistants built with Rasa.
| `rasa run actions` | Starts an action server using the Rasa SDK. |
| `rasa test e2e` | Runs end-to-end testing fully integrated with the action server that serves as acceptance testing. |
| `rasa data split nlu` | Performs a 80/20 split of your NLU training data. |
| `rasa data validate` | Checks the domain, NLU and conversation data for inconsistencies. |
| `rasa data validate` | Checks the domain, NLU, flows and conversation data for inconsistencies. |
| `rasa export` | Exports conversations from a tracker store to an event broker. |
| `rasa evaluate markers` | Extracts markers from an existing tracker store. |
| `rasa marker upload` | Upload marker configurations to Analytics Data Pipeline |
Expand Down Expand Up @@ -562,7 +562,7 @@ rasa data migrate --help

## rasa data validate

You can check your domain, NLU data, or story data for mistakes and inconsistencies.
You can check your domain, NLU data, flows or story data for mistakes and inconsistencies.
To validate your data, run this command:

```bash
Expand All @@ -574,6 +574,7 @@ identical training examples.
The validator also checks if you have any stories where different assistant actions follow from the same
dialogue history. Conflicts between stories will prevent a model from learning the correct
pattern for a dialogue.
To learn more about the checks performed by the validator on flows, continue reading in the [next section](./command-line-interface.mdx#validate-flows).

:::info Searching for the `assistant_id` key introduced in 3.5

Expand All @@ -585,6 +586,25 @@ key is missing or if the default value has not been changed.
If you pass a `max_history` value to one or more policies in your `config.yml` file, provide the
smallest of those values in the validator command using the `--max-history <max_history>` flag.

### Validate flows

The validator will perform the following checks on flows:
- determine whether flow names or descriptions are unique after stripping punctuation
- verify whether natural language conditions used in [branch steps](./concepts/flows.mdx#branching-conditions) or `collect` step [rejections](./concepts/flows.mdx#slot-validation) are valid `pypred` expressions
- determine whether slots used in flows are defined in the domain
- disallow `list` slots from being used in flows `collect` steps: CALM supports only filling slots with values of type `int`, `string` or `bool` in flows.
- disallow `dialogue_stack` internal slot from being used in flows

For every failure, the validator will log an error and exit the command with exit code 1.

You can validate flows only by running this command:

```bash
rasa data validate flows
```

### Validate story structure

You can also validate only the story structure by running this command:

```bash
Expand Down

0 comments on commit 2571266

Please sign in to comment.