-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Handle categorical slots in command generator - [ENG 569] #12876
Handle categorical slots in command generator - [ENG 569] #12876
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, just a small typing thing to take care of :)
rasa/dialogue_understanding/patterns/default_flows_for_patterns.yml
Outdated
Show resolved
Hide resolved
Kudos, SonarCloud Quality Gate passed! 5 Bugs 0.0% Coverage The version of Java (11.0.20) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I really like the idea here, I think this is close to a really good way of handling this issue 😎! A few things that I'd like to see improved:
- I think the completely right approach is to have all type checks be a default rejection
- don't just execute this type check rejection when no others are there. It should be always there
- apply it here for all types, like floats and bools and categorial slots, not only the categorial slots. This means moving some code from the command generator that catches these cases for floats, bools, etc. here
- The message should be something like "Sorry, it seems 'X' is not a valid 'Type'"
- x being the value
- type being something like "number" if the slot is float, "option" if bool and categorial
- just first suggestions, feel free to improve upon this yourself. But I think it would be really nice to see the value and the slot this is about
- Maybe you can also undo the big indentation change, it makes the diff really challenging to read
Proposed changes:
Skips slot set command if the LLM proposes a slot value that is not part of the categorical list of values defined in the domainpattern_collect_information
," is executed. The initial step defined within this pattern is theActionRunSlotRejections
action. I've implemented extra validations within this action to reset any invalid categorical values set by the LLM. This approach allows for the reuse of code responsible for fetching slot values and generating bot responses. Additionally, it ensures that user-defined rejections within the flow take precedence.Among the potential solutions considered but not yet implemented are:
Adding a new default action to implement the categorical slot check as the initial step in the
pattern_collect_information
. This method doesn't leverage most of the existing code inActionRunSlotRejections
, and its necessity for introducing a new default action is uncertain.Implementing the categorical slot check before the slotset event in the slot set command. The skip information for the slot is stored in a default slot, triggering the bot utterance step in the default pattern -
pattern_collect_information
. This approach, however, lacks computational efficiency.Status (please check what you already did):
black
(please check Readme for instructions)