Bot returns an error when using Regex validation in adaptive cards

[gtonye]

Github issues should be used for bugs and feature requests. Use Stack Overflow for general "how-to" questions.

Hi team,

Please let me know if you would need additional information, this is currently halting a few use cases in production for us.

Best,

Version

What package version of the SDK are you using.

    <PackageReference Include="Microsoft.Bot.Builder.AI.Luis" Version="4.18.1" />
    <PackageReference Include="Microsoft.Bot.Builder.AI.QnA" Version="4.18.1" />
    <PackageReference Include="Microsoft.Bot.Builder.Dialogs.Adaptive.Runtime" Version="4.18.1" />

Describe the bug

Give a clear and concise description of what the bug is.

When adding an adaptive card with escape characters, the bot throws an error:

SimpleBot.en-us.lg:Bad JSON escape sequence: \(. Path 'body[1].regex', line 17, position 19. [SendActivity_Greeting]  Error occurred when evaluating expression. [__temp__2e107cdbb4794f199d2deef102998b02]  Error occurred when evaluating expression.

In the bot code, the following stack trace:

      [OnTurnError] unhandled error : SimpleBot.en-us.lg:Bad JSON escape sequence: \(. Path 'body[1].regex', line 17, position 19. [SendActivity_Greeting]  Error occurred when evaluating expression. [__temp__3a1a41dc84584e56a5f47d4261817df8]  Error occurred when evaluating expression.,
      SimpleBot.en-us.lg:Bad JSON escape sequence: \(. Path 'body[1].regex', line 17, position 19. [SendActivity_Greeting]  Error occurred when evaluating expression. [__temp__5a7af11be1f246ae9c7f2fb57ee0bcb5]  Error occurred when evaluating expression.
      System.InvalidOperationException: SimpleBot.en-us.lg:Bad JSON escape sequence: \(. Path 'body[1].regex', line 17, position 19. [SendActivity_Greeting]  Error occurred when evaluating expression. [__temp__3a1a41dc84584e56a5f47d4261817df8]  Error occurred when evaluating expression.,
      SimpleBot.en-us.lg:Bad JSON escape sequence: \(. Path 'body[1].regex', line 17, position 19. [SendActivity_Greeting]  Error occurred when evaluating expression. [__temp__5a7af11be1f246ae9c7f2fb57ee0bcb5]  Error occurred when evaluating expression.
         at Microsoft.Bot.Builder.Dialogs.Adaptive.Generators.MultiLanguageGeneratorBase.GenerateAsync(DialogContext dialogContext, String template, Object data, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.Adaptive.Templates.ActivityTemplate.BindAsync(DialogContext dialogContext, Object data, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.Adaptive.Actions.SendActivity.BeginDialogAsync(DialogContext dc, Object options, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.DialogContext.BeginDialogAsync(String dialogId, Object options, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.Adaptive.Actions.ActionScope.BeginActionAsync(DialogContext dc, Int32 offset, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.Adaptive.Actions.ActionScope.BeginDialogAsync(DialogContext dc, Object options, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.DialogContext.BeginDialogAsync(String dialogId, Object options, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.DialogContext.ReplaceDialogAsync(String dialogId, Object options, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.Adaptive.Actions.IfCondition.BeginDialogAsync(DialogContext dc, Object options, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.DialogContext.BeginDialogAsync(String dialogId, Object options, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.Adaptive.Actions.ActionScope.BeginActionAsync(DialogContext dc, Int32 offset, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.Adaptive.Actions.ActionScope.BeginDialogAsync(DialogContext dc, Object options, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.DialogContext.BeginDialogAsync(String dialogId, Object options, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.Adaptive.Actions.ForEachElement.RunItemsAsync(DialogContext dc, Boolean beginDialog, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.Adaptive.Actions.ForEachElement.BeginDialogAsync(DialogContext dc, Object options, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.DialogContext.BeginDialogAsync(String dialogId, Object options, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.Adaptive.Actions.ActionScope.BeginActionAsync(DialogContext dc, Int32 offset, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.Adaptive.Actions.ActionScope.BeginDialogAsync(DialogContext dc, Object options, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.DialogContext.BeginDialogAsync(String dialogId, Object options, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.Adaptive.AdaptiveDialog.ContinueActionsAsync(DialogContext dc, Object options, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.Adaptive.AdaptiveDialog.BeginDialogAsync(DialogContext dc, Object options, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.DialogContext.BeginDialogAsync(String dialogId, Object options, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.DialogExtensions.InnerRunAsync(ITurnContext turnContext, String dialogId, DialogContext dialogContext, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.DialogExtensions.InternalRunAsync(ITurnContext turnContext, String dialogId, DialogContext dialogContext, DialogStateManagerConfiguration stateConfiguration, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.DialogExtensions.InternalRunAsync(ITurnContext turnContext, String dialogId, DialogContext dialogContext, DialogStateManagerConfiguration stateConfiguration, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.DialogExtensions.RunAsync(Dialog dialog, ITurnContext turnContext, IStatePropertyAccessor`1 accessor, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.Dialogs.Adaptive.AdaptiveDialogBot.OnTurnAsync(ITurnContext turnContext, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.SetSpeakMiddleware.OnTurnAsync(ITurnContext turnContext, NextDelegate next, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.MiddlewareSet.ReceiveActivityWithStatusAsync(ITurnContext turnContext, BotCallbackHandler callback, CancellationToken cancellationToken)
         at Microsoft.Bot.Builder.BotAdapter.RunPipelineAsync(ITurnContext turnContext, BotCallbackHandler callback, CancellationToken cancellationToken)

To Reproduce

Steps to reproduce the behavior:

1. Clone the repo https://github.com/gtonye/Adaptive-Card-Regex-Error-Repro
2. Run the bot locally (with the CLI or an IDE)
3. Using the emulator, connect to the bot
4. On the welcome message, it should trigger the error.

Expected behavior

Give a clear and concise description of what you expected to happen.

The bot should accept the regular expression. It is valid inside the Adaptive Card Designer (with the target version 1.3) as well as valid in https://regex101.com/

Screenshots

If applicable, add screenshots to help explain your problem.

Not applicable.

Additional context

Add any other context about the problem here.

Adaptive card I am testing with

{
    "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
    "type": "AdaptiveCard",
    "version": "1.3",
    "body": [
        {
            "type": "TextBlock",
            "text": "Present a form and submit it back to the originator"
        },
        {
            "type": "Input.Text",
            "id": "lastName",
            "placeholder": "1234567890",
            "label": "What is your phone number?",
            "isRequired": true,
            "errorMessage": "A valid number is required",
            "regex": "^\\(?([0-9]{3})\\)?[-. ]?([0-9]{3})[-. ]?([0-9 ]{4})\\ ?\\.?$"
        }
    ],
    "actions": [
        {
            "type": "Action.Submit",
            "title": "Submit"
        }
    ]
}

adaptive-card-repro-ac-designer.mov

[jamesemann]

@ceciliaavila @tracyboehrer would it be possible to triage this item? it is impacting our customers too

[ceciliaavila]

Hi @jamesemann, we'll tackle this issue this week.

[sw-joelmut]

Hi @gtonye and @jamesemann,

We've been reviewing the issue and debugging the provided bot to find why it is failing with the Bad JSON escape sequence: \(. error in the regex property.
Tracking down the problem, it leads to how the JSON prebuilt function receives and parses the AdaptiveCard JSON string value to an object.

The problem is on the double backslash, since the AdaptiveCard JSON is read as a string, the double backslash is being escaped, and when the "string to object" parsing happens, the remaining backslash is interpreted as another escaping character. More information on related issue microsoft/AdaptiveCards#4677.

To solve the issue, you need to provide 4 backslashes in the JSON, as shown in the image below.

[gtonye]

@sw-joelmut thank you for the investigation and the very detailed answer! It's very helpful context.

The challenge we have is that it's going to be difficult to expose this constraint to our customers as the corrected regular expression ^\\\\(?([0-9]{3})\\\\)?[-. ]?([0-9]{3})[-. ]?([0-9 ]{4})\\\\ ?\\\\.?$ is invalid both on Regex tester websites like regex101 and the adaptive card designer that I tested with the input:

{"$schema":"http://adaptivecards.io/schemas/adaptive-card.json","type":"AdaptiveCard","version":"1.3","body":[{"type":"TextBlock","text":"Present a form and submit it back to the originator"},{"type":"Input.Text","id":"lastName","placeholder":"1234567890","label":"What is your phone number?","isRequired":true,"errorMessage":"A valid number is required","regex":"^\\\\(?([0-9]{3})\\\\)?[-. ]?([0-9]{3})[-. ]?([0-9 ]{4})\\\\ ?\\\\.?$"}],"actions":[{"type":"Action.Submit","title":"Submit"}]}

Is there any opportunity to enhance or optimize the JSON prebuilt function to make it consistent with the behavior of the adaptive card engine?

Thank you,

[sw-joelmut]

Hi @gtonye,

As changing the default behavior of the JSON prebuilt function will break existing customers' bots, we could try adding a flag to treat the AdaptiveCard as it is, without altering its content.

[gtonye]

Apologies for the delay @sw-joelmut

we could try adding a flag to treat the AdaptiveCard as it is, without altering its content

That would work great! let me know if you have changes that would need testing. We have several regex samples our customers use that we can share.