Add Support for WhatsApp Channel via WhatsApp Cloud API

[ivarunseth]

This pull request introduces support for integrating WhatsApp as a messaging channel in the Botpress Messaging Server, leveraging the WhatsApp Cloud API. The implementation enables chatbot interactions on WhatsApp, expanding the platform's reach and providing users with a popular and widely used communication medium.

Key Features:

• WhatsApp Channel Integration: Added the capability to connect and interact with users on WhatsApp through the WhatsApp Cloud API.
• Renderer Support: Implemented support for various message renderers to ensure a seamless experience. This includes text, image, audio, video, document, and button templates.
• Exclusions: Carousel renderer support is not included in this implementation and will be considered for future updates.

Implementation Details:

1. WhatsApp API Client:

   • Integrated the WhatsApp Cloud API to handle message sending and receiving.
   • Added configuration options for WhatsApp credentials and settings.

2. Message Handling:

   • Extended message handling logic to support WhatsApp-specific message types and formats.
   • Implemented parsers and formatters for converting messages between Botpress and WhatsApp formats.

3. Renderer Support:

   • Text messages: Simple and multi-line text messages.
   • Media messages: Support for images, audio files, videos, and documents.
   • Interactive messages: Buttons and quick replies.

4. Documentation:

   • Updated README and documentation files.

Testing:

• Conducted extensive testing to ensure reliability and compatibility with the WhatsApp Cloud API.
• Verified the correct rendering of supported message types across different devices.
• Ensured smooth fallback and error handling mechanisms are in place.

Future Work:

• Implement support for Carousel renderers to complete the full range of message types.
• Continuously monitor and update the integration to accommodate any changes in the WhatsApp API.

Notes:

• These changes require an update in Botpress Studio to support whatsapp channel.

---

By integrating WhatsApp, this update significantly enhances the versatility and usability of the Botpress Messaging Server, enabling developers to create richer, more engaging chatbot experiences. Looking forward to feedback and suggestions for further enhancements.

---

[ivarunseth]

Hello @slvnperron, just wanted to kindly check in on this PR. It’s been some time, and I’m eager to contribute further based on any feedback. Thanks in advance for your time and review!

[davidvitora]

hey @ivarunseth, thanks for contributing, sorry for the late response

I will verify your request with the team and get back to you

Question, do you know how to use a custom build of the messaging server on the current distributed Botpress binaries so you can access your features right away?

it should be possible to do just by replacing the binary from the /bin folder from your docker image (at /botpress) or Botpress folder

[ivarunseth]

Hey @davidvitora,

thanks for your time to view the PR!

To answer your question, I have built the binaries of messaging server locally using yarn package command. I then replaced the messaging server binary file in botpress server located at "v12/packages/bp/dist/bin" path with the new build to verify the features.

I m yet to build it using docker and verify the same.

Please note that I had to add whatsapp in channelName array in the MessagingInteractor class located in v12/packages/bp/src/core/messaging/subservices/interactor.ts file in the v12 repository to make it work.

Looking forward to work on any comments after review.

Thanks!

[davidvitora]

@ivarunseth

I saw that you didn't implement the Card or Carrousel renderers, the way we do it at Twilio or Vonage (Which can connect to WhatsApp) is to send multiple messages (Since sending a real Card or Carousel requires approving a template with Meta), do you want to add the same to this Channel? If you don't, there is no issue with that, we just won't be able to merge this to this Repo, which in the end of the day shouldn't make much difference for you, since v12 is on LTS and we are just pushing Bugs/Security updates, meaning you would need to compile and use a custom version from the messaging server with your Botpress core

[ivarunseth]

@davidvitora

Hey, I just saw how card & carousel renderers are implemented in twilio and vonage channels.

As you mentioned, since sending templated messages requires approval from meta, I had not implemented the carousel renderer.

Since this PR is with respect to open issue #504 I will be making required changes as soon as possible.

Thanks!

[Akshat7321]

Hi @ivarunseth any updates on this ? Is this working fine or their are any issues with this ?

[ivarunseth]

Hey @davidvitora, @Akshat7321,

As of now, apart from card and carousel content types, everything is working.

I'm implementing the card and carousel content types using interactive reply buttons and list messages in Whatsapp Cloud API since it does not require approval from Meta unlike template messages (discussed earlier).

In case the content generated by a bot developer does not follow the post request parameters of the API, I'm falling back to the approach used in twilio and vonage messaging channels.

Once finished, I will test the code and address any additional feedback if needed.

[AnshuChatbot]

Hi @ivarunseth, could you please tell me how to stop the existing messaging channel and run these channels using Docker Compose? Where I have to make changes?

[ivarunseth]

Hi @ivarunseth, could you please tell me how to stop the existing messaging channel and run these channels using Docker Compose? Where I have to make changes?

Hello @AnshuChatbot,

When you make changes to messaging channels, run yarn package command to create binary files of messaging server. Then build the docker image, refer this Dockerfile.

Assuming you have replaced some existing messaging channel, you should be able to use this image with docker compose to run the services.

[AnshuChatbot]

I am able to run these messaging channels in a separate container, but I'm unable to attach them to our running host. Could you suggest how to remove the existing channel?

[ivarunseth]

I am able to run these messaging channels in a separate container, but I'm unable to attach them to our running host. Could you suggest how to remove the existing channel?

In that case, you should check if the image and port bindings are correct in your docker-compose.yml file for the messaging service.

[AnshuChatbot]

@ivarunseth,Ok can you have any documentation that how to remove the existing messaging channel and attach these one if available please provide me ? I have no idea from where we remove the by default messaging launcher that are running in port 3100 .

[ivarunseth]

Hello @davidvitora,

I have completed the card, carousel & choices implementation. Awaiting your feedback, please review :)

Thanks!

[davidvitora]

Hey @ivarunseth,

Nice, as soon as I get a chance to look at it I will let you know

Could you also document the setup for this channel just like we did for Twillio and others? I want make sure that i'm able to setup and test this channel

[davidvitora]

@ivarunseth,Ok can you have any documentation that how to remove the existing messaging channel and attach these one if available please provide me ? I have no idea from where we remove the by default messaging launcher that are running in port 3100 .

hey @AnshuChatbot, you need to build the messaging server as a binary (details on it is done can be find at our github actions workflows, but you need to do on your machine), then you can just replace the messaging server binary in your Botpress server at BOTPRESS_ROOT/bin

[AnshuChatbot]

Hey @davidvitora,

thanks for your time to view the PR!

To answer your question, I have built the binaries of messaging server locally using yarn package command. I then replaced the messaging server binary file in botpress server located at "v12/packages/bp/dist/bin" path with the new build to verify the features.

I m yet to build it using docker and verify the same.

Please note that I had to add whatsapp in channelName array in the MessagingInteractor class located in v12/packages/bp/src/core/messaging/subservices/interactor.ts file in the v12 repository to make it work.

Looking forward to work on any comments after review.

Thanks!

@ivarunseth I am done same thing but still I am facing bug that are internal server 500 and the error is look like these
{
[Messaging] API Error occurred calling route /api/v1/sync. TypeError: Cannot read property 'meta' of undefined
at SyncApi.validate (/snapshot/messaging-whatsapp-channel/packages/server/dist/sync/api.js)
at SyncApi.sync (/snapshot/messaging-whatsapp-channel/packages/server/dist/sync/api.js)
at /snapshot/messaging-whatsapp-channel/packages/framework/dist/base/api-manager.js
at /snapshot/messaging-whatsapp-channel/packages/framework/dist/base/auth/client.js
at processTicksAndRejections (node:internal/process/task_queues:96:5)
10/01/2024 11:00:50.341 Messaging Failed to configure channels
HTTP (post) URL /sync
Received "Internal Server Error"
Request failed with status code 500
}
can you help me ? As I am thinking I am done all steps that you have suggest and also I am able to connect the new messaging channel . I think I faced issue on messaging Interactor class.

[ivarunseth]

Hey @davidvitora,
thanks for your time to view the PR!
To answer your question, I have built the binaries of messaging server locally using yarn package command. I then replaced the messaging server binary file in botpress server located at "v12/packages/bp/dist/bin" path with the new build to verify the features.
I m yet to build it using docker and verify the same.
Please note that I had to add whatsapp in channelName array in the MessagingInteractor class located in v12/packages/bp/src/core/messaging/subservices/interactor.ts file in the v12 repository to make it work.
Looking forward to work on any comments after review.
Thanks!

@ivarunseth I am done same thing but still I am facing bug that are internal server 500 and the error is look like these { [Messaging] API Error occurred calling route /api/v1/sync. TypeError: Cannot read property 'meta' of undefined at SyncApi.validate (/snapshot/messaging-whatsapp-channel/packages/server/dist/sync/api.js) at SyncApi.sync (/snapshot/messaging-whatsapp-channel/packages/server/dist/sync/api.js) at /snapshot/messaging-whatsapp-channel/packages/framework/dist/base/api-manager.js at /snapshot/messaging-whatsapp-channel/packages/framework/dist/base/auth/client.js at processTicksAndRejections (node:internal/process/task_queues:96:5) 10/01/2024 11:00:50.341 Messaging Failed to configure channels HTTP (post) URL /sync Received "Internal Server Error" Request failed with status code 500 } can you help me ? As I am thinking I am done all steps that you have suggest and also I am able to connect the new messaging channel . I think I faced issue on messaging Interactor class.

Hey @AnshuChatbot, I see you were able to build the messaging server, and connect your bot to whatsapp messaging channel. Could you please share the messaging channel config in your bot configuration for whatsapp? This is a sample messaging channel config you need to add in bot.config.json file,

  "messaging": {
    "channels": {
      "whatsapp": {
        "appId": "<APP_ID>",
        "appSecret": "<APP_SECRET>",
        "accessToken": "<ACCESS_TOKEN>",
        "markRead": <true/false>,
        "phoneNumberId": "<PHONE_NUMBER_ID>",
        "verifyToken": "<VERIFY_TOKEN>",
        "version": "1.0.0" 
      }
    }
  }

I believe you did not add "version" key with "1.0.0" value in the object which is causing the issue.

Also, if you are still facing issue in interactor.ts file, there is private variable "channelNames", you just need to add "whatsapp" in the list.

I hope this helps!

[AnshuChatbot]

@ivarunseth I am getting the below information in the log. Is there any things in the /messaging-whatsapp-channel/packages/channels/src/whatsapp/channel.ts this file that I have to change to proceed ahead

here the logs
response: {
status: 400,
statusText: 'Bad Request',
headers: {
vary: 'Origin',
'x-business-use-case-usage': '{"109889978794644":[{"type":"whatsapp","call_count":1,"total_cputime":1,"total_time":1,"estimated_time_to_regain_access":0}]}',
'content-type': 'application/json',
'www-authenticate': 'OAuth "Facebook Platform" "invalid_request" "(#100) Invalid parameter"',
'access-control-allow-origin': '*',
'facebook-api-version': 'v20.0',
'strict-transport-security': 'max-age=15552000; preload',
pragma: 'no-cache',
'cache-control': 'no-store',
expires: 'Sat, 01 Jan 2000 00:00:00 GMT',
'x-fb-request-id': 'AYqUDHVx9LjxJcfD_87QoRO',
'x-fb-trace-id': 'D4wELzWb1+J',
'x-fb-rev': '1016949697',
'x-fb-debug': 'HykJ93KJW3OeBU7qHnYtm+CdKJysGoEF0XA6Q4ltiwshaTFcvVGW8lKhjhWSxYmR+bjHYJTBfYhfWbITuo3UOQ==',
date: 'Tue, 01 Oct 2024 13:24:51 GMT',
'proxy-status': 'http_request_error; e_proxy="AcKnQDLq-OyHAi6D9C7wj_zNs-67PM8DTYbaKscB0EGr6qxfa368q56uQ6k6pGj1nHXYEdDKGGfEGZwjkUWw"; e_fb_binaryversion="AcKgnySaaH3fVHZSEhl-rM8UC5vFPx_st6emyaEhHaYlFcHn43xFCmcH3AeGp9q4dstZRb4iYh1iF26W7XCrpWkcDkWF8fMZelQ"; e_fb_httpversion="AcLsbShtak_9jjEcAdGSmerIwUPxm5u0r8l8qKa1xVE9zH8WRS_qLK65O_8J"; e_fb_responsebytes="AcIX5cnJrqtIGLl7dTrj3VQOylgKLLMFRF8Uy5ngE23YGo4FzymCoL0ONFtJ"; e_fb_requesttime="AcL6SaBZDSm4hcG9duQLe5PrLtpre7gTlels4KsxAa3sjJZDim_FkgHhYZvawuxiI2XG7QhBKg"; e_fb_requesthandler="AcJmvR0JPtnGG2FsVnE-sYHI39kqAbHQBBLkTYWgi_qqrnr7vkdDTb5HOwDccPoWXnOLUQA-vNo"; e_fb_hostheader="AcKksmkplA_fjLeiM_gbngxZ7aWs9lORPgrau7OVu0ZnJW2Ok9ujGNgHWoE_p8uuacmm5RQVcCmx18mY"; e_fb_requestsequencenumber="AcIVuRh4bj-J7cEELSeRKFxmidyMah3KMIaAVktkb4-pFjaGr3QlJ7z03rUI"; e_upip="AcL7JlwI5RtKnW0FxBotQ8yU17QctCfB6ripb1KEeeX5A378IX-O3O73NZ4hg5_wl1xjGhCgIXmVDt4RYKYwV8iWvfK2DfEUDqWWP30"; e_fb_builduser="AcLIs2fpd0WElXFKmxM-3UqAejpwVKE8J-1R2qOWNOXUtIyMm9nFz47KuKJp8C6DkDI"; e_fb_vipport="AcI5Wny6aXiyXl0PMcw28Y2BFCouVVcSBk7x2ma503w4mk1qCDZqPleIM6qd"; e_clientaddr="AcK4eH-BGtxIznxa2NEX6Ww5_QVqVT69pqMV7pXU0BMMqpUJJj8r1F8EZz0Vsanof-kAPNPYZIM3E3AYGn10QgHfgr8stvRny7XSwiPHQfVQKRc"; e_fb_vipaddr="AcKiyHC6qcdWjx-0MuMiuMvEHk5WL5yQXv8ATMjjBM7ZFg1Ay4t8QUlY26f_Sey1eFvxj6zbcoVFd8wETMQOAP5CKD8GKsc"; e_fb_configversion="AcIp77zLLmX2g1rjYQtnKx7ScaofDl2It931jMbTj15ycGM1v63dTD2_fliwMQ", http_request_error; e_proxy="AcKhKOYPd2WZvjUN292SvmVawLwXXd6c11y6LHTj9H_yN4L_QanvEFU6tJSSBmF5RRlqwZm04Guiiyc"; e_fb_binaryversion="AcI9zBJ57oqwIC0FJGMRcWN4E1yTThLe0Kl9O3IYwQxHZD-RVXKenzb8Fl6I6SBBC4dH85VKHNCUPPsc2E9JfNMKQaVz_mkCh4"; e_fb_httpversion="AcIDATSA1Yn6qw1q3DOkvRqhzlDuSTUTnxYXStbaM7r-tR2kbhCLRdquXC5T"; e_fb_responsebytes="AcLTKqRB3Lzsb3FZ9sQt_3lTAPkBQdd_cN1Vrg_QGd7ixTKSMIm_16MBGqbk"; e_fb_requesttime="AcJAPidyis26PCYruWiOfcL1qalAd5CXRYx3sgAg--gJbTyp_i-Eeu9l2TiYUX17-omu08h9Mg"; e_fb_requesthandler="AcL7U6TqYgbkctOylmhGve3VD2IXWyIa4YzOg-yMmLikv-LcOqsOMjq1QP0B9trIeAczKnBjy44"; e_fb_hostheader="AcL_IuQ5mk3kgXXfSm3rLVUVTjZqlGNg-wmKW4e76HnS6PgHeHe3JZ-izZU2gfjofMNYzWN-tgjKYtqh"; e_fb_requestsequencenumber="AcLDU_Wp-8DWJUtTqF9lNEbHT127LKD0KeMtVIDi_du9xAGK8OytlJz3EQ"; e_upip="AcIoLfY_rQZwAnQGNo0PQiA9UuI66MpeqMTO5BsdKkifwlszQQgB0qF0vbw0xaU6n3ewiE1cZm-zuSYwnyqPi7Kl4PWzIxY"; e_fb_builduser="AcKiNwr4WJPb8zZOEsaItgkD9TlYYAE_3Qbnd4lXG-iVzD9g47Y3ZeNKSD1BkeyXgdk"; e_fb_vipport="AcKmkruOHE3O8W7jPRhVIIE0WESQiLFRF4IJJH28OCWqiyk2xA5IjxShg-W"; e_clientaddr="AcK20giL9hIobPVYntiqeR9LpGx5ORAaIL5N5LaDenH_Ix0mskc3RnNEpzOlnmHn03ybKpFWHp2ZQlmB2xU"; e_fb_vipaddr="AcJcJJzZ5JocKD3k0VYk5MZteOnUraQLddEikslourU4WPrBMfZhCpURENwVslV2BUIIRgFyOm8"; e_fb_configversion="AcKVaGbzib3iv2otvmQSNPn_Y3N7yTjLjeidUijQ3OpKPyP7Vh9S9m5OAXvlcg"',
'x-fb-connection-quality': 'EXCELLENT; q=0.9, rtt=14, rtx=0, c=10, mss=1392, tbw=3437, tp=-1, tpl=-1, uplat=325, ullat=0',
'alt-svc': 'h3=":443"; ma=86400',
connection: 'close',
'content-length': '226'
},
config: {
transitional: [Object],
adapter: [Function: httpAdapter],
transformRequest: [Array],
transformResponse: [Array],
timeout: 0,
xsrfCookieName: 'XSRF-TOKEN',
xsrfHeaderName: 'X-XSRF-TOKEN',
maxContentLength: -1,
maxBodyLength: -1,
validateStatus: [Function: validateStatus],
headers: [Object],
method: 'post',
url: 'https://graph.facebook.com/v20.0/111216351995192/messages',
data: '{"messaging_product":"whatsapp","recipient_type":"individual","to":"xxxx-xxx-xxx","status":"read"}'
},
request: <ref *1> ClientRequest {
_events: [Object: null prototype],
_eventsCount: 7,
_maxListeners: undefined,
outputData: [],
outputSize: 0,
writable: true,
destroyed: false,
_last: true,
chunkedEncoding: false,
shouldKeepAlive: false,
_defaultKeepAlive: true,
useChunkedEncodingByDefault: true,
sendDate: false,
_removedConnection: false,
_removedContLen: false,
_removedTE: false,
_contentLength: null,
_hasBody: true,
_trailer: '',
finished: true,
_headerSent: true,
_closed: false,
socket: [TLSSocket],
_header: 'POST /v20.0/111216351995192/messages HTTP/1.1\r\n' +
'Accept: application/json, text/plain, /\r\n' +
'Content-Type: application/json\r\n' +
'Authorization: Bearer +
'User-Agent: axios/0.25.0\r\n' +
'Content-Length: 98\r\n' +
'Host: graph.facebook.com\r\n' +
'Connection: close\r\n' +
'\r\n',
_keepAliveTimeout: 0,
_onPendingData: [Function: nop],
agent: [Agent],
socketPath: undefined,
method: 'POST',
maxHeaderSize: undefined,
insecureHTTPParser: undefined,
path: '/v20.0/111216351995192/messages',
_ended: true,
res: [IncomingMessage],
aborted: false,
timeoutCb: null,
upgradeOrConnect: false,
parser: null,
maxHeadersCount: null,
reusedSocket: false,
host: 'graph.facebook.com',
protocol: 'https:',
_redirectable: [Writable],
[Symbol(kCapture)]: false,
[Symbol(kNeedDrain)]: false,
[Symbol(corked)]: 0,
[Symbol(kOutHeaders)]: [Object: null prototype]
},
data: { error: [Object] }
},
isAxiosError: true,
toJSON: [Function: toJSON]
}

[ivarunseth]

Hey @AnshuChatbot,

You are facing an error when making a post request to whatsapp messages API. The request is intended to mark a user message as read by the bot.

I had implemented the markRead option just for testing if it works, but in most cases this is optional and shall be set to false to keep the number of requests low.

The request body structure and parameters looks correct to me, but you still got 400 status code. This could mean something is wrong with the channel configuration in your bot.

Have you created an app on Meta platform and setup the whatsapp product? You will have to configure the webhook of bot in a meta application and subscribe for messages event.

Assuming you are running this locally, you can follow the steps below to integrate your bot with whatsapp --

1. Expose your local botpress server to the internet by tunnelling using ngrok. you can run ngrok http 3000 after installing the same.

2. The above step will give a url which you need to configure in global botpress configuration as externalUrl. After that, restart the server.

3. Now go to Meta for developers platform and login.

4. Go to My Apps and click on Create App button.

5. Select use case as Other > app type as Business > give your meta app a name and create it.

6. Setup Whatsapp from the product list in your application and go through the quickstart.

7. Go to API setup and generate an accessToken for the app. You can find other required information like appId and phoneNumberId, the appSecret will be in settings.

8. Configure the whatsapp channel in your bot configuration like mentioned before and restart the botpress server.

  "messaging": {
    "channels": {
      "whatsapp": {
        "appId": "<APP_ID>",
        "appSecret": "<APP_SECRET>",
        "accessToken": "<ACCESS_TOKEN>",
        "markRead": <true/false>,
        "phoneNumberId": "<PHONE_NUMBER_ID>",
        "verifyToken": "<VERIFY_TOKEN>",
        "version": "1.0.0" 
      }
    }
  }

If everything is correct, botpress will expose a webhook of your bot which you can find in logs like this,

9. Now go back to your Meta app and configure the webhook using the verifyToken.

10. Subscribe to the messages events so that you can send and receive messages and status events.

Please share further logs if you still face issue with the API.

[ivarunseth]

Hey @ivarunseth,

Nice, as soon as I get a chance to look at it I will let you know

Could you also document the setup for this channel just like we did for Twillio and others? I want make sure that i'm able to setup and test this channel

Hello @davidvitora,

Sure I will add the documentation asap, meanwhile you can use the same steps above for the setup and testing.

Thanks!

[AnshuChatbot]

Hey @ivarunseth,

I'm encountering an issue after setting up messaging through WhatsApp. Although ngrok shows a status code 200, I'm facing a bug when I set markRead to true. When this is enabled, I get error logs, but if I set markRead to false, no errors appear. However, in that case, the response from the bot doesn't get displayed in whatsapp .

Here are the error logs that appear when markRead is set to true:
{
Request failed with status code 400
at createError (/snapshot/messaging-whatsapp-channel/node_modules/axios/lib/core/createError.js:16:15)
at settle (/snapshot/messaging-whatsapp-channel/node_modules/axios/lib/core/settle.js:17:12)
at IncomingMessage.handleStreamEnd (/snapshot/messaging-whatsapp-channel/node_modules/axios/lib/adapters/http.js:312:11)
at IncomingMessage.emit (node:events:377:35)
at IncomingMessage.emit (node:domain:470:12)
at endReadableNT (node:internal/streams/readable:1312:12)
at processTicksAndRejections (node:internal/process/task_queues:83:21)
These and the upper logs that I have shared with you earlier
}.
Could you suggest what I should do next?

[ivarunseth]

{
Request failed with status code 400
at createError (/snapshot/messaging-whatsapp-channel/node_modules/axios/lib/core/createError.js:16:15)
at settle (/snapshot/messaging-whatsapp-channel/node_modules/axios/lib/core/settle.js:17:12)
at IncomingMessage.handleStreamEnd (/snapshot/messaging-whatsapp-channel/node_modules/axios/lib/adapters/http.js:312:11)
at IncomingMessage.emit (node:events:377:35)
at IncomingMessage.emit (node:domain:470:12)
at endReadableNT (node:internal/streams/readable:1312:12)
at processTicksAndRejections (node:internal/process/task_queues:83:21)
These and the upper logs that I have shared with you earlier
}.

Hey, @AnshuChatbot.

You are getting 400 Bad Request from the Whatsapp Messages API. Without the error message from the API, I won't be able to help you.

I suggest you try hitting the Meta's API endpoint with your credentials and same payload on Postman, you will see exactly what is wrong in your configuration.