Prompt generation step is so slow, take 4 minutes

[ssling0817]

I am using DatabricksConnector for input dataframe. Its size is 2017x41, but it always took around 4 minutes to generate prompt.
I guess its because databricks_connector.to_json() needs around 3 minutes to complete.
Is there any way to speed up the process?

`2024-06-02 13:47:20 [INFO] Executing Pipeline: GenerateChatPipeline

2024-06-02 13:49:43 [INFO] Executing Step 0: ValidatePipelineInput

2024-06-02 13:49:43 [INFO] Executing Step 1: CacheLookup

2024-06-02 13:49:43 [INFO] Executing Step 2: PromptGeneration

2024-06-02 13:53:21 [INFO] Using prompt:

dfs[0]:2017x41`