Support context of conversation between User and Assistant

[Xanonymous-GitHub]

See https://platform.openai.com/docs/guides/chat for more details.

In this proposal we should think about how to reduce the cost of conversation. Currently openAI doesn't provide any session related mechanism for storing the conversation state. Hence we have to send all the conversation history to openAI in EACH chat completion request.

There may be some workarounds for solving this problem.
Firstly, establish an AI middle wall (Optimizer) between the real Answer generation and user client.

[Client] <<====>> [Optimizer] <<====>> OpenAI
                     ^^^^

In this optimizer, we shrink ALL previous chat conversion sent by the User and Assistant role into one sentence, which makes us only need to send two msgs to OpenAI each time.

              /------<-------<--------<-----<----|
             /                                   |
[Previous Context] ----o---> [OpenAI] -->---->---|
[  Next Chat Msg ] ---/

Secondly, in the optimizer, we not only do things above but also ask AI for helping us organize msg content. Some approaches like removing unnecessary words to let AI smartly fill them; Avoid spaces and use CamelCase rule to express period positions.

[Xanonymous-GitHub]

Those are articles which talked about session ID and pruning of chat completion.
https://community.openai.com/t/why-does-the-chat-completions-endpoint-not-use-a-session-id/81791
https://community.openai.com/t/openai-api-chat-completion-pruning-methods/85237