FastChat api model

[bsabri]

No description provided.

[fdalvi]

Apologies for not getting to this sooner, but I was in the middle of some refactoring that might make things easier here; Since its really just using the OpenAI (previously GPTChatCompletionModel in our case), shall we just inherit from it? Something like:

class FastChatModel(OpenAIModel):
    def __init__(api_base, api_key, model_name, **kwargs):
        api_base = api_base or os.getenv("FASTCHAT_API_URL")
        ... # something similar for api_key and model_name
        # checks for valid config settings)
        super(FastChatModel, self).__init__(api_base=api_base, api_key=api_key, model_name=model_name, **kwargs)

should do the trick without much re-implementation, what do you think? We can also do it just for ChatCompletion and not the old legacy string-based way. Let me know, we can discuss in person if you like!

[bsabri]

This sounds good! Thanks! I will update the model based on the new refactoring. I'll get back to you if needed.

[bsabri]

environment variables:
OPENAI_API_BASE= "http://10.4.64.46:5004/v1"
OPENAI_API_KEY="EMPTY"
OPENAI_MODEL= "jais-13b-chat"