Estimate token usage, cost

[ww-jermaine]

Hello, is there a way to estimate the token usage and cost per call of ai_fn, ai_model, etc.?

Something like the callback from langchain:

Tokens Used: 0
        Prompt Tokens: 0
        Completion Tokens: 0
Successful Requests: 0
Total Cost (USD): $0.0

[nejko]

I would LOVE this feature!

[pengelbrecht]

Any news here? How do folks keep track of token usage with Marvin?

[zzstoatzz]

hey all, apologies for the radio silence here. I'll try to keep better track of these going forward

I've answered similar questions in discord (come chat!) but here's a gist

tldr

• subclass MarvinClient and add some ChatCompletion processor to do what you want with the usage OpenAI gives back
• you can use some basic util like I have here to inspect payloads before we send them out. we should probably add this ourselves to make it easy