Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename Cost to Usage #403

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Rename Cost to Usage #403

wants to merge 1 commit into from

Conversation

dmontagu
Copy link
Contributor

@dmontagu dmontagu commented Dec 18, 2024

While looking through @sydney-runkle's work on execution limits, I became pretty strongly convinced that this type should be called/thought of as Usage rather than Cost, and I wanted to rename it.

Some more specific reasons:

  • I think it makes more sense to describe the method that returns this struct as "Return the usage of the request" than "Return the cost of the request" when we are just getting token counts and not a monetary value. The word "cost" just feels like it should be referring to money.

  • Right now, the docstring of the Cost class says:

    You'll need to look up the documentation of the model you're using to convent "token count" costs to monetary costs.

    I think the fact that we need to use quotes in "token count" costs is representative that it's not obvious/expected that cost isn't money, and if we call this class Usage, we can avoid that phrase "token count" costs entirely. We can just say "you'll need to look up the documentation of the model you're using to convert usage into monetary costs".

  • Usage provides a clear place to track request count (which we want to add limits to), and it doesn't feel at all out of place weird to add that field to a type called Usage in the way it kind of does feel weird to add it to a type called Cost. (Though even if we keep the name as Cost, I want to add request count to that structure rather than store usage state on something else..)

  • Anthropic uses the term "usage" for everything related to token counts — see here https://github.com/anthropics/anthropic-sdk-python#token-counting. (Sydney pointed this out to me after I suggested it; I wanted to use that term independently/before I realized Anthropic did too.) I see this as confirmation that this name is a reasonable choice, and using it also has the benefit of matching existing LLM SDK conventions (i.e., anthropic's).

I'm going to open some subsequent PRs implementing the ExecutionLimit stuff based on @sydney-runkle's work on top of this, I just wanted to make this its own PR to keep it easier to review diffs.

@dmontagu dmontagu changed the title Rename Cost to Usage Rename Cost to Usage Dec 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant