Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store tokens in u32 instead of usize #15

Merged
merged 1 commit into from
Oct 17, 2024
Merged

Store tokens in u32 instead of usize #15

merged 1 commit into from
Oct 17, 2024

Conversation

tmm1
Copy link

@tmm1 tmm1 commented Oct 17, 2024

based on upstream commit openai@c2960c1

cc openai#251

@tmm1
Copy link
Author

tmm1 commented Oct 17, 2024

Running benches/bench.rs (target/release/deps/bench-c2328a0362693adb)
cl100k_base_estimation  time:   [1.9916 ms 1.9949 ms 1.9987 ms]
                        change: [-0.3951% -0.1311% +0.1318%] (p = 0.33 > 0.05)
                        No change in performance detected.
Found 16 outliers among 150 measurements (10.67%)
  7 (4.67%) high mild
  9 (6.00%) high severe

cl100k_base             time:   [14.279 ms 14.297 ms 14.317 ms]
                        change: [-3.6656% -3.5011% -3.3338%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 6 outliers among 150 measurements (4.00%)
  2 (1.33%) high mild
  4 (2.67%) high severe

cl100k_base_ordinary    time:   [14.451 ms 14.476 ms 14.506 ms]
                        change: [-1.0329% -0.8089% -0.5677%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 10 outliers among 150 measurements (6.67%)
  2 (1.33%) high mild
  8 (5.33%) high severe

cl100k_base_50atatime   time:   [17.989 ms 18.002 ms 18.017 ms]
                        change: [-3.0768% -2.9049% -2.7420%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 150 measurements (2.67%)
  2 (1.33%) high mild
  2 (1.33%) high severe

@tmm1 tmm1 merged commit e3c7845 into main Oct 17, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant