Skip to content

Latest commit

 

History

History
36 lines (33 loc) · 4.19 KB

0.9.7.0_encode.md

File metadata and controls

36 lines (33 loc) · 4.19 KB
Raw
BenchmarkDotNet=v0.13.5, OS=macOS Ventura 13.3.1 (a) (22E772610a) [Darwin 22.4.0]
Apple M1 Pro, 1 CPU, 10 logical and 10 physical cores
.NET SDK=7.0.203
  [Host]     : .NET 7.0.5 (7.0.523.17405), Arm64 RyuJIT AdvSIMD
  Job-LDWOCB : .NET 7.0.5 (7.0.523.17405), Arm64 RyuJIT AdvSIMD DEBUG

BuildConfiguration=Debug
Method Categories Data Mean Ratio Gen0 Gen1 Gen2 Allocated Alloc Ratio
SharpTokenV1_0_28_ CountTokens 1. (...)57. [19866] 5,197,190.0 ns 1.00 601.5625 289.0625 - 3805771 B 1.00
TiktokenSharpV1_0_5_ CountTokens 1. (...)57. [19866] 1,519,773.6 ns 0.29 250.0000 125.0000 - 1571154 B 0.41
Tiktoken_ CountTokens 1. (...)57. [19866] 409,645.7 ns 0.08 49.3164 - - 309449 B 0.08
SharpTokenV1_0_28_ CountTokens Hello, World! 3,219.0 ns 1.00 0.6752 - - 4240 B 1.00
TiktokenSharpV1_0_5_ CountTokens Hello, World! 6,618.4 ns 2.06 2.1820 0.0381 - 13728 B 3.24
Tiktoken_ CountTokens Hello, World! 382.9 ns 0.12 0.0420 - - 264 B 0.06
SharpTokenV1_0_28_ CountTokens King(...)edy. [275] 62,979.4 ns 1.00 8.5449 0.3662 - 54160 B 1.00
TiktokenSharpV1_0_5_ CountTokens King(...)edy. [275] 21,826.6 ns 0.35 5.0964 0.2136 - 32096 B 0.59
Tiktoken_ CountTokens King(...)edy. [275] 5,224.0 ns 0.08 0.6409 - - 4032 B 0.07
SharpTokenV1_0_28_Encode Encode 1. (...)57. [19866] 4,933,558.4 ns 1.00 601.5625 296.8750 - 3805769 B 1.00
TiktokenSharpV1_0_5_Encode Encode 1. (...)57. [19866] 1,631,237.3 ns 0.33 251.9531 126.9531 1.9531 1571156 B 0.41
Tiktoken_Encode Encode 1. (...)57. [19866] 427,569.3 ns 0.09 59.5703 29.7852 - 375665 B 0.10
SharpTokenV1_0_28_Encode Encode Hello, World! 3,259.6 ns 1.00 0.6752 0.0038 - 4240 B 1.00
TiktokenSharpV1_0_5_Encode Encode Hello, World! 6,631.7 ns 2.03 2.1820 0.0458 - 13728 B 3.24
Tiktoken_Encode Encode Hello, World! 438.5 ns 0.13 0.1135 0.0005 - 712 B 0.17
SharpTokenV1_0_28_Encode Encode King(...)edy. [275] 72,625.5 ns 1.00 8.5449 0.4883 - 54160 B 1.00
TiktokenSharpV1_0_5_Encode Encode King(...)edy. [275] 21,666.4 ns 0.30 5.0964 0.3052 - 32096 B 0.59
Tiktoken_Encode Encode King(...)edy. [275] 4,898.7 ns 0.07 0.8011 0.0153 - 5056 B 0.09