Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix enums performance #1306

Open
Tracked by #1315
bonjourmauko opened this issue Nov 21, 2024 · 0 comments · May be fixed by #1307
Open
Tracked by #1315

Fix enums performance #1306

bonjourmauko opened this issue Nov 21, 2024 · 0 comments · May be fixed by #1307
Assignees
@bonjourmauko
Labels
kind:perf A code change that improves performance

Comments

@bonjourmauko
Copy link
Member

Hi there!

I really enjoy OpenFisca, but I recently encountered an issue.

Here is what I did:

pytest openfisca_core/indexed_enums --benchmark-only --benchmark-compare -m benchmark

Here is what I expected to happen:

------------------------------------------------------------------------------------ benchmark 'Enum.__eq__': 2 tests ------------------------------------------------------------------------------------
Name (time in us)                            Min               Max              Mean            StdDev            Median               IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_enum_eq (0001_e8b0acf)     6.1164 (1.0)      6.2392 (1.0)      6.1712 (1.0)      0.0236 (1.0)      6.1702 (1.0)      0.0284 (1.13)         30;2      162.0429 (1.0)         100       10000
test_benchmark_enum_eq (NOW)              6.1806 (1.01)     6.3476 (1.02)     6.2902 (1.02)     0.0286 (1.21)     6.2958 (1.02)     0.0252 (1.0)          14;7      158.9779 (0.98)        100       10000
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'Enum.encode (Enum)': 2 tests -----------------------------------------------------------------------------------
Name (time in ms)                                     Min               Max              Mean            StdDev            Median               IQR            Outliers       OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_enum_encode_enum (NOW)              1.8063 (1.0)      2.0201 (1.0)      1.8426 (1.0)      0.0409 (1.0)      1.8268 (1.0)      0.0383 (1.0)          13;7  542.7000 (1.0)         100          10
test_benchmark_enum_encode_enum (0001_e8b0acf)     1.9466 (1.08)     2.4946 (1.23)     2.0038 (1.09)     0.0812 (1.98)     1.9726 (1.08)     0.0592 (1.55)        15;11  499.0517 (0.92)        100          10
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------- benchmark 'Enum.encode (int)': 2 tests --------------------------------------------------------------------------------------------------
Name (time in ns)                                         Min                    Max                   Mean                StdDev                 Median                 IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_enum_encode_int (0001_e8b0acf)        600.0000 (1.0)       4,333.4001 (1.0)         660.3360 (1.0)        371.1875 (1.0)         620.9000 (1.0)       12.4500 (1.0)           1;3    1,514.3805 (1.0)         100          10
test_benchmark_enum_encode_int (NOW)              36,487.4999 (60.81)    51,941.5989 (11.99)    36,839.4600 (55.79)    1,554.0287 (4.19)     36,554.2001 (58.87)    229.0995 (18.40)        1;12       27.1448 (0.02)        100          10
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'Enum.encode (str)': 2 tests -----------------------------------------------------------------------------------
Name (time in ms)                                    Min               Max              Mean            StdDev            Median               IQR            Outliers       OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_enum_encode_str (0001_e8b0acf)     1.1514 (1.0)      5.4571 (2.05)     1.3338 (1.0)      0.5570 (4.93)     1.1804 (1.0)      0.0383 (1.15)         6;16  749.7253 (1.0)         100          10
test_benchmark_enum_encode_str (NOW)              1.7690 (1.54)     2.6589 (1.0)      1.8262 (1.37)     0.1130 (1.0)      1.7950 (1.52)     0.0332 (1.0)           7;9  547.5705 (0.73)        100          10
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'EnumArray.__eq__': 2 tests -------------------------------------------------------------------------------------
Name (time in us)                                  Min               Max              Mean            StdDev            Median               IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_enum_array_eq (NOW)              6.4008 (1.0)      7.0462 (1.0)      6.4869 (1.0)      0.0953 (1.0)      6.4546 (1.0)      0.0511 (1.05)        12;15      154.1559 (1.0)         100         100
test_benchmark_enum_array_eq (0001_e8b0acf)     8.8654 (1.39)     9.3842 (1.33)     8.9354 (1.38)     0.1009 (1.06)     8.8969 (1.38)     0.0487 (1.0)         10;15      111.9138 (0.73)        100         100
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------- benchmark 'EnumArray.decode': 2 tests --------------------------------------------------------------------------------------------
Name (time in us)                                        Min                   Max                Mean             StdDev              Median               IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_enum_array_decode (NOW)              241.8246 (1.0)        254.4829 (1.0)      243.7532 (1.0)       2.0537 (1.0)      243.1275 (1.0)      1.4817 (1.0)           9;8        4.1025 (1.0)         100         100
test_benchmark_enum_array_decode (0001_e8b0acf)     752.3354 (3.11)     1,501.3621 (5.90)     765.7738 (3.14)     74.5790 (36.31)    757.6515 (3.12)     4.3040 (2.90)          1;3        1.3059 (0.32)        100         100
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------- benchmark 'EnumArray.decode_to_str': 2 tests ------------------------------------------------------------------------------------------
Name (time in us)                                               Min                 Max                Mean            StdDev              Median               IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_enum_array_decode_to_str (NOW)              313.1738 (1.0)      321.3071 (1.0)      317.8362 (1.0)      0.9421 (1.0)      317.7827 (1.0)      0.9496 (1.0)          12;5        3.1463 (1.0)         100         100
test_benchmark_enum_array_decode_to_str (0001_e8b0acf)     861.6308 (2.75)     916.3904 (2.85)     871.6658 (2.74)     9.9436 (10.56)    868.3410 (2.73)     6.7525 (7.11)        12;11        1.1472 (0.36)        100         100
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Here is what actually happened:

------------------------------------------------------------------------------------ benchmark 'Enum.__eq__': 2 tests ------------------------------------------------------------------------------------
Name (time in us)                            Min               Max              Mean            StdDev            Median               IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_enum_eq (0001_e8b0acf)     6.1164 (1.0)      6.2392 (1.0)      6.1712 (1.0)      0.0236 (1.0)      6.1702 (1.0)      0.0284 (1.0)          30;2      162.0429 (1.0)         100       10000
test_benchmark_enum_eq (NOW)              6.1250 (1.00)     7.5333 (1.21)     6.2292 (1.01)     0.1464 (6.21)     6.2031 (1.01)     0.0381 (1.34)          5;9      160.5342 (0.99)        100       10000
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------- benchmark 'Enum.encode (Enum)': 2 tests -----------------------------------------------------------------------------------
Name (time in ms)                                     Min               Max              Mean            StdDev            Median               IQR            Outliers       OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_enum_encode_enum (NOW)              1.8037 (1.0)      1.8551 (1.0)      1.8197 (1.0)      0.0080 (1.0)      1.8193 (1.0)      0.0091 (1.0)          25;2  549.5479 (1.0)         100          10
test_benchmark_enum_encode_enum (0001_e8b0acf)     1.9466 (1.08)     2.4946 (1.34)     2.0038 (1.10)     0.0812 (10.15)    1.9726 (1.08)     0.0592 (6.50)        15;11  499.0517 (0.91)        100          10
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------- benchmark 'Enum.encode (int)': 2 tests -------------------------------------------------------------------------------------------------------------
Name (time in ns)                                             Min                        Max                       Mean                  StdDev                     Median                    IQR            Outliers             OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_enum_encode_int (0001_e8b0acf)            600.0000 (1.0)           4,333.4001 (1.0)             660.3360 (1.0)          371.1875 (1.0)             620.9000 (1.0)          12.4500 (1.0)           1;3  1,514,380.5499 (1.0)         100          10
test_benchmark_enum_encode_int (NOW)              12,811,937.5004 (>1000.0)  14,549,374.9995 (>1000.0)  12,934,624.5040 (>1000.0)  249,848.9227 (673.11)   12,870,975.0497 (>1000.0)  82,920.8489 (>1000.0)       6;7         77.3119 (0.00)        100          10
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------- benchmark 'Enum.encode (str)': 2 tests -------------------------------------------------------------------------------------
Name (time in ms)                                     Min                Max               Mean            StdDev             Median               IQR            Outliers       OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_enum_encode_str (0001_e8b0acf)      1.1514 (1.0)       5.4571 (1.0)       1.3338 (1.0)      0.5570 (3.94)      1.1804 (1.0)      0.0383 (1.0)          6;16  749.7253 (1.0)         100          10
test_benchmark_enum_encode_str (NOW)              21.4503 (18.63)    22.0977 (4.05)     21.6755 (16.25)    0.1413 (1.0)      21.6555 (18.35)    0.1709 (4.47)         25;5   46.1349 (0.06)        100          10
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------ benchmark 'EnumArray.__eq__': 2 tests -------------------------------------------------------------------------------------
Name (time in us)                                  Min               Max              Mean            StdDev            Median               IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_enum_array_eq (NOW)              6.3737 (1.0)      7.0063 (1.0)      6.4517 (1.0)      0.0893 (1.0)      6.4165 (1.0)      0.0704 (1.44)         13;8      154.9990 (1.0)         100         100
test_benchmark_enum_array_eq (0001_e8b0acf)     8.8654 (1.39)     9.3842 (1.34)     8.9354 (1.38)     0.1009 (1.13)     8.8969 (1.39)     0.0487 (1.0)         10;15      111.9138 (0.72)        100         100
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------- benchmark 'EnumArray.decode': 2 tests --------------------------------------------------------------------------------------------
Name (time in us)                                        Min                   Max                Mean             StdDev              Median               IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_enum_array_decode (NOW)              239.5858 (1.0)        253.0062 (1.0)      240.5797 (1.0)       1.4397 (1.0)      240.3013 (1.0)      0.7865 (1.0)           5;5        4.1566 (1.0)         100         100
test_benchmark_enum_array_decode (0001_e8b0acf)     752.3354 (3.14)     1,501.3621 (5.93)     765.7738 (3.18)     74.5790 (51.80)    757.6515 (3.15)     4.3040 (5.47)          1;3        1.3059 (0.31)        100         100
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------- benchmark 'EnumArray.decode_to_str': 2 tests ------------------------------------------------------------------------------------------
Name (time in us)                                               Min                 Max                Mean            StdDev              Median               IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_enum_array_decode_to_str (NOW)              317.5279 (1.0)      333.5621 (1.0)      320.0601 (1.0)      3.0285 (1.0)      318.8650 (1.0)      2.2540 (1.0)         13;12        3.1244 (1.0)         100         100
test_benchmark_enum_array_decode_to_str (0001_e8b0acf)     861.6308 (2.71)     916.3904 (2.75)     871.6658 (2.72)     9.9436 (3.28)     868.3410 (2.72)     6.7525 (3.00)        12;11        1.1472 (0.37)        100         100
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Here is data (or links to it) that can help you reproduce this issue:

#1303

Context

I identify more as a:

  • Developer
@bonjourmauko bonjourmauko added the kind:perf A code change that improves performance label Nov 21, 2024
@bonjourmauko bonjourmauko self-assigned this Nov 21, 2024
@bonjourmauko bonjourmauko moved this from Inbox to Delivery in OpenFisca International Nov 21, 2024
bonjourmauko added a commit that referenced this issue Nov 21, 2024
@bonjourmauko
perf: fix enum performance (#1306)
e2e3fb3
@bonjourmauko bonjourmauko linked a pull request Nov 21, 2024 that will close this issue
Open
bonjourmauko added a commit that referenced this issue Nov 21, 2024
@bonjourmauko
chore: version bump (fixes #1306)
f7298f3
@bonjourmauko bonjourmauko mentioned this issue Nov 21, 2024
Open
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bonjourmauko bonjourmauko

Labels
kind:perf A code change that improves performance
Projects
OpenFisca International
Status: Delivery
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

perf: fix enum performance openfisca/openfisca-core
1 participant
@bonjourmauko