Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add python package size delta report feature for SMD images #363

Merged
merged 20 commits into from
Apr 18, 2024

Conversation

aws-tianquaw
Copy link
Contributor

Issue #, if available: https://app.asana.com/0/1206816154076403/1206846565561145/f

Description of changes:

Add python package size delta report feature for SMD images. The report will include the following sections:

  1. Absolute and relative change in total image size
  2. Top-20 largest package in current image, descending sorted
  3. Absolute and relative change in size of each changed/new package, descending sorted

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@aws-tianquaw
Copy link
Contributor Author

aws-tianquaw commented Apr 4, 2024

Sharing a sample package size report for SMD release 1.6.2 based on 1.6.1

Python Package Size Report (GPU)

Target Image Version: 1.6.2 | Base Image Version: 1.6.1

Python Packages Total Size Summary

Target Version Total Size Base Version Total Size Size Change (abs) Size Change (%)
2.84GB 2.84GB 387.41KB 0.01

Top-20 Largest Python Packages

Package Version in the Target Image Size
cudatoolkit 11.8.0 682.45MB
cudnn 8.9.7.29 443.90MB
pytorch 2.0.0 403.50MB
libmagma 2.7.1 201.22MB
mkl 2022.2.1 157.26MB
nccl 2.20.5.1 120.99MB
libxgboost 1.7.6 99.28MB
qt 5.15.8 57.94MB
llvm 18.1.2 57.57MB
catboost 1.2.3 45.71MB
jaxlib 0.4.14 45.33MB
pillow 9.5.0 43.98MB
libgoogle 2.12.0 43.43MB
libllvm15 15.0.7 31.78MB
libllvm14 14.0.6 30.03MB
libarrow 12.0.1 26.43MB
pandoc 3.1.12.3 20.03MB
nodejs 18.19.0 15.18MB
scipy 1.11.4 14.55MB
ucx 1.14.1 14.51MB

Python Package Size Delta

Package Version in the Target Image Version in the Base Image Size Change (abs) Size Change (%)
llvm 18.1.2 18.1.1 802.48KB 1.38
amazon 3.0.7 0.1.2 251.98KB 777.69
libxkbcommon 1.7.0 1.6.0 18.23KB 3.25
huggingface_hub 0.22.0 0.21.4 16.34KB 7.4
importlib_resources 6.4.0 6.3.2 2.03KB 6.7
libmamba 1.5.8 1.5.7 1.96KB 0.12
typer 0.9.4 0.9.0 1.16KB 1.5
rope 1.13.0 1.12.0 1.01KB 0.69
nbconvert 7.16.3 7.16.2 976.00B 13.41
libmambapy 1.5.8 1.5.7 632.00B 0.2
wheel 0.43.0 0.42.0 495.00B 0.86
lightning 0.11.1 0.11.0 358.00B 1.33
threadpoolctl 3.4.0 3.3.0 325.00B 1.43
libaec 1.1.3 1.1.2 218.00B 0.62
thrift 0.20.0 0.19.0 145.00B 0.11
filelock 3.13.3 3.13.1 6.00B 0.04
amazon_sagemaker_sql_editor 0.1.4 0.1.3 -466.00B -0.15
requests 2.0.0 2.31.0 -30.23KB -54.6
langchain 0.1.33 0.0.28 -685.37KB -79.86

Python Package Size Report (CPU)

Target Image Version: 1.6.2 | Base Image Version: 1.6.1

Python Packages Total Size Summary

Target Version Total Size Base Version Total Size Size Change (abs) Size Change (%)
1.12GB 1.12GB 387.41KB 0.03

Top-20 Largest Python Packages

Package Version in the Target Image Size
mkl 2022.2.1 157.26MB
tensorflow 2.12.1 127.47MB
pytorch 2.0.0 67.37MB
qt 5.15.8 57.94MB
llvm 18.1.2 57.57MB
catboost 1.2.3 45.71MB
jaxlib 0.4.14 45.33MB
pillow 9.5.0 43.98MB
libgoogle 2.12.0 43.43MB
libllvm15 15.0.7 31.78MB
libllvm14 14.0.6 30.03MB
libarrow 12.0.1 26.43MB
pandoc 3.1.12.3 20.03MB
nodejs 18.19.0 15.18MB
scipy 1.11.4 14.55MB
ucx 1.14.1 14.51MB
pandas 2.1.4 11.84MB
scikit 0.19.3 11.60MB
icu 73.2 11.53MB
statsmodels 0.14.1 10.50MB

Python Package Size Delta

Package Version in the Target Image Version in the Base Image Size Change (abs) Size Change (%)
llvm 18.1.2 18.1.1 802.48KB 1.38
amazon 3.0.7 0.1.2 251.98KB 777.69
libxkbcommon 1.7.0 1.6.0 18.23KB 3.25
huggingface_hub 0.22.0 0.21.4 16.34KB 7.4
importlib_resources 6.4.0 6.3.2 2.03KB 6.7
libmamba 1.5.8 1.5.7 1.96KB 0.12
typer 0.9.4 0.9.0 1.16KB 1.5
rope 1.13.0 1.12.0 1.01KB 0.69
nbconvert 7.16.3 7.16.2 976.00B 13.41
libmambapy 1.5.8 1.5.7 632.00B 0.2
wheel 0.43.0 0.42.0 495.00B 0.86
lightning 0.11.1 0.11.0 358.00B 1.33
threadpoolctl 3.4.0 3.3.0 325.00B 1.43
libaec 1.1.3 1.1.2 218.00B 0.62
thrift 0.20.0 0.19.0 145.00B 0.11
filelock 3.13.3 3.13.1 6.00B 0.04
amazon_sagemaker_sql_editor 0.1.4 0.1.3 -466.00B -0.15
requests 2.0.0 2.31.0 -30.23KB -54.6
langchain 0.1.33 0.0.28 -685.37KB -79.86

src/package_report.py Outdated Show resolved Hide resolved
src/main.py Outdated Show resolved Hide resolved
@aws-tianquaw
Copy link
Contributor Author

balajisankar15
balajisankar15 previously approved these changes Apr 12, 2024
@aws-tianquaw aws-tianquaw merged commit 9d6813d into aws:main Apr 18, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants