Updated fetch_shas utility to use the channel manifest tomls

[UebelAndre]

Updating shas was taking much longer than it used to. I've refactored the shell script into python so to manage increased complexity. The script now fetches the channel-rust-*.toml files which contains additional sha256 values. This script still falls back to brute force fetching of artifacts to account for any that are missing. The runtime was reduced from ~20 min to ~80s.

python3.11 ./util/fetch_shas/fetch_shas.py

2024-09-06 08:57:22 - INFO - Fetching known sha256 data...
2024-09-06 08:57:22 - INFO - Downloading data...
2024-09-06 08:57:29 - INFO - Done.
2024-09-06 08:57:29 - INFO - Deserializing 119 tomls...
2024-09-06 08:57:36 - INFO - Done.
2024-09-06 08:57:36 - INFO - Parsing artifacts...
2024-09-06 08:57:36 - INFO - Done. Identified 52790 artifacts.
2024-09-06 08:57:36 - INFO - Checking for 81514 missing artifacts...
2024-09-06 08:58:26 - INFO - Done.
2024-09-06 08:58:29 - INFO - Downloading 404 missing artifacts...
2024-09-06 08:58:29 - INFO - Done.
2024-09-06 08:58:36 - INFO - Done. Wrote rust/known_shas.bzl

Note that beta sha256 values have been removed in this change.

[UebelAndre]

This could probably have been written in Rust but at the time this seemed the most expedient thing to do and closer to the previous implementation.

[illicitonion]

Thanks for fixing this all up! 🎉 🙏

I wonder if at some point we should remove versions before our MSRV from this list to speed up running the script? 🤷 We currently claim to support 1.72.1 and newer, so 64 of the 76 versions we currently fetch we claim not to support...

[UebelAndre]

Thanks for fixing this all up! 🎉 🙏

I wonder if at some point we should remove versions before our MSRV from this list to speed up running the script? 🤷 We currently claim to support 1.72.1 and newer, so 64 of the 76 versions we currently fetch we claim not to support...

I'm down for removing some of the older things. Now that users are notified when they should add the shas (#2593), I can open a PR for that.

[UebelAndre]

Thanks for fixing this all up! 🎉 🙏
I wonder if at some point we should remove versions before our MSRV from this list to speed up running the script? 🤷 We currently claim to support 1.72.1 and newer, so 64 of the 76 versions we currently fetch we claim not to support...

I'm down for removing some of the older things. Now that users are notified when they should add the shas (#2593), I can open a PR for that.

#2850