New fully automated process to build/tag/release (GitHub) & publish (PyPi) versions

[javulticat]

Description

This is a complete rewrite of the old CD process used to create new versions. All that process did was build a package from a tag whenever a new one got pushed, then published it on PyPi.

Here are some of the differences found in the new process:

• It is now a full end-to-end automation of the entire versioning process.
• That includes tagging. This means the workflow no longer requires a tag to be pushed to trigger it. Instead, the workflow can be triggered by simply clicking a button in GitHub, and the workflow will automatically take care of creating a new tag at the current head of master.
• The workflow now runs within a protected environment, which, in addition to increasing the security of the workflow, also protects against new versions being created accidentally.
• Package building is no longer done manually, which was brittle and basic. Now, it builds packages using a specific action designed to do that, which also gives us additional features like automatically inspecting the packages for various problems after building them and uploading them as artifacts that then help enable automating the other steps. This also greatly simplifies the build job in the workflow, making it much more readable and maintainable.
• Tagging and building are done in parallel to allow the workflow to run more quickly. But, if either fails, the workflow will not continue.
• Otherwise, once tagging and building are complete, the workflow will now automatically create a GitHub Release from the new tag, and it also attaches the newly built packages to the Release.
• Once the GitHub Release is created successfully, the packages are published as a new version on PyPi. The PyPi publishing process has also been changed. It now uses OICD authentication within a secure publishing environment to follow the latest best practices for securely publishing packages to PyPi. Also, 2FA has been enabled on the PyPi side, which they began requiring for all publishers at the start of 2024.
• This entire process can be dry-run! Since the CD process has been completely rewritten and does a lot more now, dry-runs will be useful to help ensure new versions will look like we expect them to before we create them for real. Dry-runs prevent creating a new tag, but shows debug output of what the result of the tagging process would have been. They also create the GitHub Releases in the draft state, which means they will only be visible to the Zappa team, but otherwise look just like the regular Releases would. Finally, they just avoid running the PyPi publishing job all together, since it does not have a dry-run mode.

It's intended that this new process will first be used "in the wild" to create version 0.59.0!

Related

Closes #1148
Closes #1319

[coveralls]

coverage: 74.831%. remained the same
when pulling ea62f93 on jav/update-cd
into 2efe244 on master.