This directory has the 'magic' for the depot_tools
windows binary update
mechanisms.
A previous Python may actually be in use when it is run, preventing us from replacing it outright without breaking running code. To ommodate this, and Python cleanup, we handle Python in two stages:
- Use CIPD to install both Git and Python at once.
- Use "win_tools.py" as a post-processor to install generated files and fix-ups.
- Python 2 and 3 (https://www.python.org/)
- Git for Windows (https://git-for-windows.github.io/)
Any time a user runs gclient
on windows, it will invoke the depot_tools
autoupdate script depot_tools.bat. This, in
turn, will run win_tools.bat, which does the bulk of the
work.
win_tools.bat
will successively look to see if the local version of the binary
package is present, and if so, if it's the expected version. If either of those
cases is not true, it will download and unpack the respective binary.
Installation of Git and Python is done by the win_tools.bat
script, which uses CIPD (via the cipd bootstrap) to acquire and
install each package into the root of the depot_tools
repository. Afterwards,
the win_tools.py Python script is invoked to install stubs,
wrappers, and support scripts into depot_tools
for end-users.
The Git and Python versions are specified in manifest.txt.
There is an associated file,
manifest_bleeding_edge.txt, that can be used
to canary new versions on select bots. Any bots with a .bleeding_edge
file
in their depot_tools
root will automatically use the bleeding edge manifest.
This allows opt-in systems to test against new versions of Python or Git. Once
those versions have been verified correct, manifest.txt
can be updated to the
same specification, which will cause the remainder of systems to update.
Git and Python bundle construction is documented in infra packaging.
Note that in order for the update to take effect, gclient
currently needs to
run twice. The first time it will update the depot_tools
repo, and the second
time it will see the new git version and update to it. This is a bug that should
be fixed, in case you're reading this and this paragraph infuriates you more
than the rest of this README.
After any modification to this script set, a test sequence should be run on a Windows bot.
The post-processing will regenerate "python.bat" and "python3.bat" to point to the current Python instance. Any previous Python installations will stick around, but new invocations will use the new instance. Old installations will die off either due to processes terminating or systems restarting. When this happens, they will be cleaned up by the post-processing script.
For each of the following test scenarios, run these commands and verify that they are working:
:: Assert that `gclient` invocation will update (and do the update).
gclient version
:: Assert that Python fundamentally works.
python -c "import Queue; print dir(Queue)"
:: Assert that Python 3 fundamentally works.
python3 -c "import queue; print(dir(queue))"
:: Assert that Python scripts work from `cmd.exe`.
git map-branches
:: Assert that `git bash` works.
git bash
## (Within `git bash`) assert that Python fundamentally works.
python -c "import Queue; print dir(Queue)"
## (Within `git bash`) assert that Python 3 fundamentally works.
python3 -c "import queue; print(dir(queue))"
## (Within `git bash`) assert that Python scripts work.
git map-branches
Run this sequence through the following upgrade/downgrade procedures:
- Cold default installation.
- Clean
depot_tools
via:git clean -x -f -d .
- Run through test steps.
- Test upgrade to bleeding edge (if it differs).
- Run
python.bat
in another shell, keep it open - Run
python3.bat
in another shell, keep it open - Add
.bleeding_edge
todepot_tools
root. - Run through test steps.
- In the old
python.bat
shell, runimport Queue
, confirm that it works. - In the old
python3.bat
shell, runimport queue
, confirm that it works. - Close the Python shell, run
gclient version
, ensure that old directory is cleaned.
- Run
- Clean
- Cold bleeding edge installation.
- Clean
depot_tools
via:git clean -x -f -d .
- Add
.bleeding_edge
todepot_tools
root. - Run through test steps.
- Test downgrade to default (if it differs).
- Run
python.bat
in another shell, keep it open - Run
python3.bat
in another shell, keep it open - Delete
.bleeding_edge
fromdepot_tools
root. - Run through test steps.
- In the old
python.bat
shell, runimport Queue
, confirm that it works. - In the old
python3.bat
shell, runimport queue
, confirm that it works. - Close the Python shell, run
gclient version
, ensure that old directory is cleaned.
- Run
- Clean
- Warm bleeding edge upgrade.
- Clean
depot_tools
via:git clean -x -f -d .
- Run
gclient version
to load defaults. - Run
python.bat
in another shell, keep it open - Run
python3.bat
in another shell, keep it open - Add
.bleeding_edge
todepot_tools
root. - Run through test steps.
- In the old
python.bat
shell, runimport Queue
, confirm that it works. - In the old
python3.bat
shell, runimport queue
, confirm that it works. - Close the Python shell, run
gclient version
, ensure that old directory is cleaned.
- Clean
- Upgradable and Revertible.
- Checkout current
HEAD
. - Run
gclient version
to load HEAD toolchain (initial). - Apply the candidate patch.
- Run through test steps (upgrade).
- Checkout current
HEAD
again. - Run
gclient version
to load HEAD toolchain (revert). - Run through test steps.
- Checkout current
This will take some time, but will ensure that all affected bots and users should not encounter any problems due to the change. As systems and users are migrated off of this implicit bootstrap, the testing procedure will become less critical.