milatools setup for Windows

[stepgazaille]

Hello,
I've been having some issues getting VS code's debugger to run on a compute node from my Windows laptop.

How to reproduce the issue:

1. Install Python 3 on Windows
2. Execute the following commands in a Windows PowerShell terminal:

pip install milatools
mila init

At least in my case, the mila init command hangs after displaying the Your public key does not appear be registered on the cluster. Register it? [Y/n] message.

The workaround:
First you need to successfully run mila init to complete your cluster SSH setup.

1. Install Linux on Windows with WSL
2. Install Ubuntu on Windows
3. Launch an Ubuntu terminal from the Windows Start menu (on first launch this will trigger an OS user account setup procedure; proceed with the setup)
4. Install Anaconda in the Ubuntu app:
   4.1 Download the Anaconda linux setup script from here. For example: wget https://repo.anaconda.com/archive/Anaconda3-2022.05-Linux-x86_64.sh
   4.2 Install Anaconda. For example: bash Anaconda3-2022.05-Linux-x86_64.sh
   Answer yes to the following question: Do you wish the installer to initialize Anaconda3 by running conda init? [yes|no]
   4.3 Launch a new Ubuntu terminal from Windows Start menu and confirm that the default Python’s version is now >= 3.8 (this is a milatools requirement) by executing the following command: python -V
5. Install and initialize milatools (still in Ubuntu):

pip install milatools
mila init

Then you need to generate a public RSA key on your Windows host and add it to the list of authorized keys on the cluster:

1. If not already done, install Python 3 on Windows
2. Open a Windows PowerShell terminal from the Windows Start menu and install milatools by executing the following command: pip install milatools
3. On Windows, installation of milatools will end on a warning such as: WARNING: The script mila.exe is installed in 'C:\Users\[USERNAME]\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\Scripts' which is not on PATH. Take in note the absolute path of milatools’ installation folder
4. Add the absolute path of milatools’ installation folder to Windows’ PATH environment variable by following this procedure
5. Open a new Windows PowerShell terminal from the Windows Start menu and execute the following command: mila init
   5.1 Due to a bug, the mila init command will hang after displaying the following message: Your public key does not appear be registered on the cluster. Register it? [Y/n]
6. Open a new PowerShell terminal and execute the following command to print your public RSA key: type $env:USERPROFILE\.ssh\id_rsa.pub
7. Use the following command to open an SSH connection to the mila cluster: ssh mila
8. Append your public RSA key (see step 6 above) at the end of the ~/.ssh/authorized_keys file on Mila’s cluster’s host

Setup is done!
Now you can use the following procedure to launch VS code using a compute node from Mila’s cluster as as development environment:

1. Install Microsoft’s Remote SSH VS Code plugin
2. On Mila login node, ask for an interactive session and keep a note of the compute node hostname. For example: salloc --cpus-per-task 1 --gres gpu:0 --time=0-1:00:00
3. Execute the following command in a Windows PowerShell to launch VS Code using the compute node from the previous step as as development environment: code --remote ssh-remote+[COMPUTE_NODE_HOSTNAME].server.mila.quebec [SRC_CODE_DIRECTORY_ON_CLUSTER]

Lead to a bugfix:
The problem seems to come from trying to execute the ssh-copy-id command on a Windows host (this command does not exist on Windows).

I found a one liner that is supposed to be Windows' equivalent of ssh-copy-id, but that also hanged on me and I haven't had time to investigate further yet.

I will try to find some bandwidth to come up with a bugfix PR in the next few days.
Hopefully the workaround above will save some people some time in the meanwhile.

[tianyu-z]

After the above steps, when I run:
mila code /home/mila/t/tianyu.zhang/ --alloc --gres=gpu:rtx8000:1 -c 6 --mem=32G -t 48:00:00 --partition=unkillable
I got this:

Traceback (most recent call last):
File "C:\ProgramData\Miniconda3\lib\site-packages\milatools\cli\commands.py", line 42, in main
auto_cli(milatools)
File "C:\ProgramData\Miniconda3\lib\site-packages\coleo\cli.py", line 656, in auto_cli
result = run_cli(entry, args, **kwargs)
File "C:\ProgramData\Miniconda3\lib\site-packages\coleo\cli.py", line 628, in run_cli
return call(opts=opts, args=args)
File "C:\ProgramData\Miniconda3\lib\site-packages\coleo\cli.py", line 587, in thunk
result = fn(*args)
File "C:\ProgramData\Miniconda3\lib\site-packages\milatools\cli\commands.py", line 308, in code
remote = Remote("mila")
File "C:\ProgramData\Miniconda3\lib\site-packages\milatools\cli\remote.py", line 83, in init
connection.open()
File "C:\ProgramData\Miniconda3\lib\site-packages\fabric\connection.py", line 636, in open
self.client.connect(**kwargs)
File "C:\ProgramData\Miniconda3\lib\site-packages\paramiko\client.py", line 459, in connect
self._auth(
File "C:\ProgramData\Miniconda3\lib\site-packages\paramiko\client.py", line 790, in _auth
raise saved_exception
File "C:\ProgramData\Miniconda3\lib\site-packages\paramiko\client.py", line 760, in _auth
key = self._key_from_filepath(
File "C:\ProgramData\Miniconda3\lib\site-packages\paramiko\client.py", line 612, in _key_from_filepath
key = klass.from_private_key_file(key_path, password)
File "C:\ProgramData\Miniconda3\lib\site-packages\paramiko\pkey.py", line 232, in from_private_key_file
key = cls(filename=filename, password=password)
File "C:\ProgramData\Miniconda3\lib\site-packages\paramiko\rsakey.py", line 63, in init
self._from_private_key_file(filename, password)
File "C:\ProgramData\Miniconda3\lib\site-packages\paramiko\rsakey.py", line 186, in _from_private_key_file
data = self._read_private_key_file("RSA", filename, password)
File "C:\ProgramData\Miniconda3\lib\site-packages\paramiko\pkey.py", line 305, in _read_private_key_file
data = self._read_private_key(tag, f, password)
File "C:\ProgramData\Miniconda3\lib\site-packages\paramiko\pkey.py", line 336, in _read_private_key
data = self._read_private_key_openssh(lines[start:end], password)
File "C:\ProgramData\Miniconda3\lib\site-packages\paramiko\pkey.py", line 437, in _read_private_key_openssh
raise PasswordRequiredException(
paramiko.ssh_exception.PasswordRequiredException: private key file is encrypted

An error occured during the execution of the command code. Please try updating milatools by running
pip install milatools --upgrade
in the terminal. If the issue persists, consider filling a bug report at https://github.com/mila-iqia/milatools/issues/new?labels=code%2C0.0.16&template=bug_report.md&title=%5Bv0.0.16%5D+Issue+running+the+command+%60mila+code%60

Is there any recommendation to solve this? Thank you!

[tianyu-z]

Same thing happens when I try to WSL2 in Win11

(base) tiany@Workspace:~$ mila code /home/mila/t/tianyu.zhang
Traceback (most recent call last):
File "/home/tiany/miniconda3/lib/python3.9/site-packages/milatools/cli/commands.py", line 42, in main
auto_cli(milatools)
File "/home/tiany/miniconda3/lib/python3.9/site-packages/coleo/cli.py", line 656, in auto_cli
result = run_cli(entry, args, **kwargs)
File "/home/tiany/miniconda3/lib/python3.9/site-packages/coleo/cli.py", line 628, in run_cli
return call(opts=opts, args=args)
File "/home/tiany/miniconda3/lib/python3.9/site-packages/coleo/cli.py", line 587, in thunk
result = fn(*args)
File "/home/tiany/miniconda3/lib/python3.9/site-packages/milatools/cli/commands.py", line 308, in code
remote = Remote("mila")
File "/home/tiany/miniconda3/lib/python3.9/site-packages/milatools/cli/remote.py", line 83, in init
connection.open()
File "/home/tiany/miniconda3/lib/python3.9/site-packages/fabric/connection.py", line 636, in open
self.client.connect(**kwargs)
File "/home/tiany/miniconda3/lib/python3.9/site-packages/paramiko/client.py", line 459, in connect
self._auth(
File "/home/tiany/miniconda3/lib/python3.9/site-packages/paramiko/client.py", line 790, in _auth
raise saved_exception
File "/home/tiany/miniconda3/lib/python3.9/site-packages/paramiko/client.py", line 760, in _auth
key = self._key_from_filepath(
File "/home/tiany/miniconda3/lib/python3.9/site-packages/paramiko/client.py", line 612, in _key_from_filepath
key = klass.from_private_key_file(key_path, password)
File "/home/tiany/miniconda3/lib/python3.9/site-packages/paramiko/pkey.py", line 232, in from_private_key_file
key = cls(filename=filename, password=password)
File "/home/tiany/miniconda3/lib/python3.9/site-packages/paramiko/rsakey.py", line 63, in init
self._from_private_key_file(filename, password)
File "/home/tiany/miniconda3/lib/python3.9/site-packages/paramiko/rsakey.py", line 186, in _from_private_key_file
data = self._read_private_key_file("RSA", filename, password)
File "/home/tiany/miniconda3/lib/python3.9/site-packages/paramiko/pkey.py", line 305, in _read_private_key_file
data = self._read_private_key(tag, f, password)
File "/home/tiany/miniconda3/lib/python3.9/site-packages/paramiko/pkey.py", line 336, in _read_private_key
data = self._read_private_key_openssh(lines[start:end], password)
File "/home/tiany/miniconda3/lib/python3.9/site-packages/paramiko/pkey.py", line 437, in _read_private_key_openssh
raise PasswordRequiredException(
paramiko.ssh_exception.PasswordRequiredException: private key file is encrypted

[Basile-Terv]

Same thing happens when I try to WSL2 in Win11

(base) tiany@Workspace:~$ mila code /home/mila/t/tianyu.zhang
Traceback (most recent call last):
File "/home/tiany/miniconda3/lib/python3.9/site-packages/milatools/cli/commands.py", line 42, in main
auto_cli(milatools)
File "/home/tiany/miniconda3/lib/python3.9/site-packages/coleo/cli.py", line 656, in auto_cli
result = run_cli(entry, args, **kwargs)
File "/home/tiany/miniconda3/lib/python3.9/site-packages/coleo/cli.py", line 628, in run_cli
return call(opts=opts, args=args)
File "/home/tiany/miniconda3/lib/python3.9/site-packages/coleo/cli.py", line 587, in thunk
result = fn(*args)
File "/home/tiany/miniconda3/lib/python3.9/site-packages/milatools/cli/commands.py", line 308, in code
remote = Remote("mila")
File "/home/tiany/miniconda3/lib/python3.9/site-packages/milatools/cli/remote.py", line 83, in init
connection.open()
File "/home/tiany/miniconda3/lib/python3.9/site-packages/fabric/connection.py", line 636, in open
self.client.connect(**kwargs)
File "/home/tiany/miniconda3/lib/python3.9/site-packages/paramiko/client.py", line 459, in connect
self._auth(
File "/home/tiany/miniconda3/lib/python3.9/site-packages/paramiko/client.py", line 790, in _auth
raise saved_exception
File "/home/tiany/miniconda3/lib/python3.9/site-packages/paramiko/client.py", line 760, in _auth
key = self._key_from_filepath(
File "/home/tiany/miniconda3/lib/python3.9/site-packages/paramiko/client.py", line 612, in _key_from_filepath
key = klass.from_private_key_file(key_path, password)
File "/home/tiany/miniconda3/lib/python3.9/site-packages/paramiko/pkey.py", line 232, in from_private_key_file
key = cls(filename=filename, password=password)
File "/home/tiany/miniconda3/lib/python3.9/site-packages/paramiko/rsakey.py", line 63, in init
self._from_private_key_file(filename, password)
File "/home/tiany/miniconda3/lib/python3.9/site-packages/paramiko/rsakey.py", line 186, in _from_private_key_file
data = self._read_private_key_file("RSA", filename, password)
File "/home/tiany/miniconda3/lib/python3.9/site-packages/paramiko/pkey.py", line 305, in _read_private_key_file
data = self._read_private_key(tag, f, password)
File "/home/tiany/miniconda3/lib/python3.9/site-packages/paramiko/pkey.py", line 336, in _read_private_key
data = self._read_private_key_openssh(lines[start:end], password)
File "/home/tiany/miniconda3/lib/python3.9/site-packages/paramiko/pkey.py", line 437, in _read_private_key_openssh
raise PasswordRequiredException(
paramiko.ssh_exception.PasswordRequiredException: private key file is encrypted

Hello, I have the same issue, have you been able to solve it?

[lebrice]

For anyone seeing this, this should now be solved with #65