[16.0] fs_storage: support SSH private keys authentication

[sebalix]

SSH connections can now be done with private keys by setting the pkey+ passphrase options. Coupled with the eval_options_from_env this allows to set these ones from the environment, e.g:

{"host": "sftp.example.net", "username": "odoo", "pkey": "$SSH_KEY", "passphrase": "$SSH_PASSPHRASE", "port": 22}

[simahawk]

LG. Could we have some test cov?

[sebalix]

@simahawk what's the best approach here to test? Current module doesn't depend on paramiko, if this one is installed then fsspec can use it. To test the feature I would need to add paramiko as tests dependencies, is it OK?

Or should we put tests in a dedicated test_fs_storage_ssh module?

[sebalix]

I added an extra parameter pkey_type to force the key type (RSA, DSS, OPENSSH...).
It happens that some versions of ssh-keygen generates OPENSSH keys even with the -t rsa parameter if -m PEM is not set.
https://serverfault.com/questions/939909/ssh-keygen-does-not-create-rsa-private-key

Still need to test it a bit more.

EDIT: this doesn't work neither. Better to generate the key in a proper manner with ssh-keygen -t rsa -m PEM [...], and everything works as expected. I reverted my last changes.

[simahawk]

@simahawk what's the best approach here to test? Current module doesn't depend on paramiko, if this one is installed then fsspec can use it. To test the feature I would need to add paramiko as tests dependencies, is it OK?

Or should we put tests in a dedicated test_fs_storage_ssh module?

In fact, we are adding too many SFTP related feature to the base module... IMO we need a separated module for sftp: fs_storage_sftp.

[github-actions]

There hasn't been any activity on this pull request in the past 4 months, so it has been marked as stale and it will be closed automatically if no further activity occurs in the next 30 days.
If you want this PR to never become stale, please ask a PSC member to apply the "no stale" label.

[florian-dacosta]

@sebalix Do you plan to continue and create a new module with the feature ?

FYI, I did manage a SFTP connection with a private key, using sshfs (https://github.com/fsspec/sshfs). If you install this lib, it will replace sftp default paramiko implementation by another based on asyncssh.
This allow to manage keys out of the box (at least, it works in my case, without any additional code).

IMO, we could drop this and maybe write somewhere to install sshfs if you want to manage keys, or do you see any reason not to use the sshfs implementation ?

[lmignon]

ping @sebalix @simahawk Do you've seen the last comment from @florian-dacosta? Can we close-it?