Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

configure persistenced service to turn on persistence mode fails in Ubuntu 18.04 #66

Open
gabrik opened this issue Aug 31, 2022 · 1 comment
Open

configure persistenced service to turn on persistence mode fails in Ubuntu 18.04 #66

gabrik opened this issue Aug 31, 2022 · 1 comment

Comments

@gabrik
Copy link

gabrik commented Aug 31, 2022 •
edited
Loading

First, thanks for this very useful role.

I'm having an issue when trying to install the drivers on an Ubuntu 18.04 VM with PCIe passthrough (on plain KVM).

I ran the playbook like this:

ansible-playbook -i vm-inventory.yml -e host=test -e nvidia_driver_ubuntu_install_from_cuda_repo=yes -e nvidia_driver_persistence_mode_on=no nvidia-driver.yml

and everything seems fine, except that it still runs the configure persistenced service to turn on persistence task than then fails

TASK [nvidia.nvidia_driver : configure persistenced service to turn on persistence mode] ***************************************************************************************************************************************************************************************
fatal: [test]: FAILED! => {"changed": false, "checksum": "d9db4b73d83eca98ac2179a02847f1ab0a2cb866", "msg": "Source /root/.ansible/tmp/ansible-tmp-1661936608.0951219-393830-90866238334705/source not found"}

Below the verbose output:


TASK [nvidia.nvidia_driver : configure persistenced service to turn on persistence mode] ***************************************************************************************************************************************************************************************
task path: /home/ato/.ansible/roles/nvidia.nvidia_driver/tasks/main.yml:26
<test> CONNECT TO qemu+ssh://ato@majinbu/system
<test> FIND DOMAIN test
<test> ESTABLISH community.libvirt.libvirt_qemu CONNECTION
<test> EXEC /bin/sh -c 'echo ~ && sleep 0'
<test> GA send: {"execute": "guest-exec", "arguments": {"path": "/bin/sh", "capture-output": true, "arg": ["-c", "echo ~ && sleep 0"]}}
<test> GA return: {'return': {'pid': 31557}}
<test> GA send: {"execute": "guest-exec-status", "arguments": {"pid": 31557}}
<test> GA return: {'return': {'exited': False}}
<test> GA return: {'return': {'exitcode': 0, 'out-data': 'fgo=', 'exited': True}}
<test> GA stdout: ~
<test> GA stderr:
<test> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo ~/.ansible/tmp `"&& mkdir "` echo ~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430 `" && echo ansible-tmp-1661936792.938617-393878-3538192051430="` echo ~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430 `" ) && sleep 0'
<test> GA send: {"execute": "guest-exec", "arguments": {"path": "/bin/sh", "capture-output": true, "arg": ["-c", "( umask 77 && mkdir -p \"` echo ~/.ansible/tmp `\"&& mkdir \"` echo ~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430 `\" && echo ansible-tmp-1661936792.938617-393878-3538192051430=\"` echo ~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430 `\" ) && sleep 0"]}}
<test> GA return: {'return': {'pid': 31559}}
<test> GA send: {"execute": "guest-exec-status", "arguments": {"pid": 31559}}
<test> GA return: {'return': {'exited': False}}
<test> GA return: {'return': {'exitcode': 0, 'out-data': 'YW5zaWJsZS10bXAtMTY2MTkzNjc5Mi45Mzg2MTctMzkzODc4LTM1MzgxOTIwNTE0MzA9fi8uYW5zaWJsZS90bXAvYW5zaWJsZS10bXAtMTY2MTkzNjc5Mi45Mzg2MTctMzkzODc4LTM1MzgxOTIwNTE0MzAK', 'exited': True}}
<test> GA stdout: ansible-tmp-1661936792.938617-393878-3538192051430=~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430
<test> GA stderr:
Using module file /home/ato/.local/lib/python3.8/site-packages/ansible/modules/stat.py
<test> PUT /home/ato/.ansible/tmp/ansible-local-3938357gq_ahvb/tmpx3vmg1rs TO ~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/AnsiballZ_stat.py
<test> GA send: {"execute": "guest-file-open", "arguments": {"path": "~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/AnsiballZ_stat.py", "mode": "wb+"}}
<test> GA return: {'return': 1088}
<test> GA send: {"execute": "guest-file-close", "arguments": {"handle": 1088}}
<test> GA return: {'return': {}}
<test> EXEC /bin/sh -c 'chmod u+x '"'"'~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/'"'"' '"'"'~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/AnsiballZ_stat.py'"'"' && sleep 0'
<test> GA send: {"execute": "guest-exec", "arguments": {"path": "/bin/sh", "capture-output": true, "arg": ["-c", "chmod u+x '~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/' '~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/AnsiballZ_stat.py' && sleep 0"]}}
<test> GA return: {'return': {'pid': 31567}}
<test> GA send: {"execute": "guest-exec-status", "arguments": {"pid": 31567}}
<test> GA return: {'return': {'exited': False}}
<test> GA return: {'return': {'exitcode': 0, 'exited': True}}
<test> GA stdout:
<test> GA stderr:
<test> EXEC /bin/sh -c '/usr/bin/python3.6 '"'"'~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/AnsiballZ_stat.py'"'"' && sleep 0'
<test> GA send: {"execute": "guest-exec", "arguments": {"path": "/bin/sh", "capture-output": true, "arg": ["-c", "/usr/bin/python3.6 '~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/AnsiballZ_stat.py' && sleep 0"]}}
<test> GA return: {'return': {'pid': 31570}}
<test> GA send: {"execute": "guest-exec-status", "arguments": {"pid": 31570}}
<test> GA return: {'return': {'exited': False}}
<test> GA return: {'return': {'exitcode': 0, 'out-data': 'CnsiY2hhbmdlZCI6IGZhbHNlLCAic3RhdCI6IHsiZXhpc3RzIjogZmFsc2V9LCAiaW52b2NhdGlvbiI6IHsibW9kdWxlX2FyZ3MiOiB7InBhdGgiOiAiL2V0Yy9zeXN0ZW1kL3N5c3RlbS9udmlkaWEtcGVyc2lzdGVuY2VkLnNlcnZpY2UuZC9vdmVycmlkZS5jb25mIiwgImZvbGxvdyI6IGZhbHNlLCAiZ2V0X2NoZWNrc3VtIjogdHJ1ZSwgImNoZWNrc3VtX2FsZ29yaXRobSI6ICJzaGExIiwgImdldF9tZDUiOiBmYWxzZSwgImdldF9taW1lIjogdHJ1ZSwgImdldF9hdHRyaWJ1dGVzIjogdHJ1ZX19fQo=', 'exited': True}}
<test> GA stdout:
{"changed": false, "stat": {"exists": false}, "invocation": {"module_args": {"path": "/etc/systemd/system/nvidia-persistenced.service.d/override.conf", "follow": false, "get_checksum": true, "checksum_algorithm": "sha1", "get_md5": false, "get_mime": true, "get_attributes": true}}}
<test> GA stderr:
<test> PUT /home/ato/.ansible/roles/nvidia.nvidia_driver/files/nvidia-persistenced-override.conf TO ~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/source
<test> GA send: {"execute": "guest-file-open", "arguments": {"path": "~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/source", "mode": "wb+"}}
<test> GA return: {'return': 1089}
<test> GA send: {"execute": "guest-file-close", "arguments": {"handle": 1089}}
<test> GA return: {'return': {}}
<test> EXEC /bin/sh -c 'chmod u+x '"'"'~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/'"'"' '"'"'~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/source'"'"' && sleep 0'
<test> GA send: {"execute": "guest-exec", "arguments": {"path": "/bin/sh", "capture-output": true, "arg": ["-c", "chmod u+x '~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/' '~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/source' && sleep 0"]}}
<test> GA return: {'return': {'pid': 31574}}
<test> GA send: {"execute": "guest-exec-status", "arguments": {"pid": 31574}}
<test> GA return: {'return': {'exited': False}}
<test> GA return: {'return': {'exitcode': 0, 'exited': True}}
<test> GA stdout:
<test> GA stderr:
Using module file /home/ato/.local/lib/python3.8/site-packages/ansible/modules/copy.py
<test> PUT /home/ato/.ansible/tmp/ansible-local-3938357gq_ahvb/tmp9ikfkhqz TO ~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/AnsiballZ_copy.py
<test> GA send: {"execute": "guest-file-open", "arguments": {"path": "~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/AnsiballZ_copy.py", "mode": "wb+"}}
<test> GA return: {'return': 1090}
<test> GA send: {"execute": "guest-file-close", "arguments": {"handle": 1090}}
<test> GA return: {'return': {}}
<test> EXEC /bin/sh -c 'chmod u+x '"'"'~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/'"'"' '"'"'~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/AnsiballZ_copy.py'"'"' && sleep 0'
<test> GA send: {"execute": "guest-exec", "arguments": {"path": "/bin/sh", "capture-output": true, "arg": ["-c", "chmod u+x '~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/' '~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/AnsiballZ_copy.py' && sleep 0"]}}
<test> GA return: {'return': {'pid': 31577}}
<test> GA send: {"execute": "guest-exec-status", "arguments": {"pid": 31577}}
<test> GA return: {'return': {'exited': False}}
<test> GA return: {'return': {'exitcode': 0, 'exited': True}}
<test> GA stdout:
<test> GA stderr:
<test> EXEC /bin/sh -c '/usr/bin/python3.6 '"'"'~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/AnsiballZ_copy.py'"'"' && sleep 0'
<test> GA send: {"execute": "guest-exec", "arguments": {"path": "/bin/sh", "capture-output": true, "arg": ["-c", "/usr/bin/python3.6 '~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/AnsiballZ_copy.py' && sleep 0"]}}
<test> GA return: {'return': {'pid': 31580}}
<test> GA send: {"execute": "guest-exec-status", "arguments": {"pid": 31580}}
<test> GA return: {'return': {'exited': False}}
<test> GA return: {'return': {'exitcode': 1, 'out-data': 'CnsiZmFpbGVkIjogdHJ1ZSwgIm1zZyI6ICJTb3VyY2UgL3Jvb3QvLmFuc2libGUvdG1wL2Fuc2libGUtdG1wLTE2NjE5MzY3OTIuOTM4NjE3LTM5Mzg3OC0zNTM4MTkyMDUxNDMwL3NvdXJjZSBub3QgZm91bmQiLCAiaW52b2NhdGlvbiI6IHsibW9kdWxlX2FyZ3MiOiB7InNyYyI6ICIvcm9vdC8uYW5zaWJsZS90bXAvYW5zaWJsZS10bXAtMTY2MTkzNjc5Mi45Mzg2MTctMzkzODc4LTM1MzgxOTIwNTE0MzAvc291cmNlIiwgImRlc3QiOiAiL2V0Yy9zeXN0ZW1kL3N5c3RlbS9udmlkaWEtcGVyc2lzdGVuY2VkLnNlcnZpY2UuZC9vdmVycmlkZS5jb25mIiwgIl9vcmlnaW5hbF9iYXNlbmFtZSI6ICJudmlkaWEtcGVyc2lzdGVuY2VkLW92ZXJyaWRlLmNvbmYiLCAiZm9sbG93IjogZmFsc2UsICJjaGVja3N1bSI6ICJkOWRiNGI3M2Q4M2VjYTk4YWMyMTc5YTAyODQ3ZjFhYjBhMmNiODY2IiwgImJhY2t1cCI6IGZhbHNlLCAiZm9yY2UiOiB0cnVlLCAidW5zYWZlX3dyaXRlcyI6IGZhbHNlLCAiY29udGVudCI6IG51bGwsICJ2YWxpZGF0ZSI6IG51bGwsICJkaXJlY3RvcnlfbW9kZSI6IG51bGwsICJyZW1vdGVfc3JjIjogbnVsbCwgImxvY2FsX2ZvbGxvdyI6IG51bGwsICJtb2RlIjogbnVsbCwgIm93bmVyIjogbnVsbCwgImdyb3VwIjogbnVsbCwgInNldXNlciI6IG51bGwsICJzZXJvbGUiOiBudWxsLCAic2VsZXZlbCI6IG51bGwsICJzZXR5cGUiOiBudWxsLCAiYXR0cmlidXRlcyI6IG51bGx9fX0K', 'exited': True}}
<test> GA stdout:
{"failed": true, "msg": "Source /root/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/source not found", "invocation": {"module_args": {"src": "/root/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/source", "dest": "/etc/systemd/system/nvidia-persistenced.service.d/override.conf", "_original_basename": "nvidia-persistenced-override.conf", "follow": false, "checksum": "d9db4b73d83eca98ac2179a02847f1ab0a2cb866", "backup": false, "force": true, "unsafe_writes": false, "content": null, "validate": null, "directory_mode": null, "remote_src": null, "local_follow": null, "mode": null, "owner": null, "group": null, "seuser": null, "serole": null, "selevel": null, "setype": null, "attributes": null}}}
<test> GA stderr:
<test> EXEC /bin/sh -c 'rm -f -r '"'"'~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/'"'"' > /dev/null 2>&1 && sleep 0'
<test> GA send: {"execute": "guest-exec", "arguments": {"path": "/bin/sh", "capture-output": true, "arg": ["-c", "rm -f -r '~/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/' > /dev/null 2>&1 && sleep 0"]}}
<test> GA return: {'return': {'pid': 31583}}
<test> GA send: {"execute": "guest-exec-status", "arguments": {"pid": 31583}}
<test> GA return: {'return': {'exited': False}}
<test> GA return: {'return': {'exitcode': 0, 'exited': True}}
<test> GA stdout:
<test> GA stderr:
fatal: [test]: FAILED! => {
    "changed": false,
    "checksum": "d9db4b73d83eca98ac2179a02847f1ab0a2cb866",
    "diff": [],
    "invocation": {
        "module_args": {
            "_original_basename": "nvidia-persistenced-override.conf",
            "attributes": null,
            "backup": false,
            "checksum": "d9db4b73d83eca98ac2179a02847f1ab0a2cb866",
            "content": null,
            "dest": "/etc/systemd/system/nvidia-persistenced.service.d/override.conf",
            "directory_mode": null,
            "follow": false,
            "force": true,
            "group": null,
            "local_follow": null,
            "mode": null,
            "owner": null,
            "remote_src": null,
            "selevel": null,
            "serole": null,
            "setype": null,
            "seuser": null,
            "src": "/root/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/source",
            "unsafe_writes": false,
            "validate": null
        }
    },
    "msg": "Source /root/.ansible/tmp/ansible-tmp-1661936792.938617-393878-3538192051430/source not found"
}

Any suggestion?
@gabrik
Copy link
Author

gabrik commented Aug 31, 2022

Running directly on the VM seems to work, strange.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@gabrik