Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Uncaught IndexError exception #104

Open
namniav opened this issue Jan 20, 2024 · 0 comments
Open

Uncaught IndexError exception #104

namniav opened this issue Jan 20, 2024 · 0 comments

Comments

@namniav
Copy link

namniav commented Jan 20, 2024

Today I found this log:

/var/log/pyznap.log.1

[...]
Jan 01 00:00:01 INFO: Starting pyznap...
[...]
Jan 01 00:25:11 INFO: Deleting snapshot rpool/var/lib/docker/wbuu1gj1afus39gfy6arms4sx@pyznap_2023-12-21_02:11:01_monthly...
[...]

/var/log/pyznap.log

Jan 01 00:15:01 INFO: Starting pyznap...
[...]
Jan 01 00:25:11 INFO: Deleting snapshot rpool/var/lib/docker/wbuu1gj1afus39gfy6arms4sx@pyznap_2023-12-21_02:11:01_monthly...
Traceback (most recent call last):
  File "/opt/pyznap/venv/bin/pyznap", line 8, in <module>
    sys.exit(main())
  File "/opt/pyznap/venv/lib/python3.10/site-packages/pyznap/main.py", line 170, in main
    return _main()
  File "/opt/pyznap/venv/lib/python3.10/site-packages/pyznap/main.py", line 115, in _main
    clean_config(config)
  File "/opt/pyznap/venv/lib/python3.10/site-packages/pyznap/clean.py", line 165, in clean_config
    clean_filesystem(child, conf)
  File "/opt/pyznap/venv/lib/python3.10/site-packages/pyznap/clean.py", line 85, in clean_filesystem
    clean_snap(snap)
  File "/opt/pyznap/venv/lib/python3.10/site-packages/pyznap/clean.py", line 33, in clean_snap
    snap.destroy()
  File "/opt/pyznap/venv/lib/python3.10/site-packages/pyznap/pyzfs.py", line 249, in destroy
    check_output(cmd, ssh=self.ssh)
  File "/opt/pyznap/venv/lib/python3.10/site-packages/pyznap/process.py", line 105, in check_output
    ret.check_returncode()
  File "/opt/pyznap/venv/lib/python3.10/site-packages/pyznap/process.py", line 56, in check_returncode
    match = re.search(pattern, self.stderr.splitlines()[0])

It happened when two cron jobs was running at the same time. Before the first job managed to clean a lot of snapshots of docker containers/volumes which need long time, next job started deleting snapshots at the same time.

I don't know why the zfs command of second job failed with empty stderr. I checked syslog but no related log. I have been using pyznap for about 1.5 year with no error except this one.

It can be prevented by modifying /etc/cron.d/pyznap to use flock: flock -w 600 /var/lock/pyznap.lock /opt/pyznap/venv/bin/pyznap snap >> /var/log/pyznap.log 2>&1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant