Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preservation batch fails if media node ID does not exist as a Drupal node #19

Open
jefferya opened this issue Aug 19, 2024 · 1 comment
Assignees

Comments

@jefferya
Copy link
Contributor

jefferya commented Aug 19, 2024

The preservation run fails if there is a media item with a node ID and that node ID does not exist in the Drupal nodes. There might have been a bug that when a Drupal node was deleted the associated media items were not deleted. This causes the preservation run to fail with an exception.

bagger-1  | Traceback (most recent call last):
bagger-1  |   File "/var/www/leaf-isle-bagger/leaf-bagger.py", line 130, in <module>
bagger-1  |     main()
bagger-1  |   File "/var/www/leaf-isle-bagger/leaf-bagger.py", line 126, in main
bagger-1  |     process(args, session)
bagger-1  |   File "/var/www/leaf-isle-bagger/leaf-bagger.py", line 105, in process
bagger-1  |     swiftUtilities.upload_aip(
bagger-1  |   File "/var/www/leaf-isle-bagger/swift/utilities.py", line 41, in upload_aip
bagger-1  |     checksums = file_checksum(aip_path)
bagger-1  |                 ^^^^^^^^^^^^^^^^^^^^^^^
bagger-1  |   File "/var/www/leaf-isle-bagger/swift/utilities.py", line 136, in file_checksum
bagger-1  |     with open(path, "rb") as f:
bagger-1  |          ^^^^^^^^^^^^^^^^
bagger-1  | FileNotFoundError: [Errno 2] No such file or directory: '/data/aip/aip_59.zip'

Why do the following fail? Are the all the same reason as listed above?

bagger-1  | INFO:root:  Generating AIP: 59
bagger-1  | ERROR:root:Command '['./bin/console', 'app:islandora_bagger:create_bag', '-vvv', '--settings=var/sample_per_bag_config.yaml', '--node=59']' returned non-zero exit status 255.
bagger-1  | INFO:root:  Generating AIP: 21
bagger-1  | ERROR:root:Command '['./bin/console', 'app:islandora_bagger:create_bag', '-vvv', '--settings=var/sample_per_bag_config.yaml', '--node=21']' returned non-zero exit status 255.
bagger-1  | INFO:root:  Generating AIP: 32
bagger-1  | ERROR:root:Command '['./bin/console', 'app:islandora_bagger:create_bag', '-vvv', '--settings=var/sample_per_bag_config.yaml', '--node=32']' returned non-zero exit status 255.
bagger-1  | INFO:root:  Generating AIP: 13
bagger-1  | ERROR:root:Command '['./bin/console', 'app:islandora_bagger:create_bag', '-vvv', '--settings=var/sample_per_bag_config.yaml', '--node=13']' returned non-zero exit status 255.
bagger-1  | INFO:root:  Generating AIP: 12
bagger-1  | ERROR:root:Command '['./bin/console', 'app:islandora_bagger:create_bag', '-vvv', '--settings=var/sample_per_bag_config.yaml', '--node=12']' returned non-zero exit status 255.
bagger-1  | INFO:root:  Generating AIP: 15
bagger-1  | ERROR:root:Command '['./bin/console', 'app:islandora_bagger:create_bag', '-vvv', '--settings=var/sample_per_bag_config.yaml', '--node=15']' returned non-zero exit status 255.
bagger-1  | INFO:root:  Generating AIP: 29
bagger-1  | ERROR:root:Command '['./bin/console', 'app:islandora_bagger:create_bag', '-vvv', '--settings=var/sample_per_bag_config.yaml', '--node=29']' returned non-zero exit status 255.
bagger-1  | INFO:root:  Generating AIP: 6
bagger-1  | ERROR:root:Command '['./bin/console', 'app:islandora_bagger:create_bag', '-vvv', '--settings=var/sample_per_bag_config.yaml', '--node=6']' returned non-zero exit status 255.
bagger-1  | INFO:root:  Generating AIP: 17
bagger-1  | ERROR:root:Command '['./bin/console', 'app:islandora_bagger:create_bag', '-vvv', '--settings=var/sample_per_bag_config.yaml', '--node=17']' returned non-zero exit status 255.
bagger-1  | INFO:root:  Generating AIP: 3
bagger-1  | ERROR:root:Command '['./bin/console', 'app:islandora_bagger:create_bag', '-vvv', '--settings=var/sample_per_bag_config.yaml', '--node=3']' returned non-zero exit status 255.

Todo:

  • add exception handling to log failure and gracefully continue
@jefferya
Copy link
Contributor Author

The following is a list of Drupal IDs that are failing preservation. Best guess, the Drupal node was deleted however the associated Drupal media was not deleted thus are failing preservation because of the missing Drupal node component during the archival package creation

drupal_id,drupal_changed,swift_id,swift_timestamp,swift_meta_changed,swift_bytes,status
59,2024-03-25T16:01:12+00:00,,,,,xm
21,2022-05-18T13:29:32+00:00,,,,,xm
32,2022-05-18T13:29:28+00:00,,,,,xm
13,2022-05-18T13:29:24+00:00,,,,,xm
12,2022-05-18T13:29:21+00:00,,,,,xm
15,2022-05-18T13:29:13+00:00,,,,,xm
29,2022-05-18T13:29:10+00:00,,,,,xm
6,2022-05-18T13:29:06+00:00,,,,,xm
17,2022-05-18T13:29:03+00:00,,,,,xm
3,2022-05-18T13:28:58+00:00,,,,,xm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant