Problem: Archivematica not deleting first normalized objects on reingest

[sromkey]

Expected behaviour
When an AIP is reingested and normalized for preservation a second time, the preservation derivatives that were created through normalization on the initial ingest should be removed from the AIP and be replaced with the newly created preservation derivatives.

Current behaviour
The newly created preservation derivatives are added to the AIP but the first preservation derivatives also remain.

Interestingly the first preservation derivatives are correctly denoted in the AIP METS file as being in <mets:fileGrp USE="deleted">

Steps to reproduce

1. Create an AIP which has normalized preservation derivatives in it (for example, JPEGs, and ensure that a normalization rule for that version of JPEGs is enabled. I used archivematica-sampledata/SampleTransfers/Images/pictures and enabled the rule to normalize JPEG 1.01 to tiff and TGA 2.0 to tiff.
2. After the AIP is stored, do a full reingest, ensuring that you choose Normalize for Preservation. Optionally you can first adjust the rules to normalize the format to something else, but it shouldn't matter- normalization should run anyway and the preservation derivatives should still be replaced.
3. Open your reingested AIP. You will find the originals plus 2 preservation derivatives for each.

Your environment (version of Archivematica, operating system, other relevant details)
AM 1.16 on Ubuntu 22.04 Jammy

---

For Artefactual use:

Before you close this issue, you must check off the following:

• All pull requests related to this issue are properly linked
• All pull requests related to this issue have been merged
• A testing plan for this issue has been implemented and passed (testing plan information should be included in the issue body or comments)
• Documentation regarding this issue has been written and merged (if applicable)
• Details about this issue have been added to the release notes (if applicable)

[Dhwaniartefact]

Thank you for reporting this issue.

I am able to produce this issue and noticed that the pipeline is deleting the previous derivative file as expected but in storage service "finish_reingest" does not delete it. Storage service performs it with the file system of the AIP.

Due to pipeline functioning as planned, we can use METS structMap information to compare it with the re-ingested AIP which may come up as a potential solution in the future. There is still work to be done on that.