Speed up the search for chunks to skip deletion for

Speed up the search for chunks to skip deletion for #1126