Skip to content

1.29.4_arenadata7

Compare
Choose a tag to compare
@deart2k deart2k released this 10 Dec 12:14
· 246 commits to master since this release
9ef2f51
Fix pipe recreation when restoring to smaller cluster (#50)

Restoring the same table to smaller cluster is logically divided to batches.
Each batch (one or more) represents a segment's data from larger cluster. The
writer (gprestore helper) writes batches to pipe while the reader (COPY command
initiated by gprestore) reads and applies all data from it. If reader is fast
enough, there can be a race between it and writer. For the next batch, reader
can open flushed, but not yet deleted pipe. This cause writer to wait endlessly
on ENXIO loop and reader to wait on deleted pipe. The solution is to recreate
pipe before flushing. The reader still able to read from such unlinked pipe and
the next reader loop will open a new pipe with the same name.

I spent much time trying to find an explanation of "data holdover bug" mentioned
in pipe recreation comment. To save our time in the future, I decided to update
a comment to clear this problem too. The comment also highlights general
suboptimality of current solution with a plan of future refactoring.

I found no easy way to automate testing of this race condition. To prove a bug,
I advice to add a sleep after flush on non-patched version. Performing a restore
from larger to smaller after this will cause the restore process to hang.