Skip to content

Commit

Permalink
[fix](clone) Fix the skipped link file due to the stale value (apache…
Browse files Browse the repository at this point in the history
…#46009)

The clone task will compare the differences and similarities between
local and downloaded files. If the md5sum of two binlog files is the
same, it will set `skip_link_file` to skip the linking of that file.
However, `skip_link_file` isn't reset on time. As a result, subsequent
files will all be skipped because `skip_link_file` has been set, which
leads to the problem that for some rowsets, the meta exists while the
segment file doesn't.

The error msg:

```
[NOT_FOUND]failed to get file size /mnt/disk3/VEC_ASAN2/doris.HDD/data/900/13372467/1624966973/02000000002e1250744766e3a98c0e213d847daea65158a9_0.dat
```
  • Loading branch information
w41ter authored Dec 27, 2024
1 parent ae46453 commit 65bfa23
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions be/src/olap/task/engine_clone_task.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -813,15 +813,15 @@ Status EngineCloneTask::_finish_clone(Tablet* tablet, const std::string& clone_d
/// Traverse all downloaded clone files in CLONE dir.
/// If it does not exist in local tablet dir, link the file to local tablet dir
/// And save all linked files in linked_success_files.
/// if binlog exist in clone dir and md5sum equal, then skip link file
bool skip_link_file = false;
for (const string& clone_file : clone_file_names) {
if (local_file_names.find(clone_file) != local_file_names.end()) {
VLOG_NOTICE << "find same file when clone, skip it. "
<< "tablet=" << tablet->tablet_id() << ", clone_file=" << clone_file;
continue;
}

/// if binlog exist in clone dir and md5sum equal, then skip link file
bool skip_link_file = false;
std::string to;
if (clone_file.ends_with(".binlog") || clone_file.ends_with(".binlog-index")) {
if (!contain_binlog) {
Expand Down

0 comments on commit 65bfa23

Please sign in to comment.