Skip to content

Commit

Permalink
Always sort WAL segments on restore point calc
Browse files Browse the repository at this point in the history
Some S3 implementations, like Storj, don't return objects in order
unlike the real S3 API. It's against AWS documentation but they
have their reasons not to pre-sort them.

Since there's only one place where the order matters and we're
going to iterate all of them anyway and there's a sort
implementation for a WAL segment slice we can do just that to
add compatibility without any meaningful performance hit.

This same sorting is already done when listing snapshots.

Fixes benbjohnson#535
  • Loading branch information
hifi committed Dec 15, 2023
1 parent 85ddf32 commit 73fbfc0
Showing 1 changed file with 7 additions and 2 deletions.
9 changes: 7 additions & 2 deletions replica.go
Original file line number Diff line number Diff line change
Expand Up @@ -1287,10 +1287,15 @@ func (r *Replica) walSegmentMap(ctx context.Context, generation string, maxIndex
}
defer itr.Close()

m := make(map[int][]int64)
a := []WALSegmentInfo{}
for itr.Next() {
info := itr.WALSegment()
a = append(a, itr.WALSegment())
}

sort.Sort(WALSegmentInfoSlice(a))

m := make(map[int][]int64)
for _, info := range a {
// Exit if we go past the max timestamp or index.
if !maxTimestamp.IsZero() && info.CreatedAt.After(maxTimestamp) {
break // after max timestamp, skip
Expand Down

0 comments on commit 73fbfc0

Please sign in to comment.