Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](string64) fix coredump caused by ColumnArray<ColumnStr<uint64_t>>::insert_indices_from #43624

Merged
merged 3 commits into from
Nov 13, 2024
Merged

[fix](string64) fix coredump caused by ColumnArray<ColumnStr<uint64_t>>::insert_indices_from #43624

merged 3 commits into from
Nov 13, 2024

Conversation

jacktengg
Copy link
Contributor

@jacktengg jacktengg commented Nov 11, 2024 •
edited
Loading

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
In order to avoid the problem of total string length of a ColumnStr<uint32_t> column exceeds the 4G limit, hash join operator call MutableBlock::merge_ignore_overflow to collect build data.

After finishing collecting build data, a ColumnStr<uint32_t> column will be converted to ColumnStr<uint64_t> if the total string length exceeds the 4G limit by calling Block::replace_if_overflow.

When hash join operator ouput join result block, it need to calls ColumnStr<T>::insert_indices_from to fill a string column, which is able to handle both ColumnStr<uint32_t> and ColumnStr<uint64_t>. Every column which may include string sub-columns should also call insert_indices_from method of it's sub-column to handle both ColumnStr<uint32_t> and ColumnStr<uint64_t>, or else column cast will fail.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@jacktengg
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@jacktengg jacktengg force-pushed the fix-1108 branch 2 times, most recently from 3241ad5 to 1ebcfc0 Compare November 11, 2024 16:07
@jacktengg
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@jacktengg jacktengg force-pushed the fix-1108 branch 2 times, most recently from 6fe1301 to d6b0468 Compare November 12, 2024 07:07
@jacktengg
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@jacktengg jacktengg force-pushed the fix-1108 branch 2 times, most recently from a0c5a3a to 9482abe Compare November 12, 2024 07:43
@jacktengg
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@yiguolei yiguolei added the p0_c label Nov 12, 2024
@jacktengg
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 13, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

HappenLee
HappenLee approved these changes Nov 13, 2024
View reviewed changes
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yiguolei yiguolei merged commit 0d7b63f into apache:master Nov 13, 2024
25 of 28 checks passed
github-actions bot pushed a commit that referenced this pull request Nov 13, 2024
@jacktengg
[fix](string64) fix coredump caused by ColumnArray<ColumnStr<uint64_t…
2ff0ad9 
…>>::insert_indices_from (#43624)
@github-actions github-actions bot mentioned this pull request Nov 13, 2024
Merged
github-actions bot pushed a commit that referenced this pull request Nov 13, 2024
@jacktengg
[fix](string64) fix coredump caused by ColumnArray<ColumnStr<uint64_t…
91abfd0 
…>>::insert_indices_from (#43624)
@github-actions github-actions bot mentioned this pull request Nov 13, 2024
Merged
zy-kkk pushed a commit that referenced this pull request Nov 13, 2024
@CalvinKirs
[chore](auto-pick)Add the original PR number suffix to the automatica…
3fcd700 
…lly picked PR title (#43866)

```
**before**
branch-2.1: [fix](string64) fix coredump caused by ColumnArray<ColumnStr<uint64_t>>::insert_indices_from
**after**
branch-2.1: [fix](string64) fix coredump caused by ColumnArray<ColumnStr<uint64_t>>::insert_indices_from #43624
```
yiguolei pushed a commit that referenced this pull request Nov 13, 2024
@github-actions @jacktengg
branch-3.0: [fix](string64) fix coredump caused by ColumnArray<Column…
8febc99 
…Str<uint64_t>>::insert_indices_from (#43861)

Cherry-picked from #43624

Co-authored-by: TengJianPing <[email protected]>
yiguolei pushed a commit that referenced this pull request Nov 13, 2024
@github-actions @jacktengg
branch-2.1: [fix](string64) fix coredump caused by ColumnArray<Column…
d4712ae 
…Str<uint64_t>>::insert_indices_from (#43862)

Cherry-picked from #43624

Co-authored-by: TengJianPing <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@HappenLee HappenLee HappenLee approved these changes

@yiguolei yiguolei yiguolei approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.8-merged dev/3.0.3-merged p0_c reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@jacktengg @doris-robot @yiguolei @HappenLee