Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](memory) Disable Jemalloc Hook #45210

Merged
merged 1 commit into from
Dec 13, 2024
Merged

[fix](memory) Disable Jemalloc Hook #45210

merged 1 commit into from
Dec 13, 2024

Conversation

xinyiZzz
Copy link
Contributor

@xinyiZzz xinyiZzz commented Dec 9, 2024 •
edited
Loading

What problem does this PR solve?

fix:

*** Aborted at 1733734087 (unix time) try "date -d @1733734087" if you are using GNU date ***
*** Current BE git commitID: 43f06a5e26 ***
*** SIGSEGV address not mapped to object (@0x0) received by PID 1671420 (TID 1671420 OR 0x7f4f35f74ac0) from PID 0; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_release/doris/be/src/common/signal_handler.h:421
 1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in /usr/local/jdk-17.0.2/lib/server/libjvm.so
 2# JVM_handle_linux_signal in /usr/local/jdk-17.0.2/lib/server/libjvm.so
 3# 0x00007F4F34340400 in /lib64/libc.so.6
 4# je_arena_dalloc_promoted at ../src/arena.c:1277
 5# je_free_default at ../src/jemalloc.c:3014
 6# __pthread_create_2_1 in /lib64/libpthread.so.0
 7# os::create_thread(Thread*, os::ThreadType, unsigned long) in /usr/local/jdk-17.0.2/lib/server/libjvm.so
 8# JVM_StartThread in /usr/local/jdk-17.0.2/lib/server/libjvm.so
 9# 0x00007F4F0A96C918

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Dec 9, 2024

clang-tidy review says "All clean, LGTM! 👍"

@xinyiZzz xinyiZzz force-pushed the 20241209_disable_jemallochook branch from eae828c to 05dddc3 Compare December 9, 2024 13:42
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Dec 9, 2024

clang-tidy review says "All clean, LGTM! 👍"

@xinyiZzz
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.78% (10102/26047)
Line Coverage: 29.70% (84718/285285)
Region Coverage: 28.77% (43492/151183)
Branch Coverage: 25.32% (22092/87248)
Coverage Report: http://coverage.selectdb-in.cc/coverage/05dddc3005b6fc6d28437afc524a0f8e22a1f346_05dddc3005b6fc6d28437afc524a0f8e22a1f346/report/index.html

@xinyiZzz xinyiZzz force-pushed the 20241209_disable_jemallochook branch from 05dddc3 to 38503bf Compare December 12, 2024 03:05
@xinyiZzz
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.75% (10104/26073)
Line Coverage: 29.69% (84764/285526)
Region Coverage: 28.75% (43500/151325)
Branch Coverage: 25.30% (22097/87330)
Coverage Report: http://coverage.selectdb-in.cc/coverage/38503bf1c863bd7af73554779fa922f759331c5e_38503bf1c863bd7af73554779fa922f759331c5e/report/index.html

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 12, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

wangbo
wangbo approved these changes Dec 13, 2024
View reviewed changes
Copy link
Contributor

@wangbo wangbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xinyiZzz xinyiZzz merged commit c5a2307 into apache:master Dec 13, 2024
27 of 30 checks passed
@xinyiZzz xinyiZzz changed the title [fix](memory) Disable Jemalloc Hook and loading libjvm at runtime [fix](memory) Disable Jemalloc Hook Dec 13, 2024
BiteTheDDDDt pushed a commit that referenced this pull request Dec 13, 2024
@xinyiZzz @BiteTheDDDDt
[fix](memory) Disable Jemalloc Hook and loading libjvm at runtime (#4…
8218bfd 
…5210)

fix:
```
*** Aborted at 1733734087 (unix time) try "date -d @1733734087" if you are using GNU date ***
*** Current BE git commitID: 43f06a5 ***
*** SIGSEGV address not mapped to object (@0x0) received by PID 1671420 (TID 1671420 OR 0x7f4f35f74ac0) from PID 0; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_release/doris/be/src/common/signal_handler.h:421
 1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in /usr/local/jdk-17.0.2/lib/server/libjvm.so
 2# JVM_handle_linux_signal in /usr/local/jdk-17.0.2/lib/server/libjvm.so
 3# 0x00007F4F34340400 in /lib64/libc.so.6
 4# je_arena_dalloc_promoted at ../src/arena.c:1277
 5# je_free_default at ../src/jemalloc.c:3014
 6# __pthread_create_2_1 in /lib64/libpthread.so.0
 7# os::create_thread(Thread*, os::ThreadType, unsigned long) in /usr/local/jdk-17.0.2/lib/server/libjvm.so
 8# JVM_StartThread in /usr/local/jdk-17.0.2/lib/server/libjvm.so
 9# 0x00007F4F0A96C918
```
BiteTheDDDDt added a commit that referenced this pull request Dec 13, 2024
@BiteTheDDDDt
Revert "[fix](memory) Disable Jemalloc Hook and loading libjvm at run…
318087e 
…time (#45210)"

This reverts commit 8218bfd.
@coldWeather66260
Copy link

Building failed on my mac, it said can't find mallctl symbol. Because jemalloc's default prefix on macOS is "je_", not "".
from doc:

By default, the prefix is "", except on OS X, where it is "je_". On OS X, jemalloc overlays the default malloc zone, but makes no attempt to actually replace the "malloc", "calloc", etc. symbols.

xinyiZzz added a commit to xinyiZzz/incubator-doris that referenced this pull request Dec 16, 2024
@xinyiZzz
[fix](memory) Disable Jemalloc Hook and loading libjvm at runtime (ap…
7046871 
…ache#45210)

fix:
```
*** Aborted at 1733734087 (unix time) try "date -d @1733734087" if you are using GNU date ***
*** Current BE git commitID: 43f06a5 ***
*** SIGSEGV address not mapped to object (@0x0) received by PID 1671420 (TID 1671420 OR 0x7f4f35f74ac0) from PID 0; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_release/doris/be/src/common/signal_handler.h:421
 1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in /usr/local/jdk-17.0.2/lib/server/libjvm.so
 2# JVM_handle_linux_signal in /usr/local/jdk-17.0.2/lib/server/libjvm.so
 3# 0x00007F4F34340400 in /lib64/libc.so.6
 4# je_arena_dalloc_promoted at ../src/arena.c:1277
 5# je_free_default at ../src/jemalloc.c:3014
 6# __pthread_create_2_1 in /lib64/libpthread.so.0
 7# os::create_thread(Thread*, os::ThreadType, unsigned long) in /usr/local/jdk-17.0.2/lib/server/libjvm.so
 8# JVM_StartThread in /usr/local/jdk-17.0.2/lib/server/libjvm.so
 9# 0x00007F4F0A96C918
```
@xinyiZzz xinyiZzz mentioned this pull request Dec 16, 2024
Closed
16 tasks
xinyiZzz added a commit to xinyiZzz/incubator-doris that referenced this pull request Dec 16, 2024
@xinyiZzz
[fix](memory) Disable Jemalloc Hook and loading libjvm at runtime (ap…
a3810f6 
…ache#45210)

### What problem does this PR solve?

fix:
```
*** Aborted at 1733734087 (unix time) try "date -d @1733734087" if you are using GNU date ***
*** Current BE git commitID: 43f06a5 ***
*** SIGSEGV address not mapped to object (@0x0) received by PID 1671420 (TID 1671420 OR 0x7f4f35f74ac0) from PID 0; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_release/doris/be/src/common/signal_handler.h:421
 1# PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] in /usr/local/jdk-17.0.2/lib/server/libjvm.so
 2# JVM_handle_linux_signal in /usr/local/jdk-17.0.2/lib/server/libjvm.so
 3# 0x00007F4F34340400 in /lib64/libc.so.6
 4# je_arena_dalloc_promoted at ../src/arena.c:1277
 5# je_free_default at ../src/jemalloc.c:3014
 6# __pthread_create_2_1 in /lib64/libpthread.so.0
 7# os::create_thread(Thread*, os::ThreadType, unsigned long) in /usr/local/jdk-17.0.2/lib/server/libjvm.so
 8# JVM_StartThread in /usr/local/jdk-17.0.2/lib/server/libjvm.so
 9# 0x00007F4F0A96C918
```
@xinyiZzz xinyiZzz mentioned this pull request Dec 16, 2024
Open
16 tasks
xinyiZzz added a commit to xinyiZzz/incubator-doris that referenced this pull request Dec 17, 2024
@xinyiZzz
Revert "[fix](memory) Disable Jemalloc Hook and loading libjvm at run…
917ac73 
…time (apache#45210)"

This reverts commit c5a2307.
xinyiZzz added a commit to xinyiZzz/incubator-doris that referenced this pull request Dec 17, 2024
@xinyiZzz
Revert "[fix](memory) Disable Jemalloc Hook and loading libjvm at run…
068d5bb 
…time (apache#45210)"

This reverts commit c5a2307.
@xinyiZzz xinyiZzz mentioned this pull request Dec 17, 2024
Closed
16 tasks
xinyiZzz added a commit to xinyiZzz/incubator-doris that referenced this pull request Dec 17, 2024
@xinyiZzz
Revert "[fix](memory) Disable Jemalloc Hook and loading libjvm at run…
fb60f06 
…time (apache#45210)"

This reverts commit c5a2307.
@xinyiZzz
Copy link
Contributor Author

Building failed on my mac, it said can't find mallctl symbol. Because jemalloc's default prefix on macOS is "je_", not "". from doc:

By default, the prefix is "", except on OS X, where it is "je_". On OS X, jemalloc overlays the default malloc zone, but makes no attempt to actually replace the "malloc", "calloc", etc. symbols.

Hi, @coldWeather66260, I forgot to check Mac compilation.

I reverted this PR #45534, you can try USE_JEMALLOC=OFF ./build.sh --be to compile on Mac

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wangbo wangbo wangbo approved these changes

@yiguolei yiguolei yiguolei approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@xinyiZzz @doris-robot @coldWeather66260 @yiguolei @wangbo