New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

`malloc_usable_size()` segfaults under the proxy library based on the jemalloc pool #894

Open

ldorau opened this issue Nov 13, 2024 · 0 comments

Assignees

Labels

bug

Contributor

ldorau commented Nov 13, 2024 •

edited

Loading

malloc_usable_size() segfaults under the proxy library based on the jemalloc pool.

When the proxy library is based on the scalable pool malloc_usable_size() works correctly.

Ref: #883

Root cause:

op_malloc_usable_size() in pool_jemalloc calls malloc_usable_size@libumf_proxy instead of malloc_usable_size@jemalloc what causes an infinite recursive loop of function calls and a segfault as a result.

The text was updated successfully, but these errors were encountered:

ldorau added the bug label

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library

20a51e6

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library. The size threshold is set to 64 bytes
in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library

60abe5f

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library. The size threshold is set to 64 bytes
in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

This was referenced Nov 13, 2024

Add threshold to proxy lib to call system allocator #883

Merged

jemalloc_pool should not require user to link with jemalloc #891

Open

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library. The size threshold is set to 64 bytes
in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library

913af51

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library. The size threshold is set to 64 bytes
in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library

7d2f6b1

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library. The size threshold is set to 64 bytes
in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library

941b6b6

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library. The size threshold is set to 64 bytes
in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library

f455600

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library. The size threshold is set to 64 bytes
in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library

b5d4fae

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library. The size threshold is set to 64 bytes
in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library

927be5e

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library. The size threshold is set to 64 bytes
in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library

0ae9670

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library. The size threshold is set to 64 bytes
in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library

d25b7c5

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library. The size threshold is set to 64 bytes
in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library

456d29c

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library. The size threshold is set to 64 bytes
in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library

8ffcdaa

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library. The size threshold is set to 64 bytes
in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library

f8255ba

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library. The size threshold is set to 64 bytes
in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library

bce112f

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library. The size threshold is set to 64 bytes
in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library

9f4c98d

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library. The size threshold is set to 64 bytes
in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library (Linux only)

fb1d98c

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library (Linux only yet). The size threshold
is set to 64 bytes in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library (Linux only)

89da0bf

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library (Linux only yet). The size threshold
is set to 64 bytes in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library (Linux only)

4ac6b3e

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library (Linux only yet). The size threshold
is set to 64 bytes in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library (Linux only)

846f30b

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library (Linux only yet). The size threshold
is set to 64 bytes in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library (Linux only)

cd33d7a

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library (Linux only yet). The size threshold
is set to 64 bytes in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library (Linux only)

40f9cfc

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library (Linux only yet). The size threshold
is set to 64 bytes in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library (Linux only)

e1ca431

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library (Linux only yet). The size threshold
is set to 64 bytes in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library (Linux only)

653bd3d

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library (Linux only yet). The size threshold
is set to 64 bytes in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau changed the title ~~malloc_usable_size() segfaults under the proxy library based on the jemalloc pool with the size threshold enabled~~ malloc_usable_size() segfaults under the proxy library based on the jemalloc pool

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add WA for the issue

f99b905

This WA for the issue:
oneapi-src#894
It protects us from a recursion in malloc_usable_size()
when the JEMALLOC proxy_lib_pool is used.

TODO: remove this WA when the issue is fixed.

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library (Linux only)

ec89615

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library (Linux only yet). The size threshold
is set to 64 bytes in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add WA for the issue

19b65e8

This WA for the issue:
oneapi-src#894
It protects us from a recursion in malloc_usable_size()
when the JEMALLOC proxy_lib_pool is used.

TODO: remove this WA when the issue is fixed.

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library (Linux only)

d362ff2

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library (Linux only yet). The size threshold
is set to 64 bytes in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add WA for the issue

d8bac48

This WA for the issue:
oneapi-src#894
It protects us from a recursion in malloc_usable_size()
when the JEMALLOC proxy_lib_pool is used.

TODO: remove this WA when the issue is fixed.

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library (Linux only)

0a81104

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library (Linux only yet). The size threshold
is set to 64 bytes in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add WA for the issue

ea3599d

This WA for the issue:
oneapi-src#894
It protects us from a recursion in malloc_usable_size()
when the JEMALLOC proxy_lib_pool is used.

TODO: remove this WA when the issue is fixed.

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library (Linux only)

efc409a

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library (Linux only yet). The size threshold
is set to 64 bytes in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add WA for the issue

c0ef772

This WA for the issue:
oneapi-src#894
It protects us from a recursion in malloc_usable_size()
when the JEMALLOC proxy_lib_pool is used.

TODO: remove this WA when the issue is fixed.

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library (Linux only)

03b73e5

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library (Linux only yet). The size threshold
is set to 64 bytes in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add WA for the issue

a4fced6

This WA for the issue:
oneapi-src#894
It protects us from a recursion in malloc_usable_size()
when the JEMALLOC proxy_lib_pool is used.

TODO: remove this WA when the issue is fixed.

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Add tests for the size threshold of the proxy library (Linux only)

3966c3a

The proxyLib_size_threshold_* tests test the size threshold
of the proxy library (Linux only yet). The size threshold
is set to 64 bytes in this test, so all allocations of:
1) size <  64 go through the default system allocator
   and (umfPoolByPtr(ptr_size < 64) == nullptr)
2) size >= 64 go through the proxy lib allocator
   and (umfPoolByPtr(ptr_size >= 64) != nullptr).

Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau self-assigned this

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Load jemalloc library dynamically

4241d06

Do not link with the jemalloc library explicitly,
but load it dynamically (using dlopen()).

Ref: oneapi-src#891
Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

ldorau mentioned this issue

[DRAFT] Load jemalloc library dynamically #919

Draft

3 tasks

ldorau added a commit to ldorau/unified-memory-framework that referenced this issue


          Load jemalloc library dynamically

5fa5210

Do not link with the jemalloc library explicitly,
but load it dynamically (using dlopen()).

Ref: oneapi-src#891
Ref: oneapi-src#894

Signed-off-by: Lukasz Dorau <[email protected]>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment