Update CONTRIBUTING.md #25
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This issue is available for anyone to work on. Make sure to reference this issue in your pull request. ✨ Thank you for your contribution! ✨ |
|
Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR |
CalvinKirs
pushed a commit
that referenced
this pull request
Oct 24, 2024
…dir when index file writer open index (apache#42207) ## Proposed changes Fix UAF as below ``` ==16442==ERROR: AddressSanitizer: heap-use-after-free on address 0x60f0008718a0 at pc 0x5586663cea7a bp 0x7f02673d0530 sp 0x7f02673d0528 READ of size 8 at 0x60f0008718a0 thread T490 (CumuCompactionT) #0 0x5586663cea79 in doris::segment_v2::InvertedIndexColumnWriterImpl<(doris::FieldType)5>::finish() /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp:549:63 #1 0x5586663360bc in doris::segment_v2::ScalarColumnWriter::write_inverted_index() /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/column_writer.cpp:640:41 #2 0x5586662ce160 in doris::segment_v2::SegmentWriter::_write_inverted_index() /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:1435:9 #3 0x5586662ccb13 in doris::segment_v2::SegmentWriter::finalize_columns_index(unsigned long*) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:1317:5 #4 0x55866653c074 in doris::VerticalBetaRowsetWriter::_flush_columns(doris::segment_v2::SegmentWriter*, bool) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/vertical_beta_rowset_writer.cpp:127:5 #5 0x558666534e3f in doris::VerticalBetaRowsetWriter::flush_columns(bool) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/vertical_beta_rowset_writer.cpp:153:5 #6 0x558665a8d8b2 in doris::Merger::vertical_compact_one_group(std::shared_ptr, doris::ReaderType, doris::TabletSchema const&, bool, std::vector> const&, doris::vectorized::RowSourcesBuffer*, std::vector, std::allocator>> const&, doris::RowsetWriter*, long, doris::Merger::Statistics*, std::vector>, long, doris::CompactionSampleInfo*) /home/zcp/repo_center/doris_master/doris/be/src/olap/merger.cpp:333:5 #7 0x558665a93190 in doris::Merger::vertical_merge_rowsets(std::shared_ptr, doris::ReaderType, doris::TabletSchema const&, std::vector, std::allocator>> const&, doris::RowsetWriter*, long, long, doris::Merger::Statistics*) /home/zcp/repo_center/doris_master/doris/be/src/olap/merger.cpp:471:21 #8 0x558665a00fd7 in doris::Compaction::merge_input_rowsets() /home/zcp/repo_center/doris_master/doris/be/src/olap/compaction.cpp:192:19 #9 0x558665a329a3 in doris::CloudCompactionMixin::execute_compact_impl(long) /home/zcp/repo_center/doris_master/doris/be/src/olap/compaction.cpp:1158:5 #10 0x558665a33381 in doris::CloudCompactionMixin::execute_compact() /home/zcp/repo_center/doris_master/doris/be/src/olap/compaction.cpp:1173:5 #11 0x55869a893d89 in doris::CloudCumulativeCompaction::execute_compact() /home/zcp/repo_center/doris_master/doris/be/src/cloud/cloud_cumulative_compaction.cpp:191:38 #12 0x55869a861fac in doris::CloudStorageEngine::_submit_cumulative_compaction_task(std::shared_ptr const&)::$_1::operator()() const /home/zcp/repo_center/doris_master/doris/be/src/cloud/cloud_storage_engine.cpp:693:31 #13 0x55869a861fac in void std::__invoke_impl const&)::$_1&>(std::__invoke_other, doris::CloudStorageEngine::_submit_cumulative_compaction_task(std::shared_ptr const&)::$_1&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 #14 0x55869a861fac in std::enable_if const&)::$_1&>, void>::type std::__invoke_r const&)::$_1&>(doris::CloudStorageEngine::_submit_cumulative_compaction_task(std::shared_ptr const&)::$_1&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2 #15 0x55869a861fac in std::_Function_handler const&)::$_1>::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 #16 0x5586679033fb in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:543:24 #17 0x5586678db057 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5 #18 0x7f0639d7bac2 in start_thread nptl/pthread_create.c:442:8 #19 0x7f0639e0d84f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81 0x60f0008718a0 is located 0 bytes inside of 168-byte region [0x60f0008718a0,0x60f000871948) freed by thread T490 (CumuCompactionT) here: #0 0x558663259d9d in operator delete(void*) (/mnt/hdd01/ci/master-deploy/cluster0/be/lib/doris_be+0x337a1d9d) (BuildId: fa8094411569cb8d) #1 0x558665a655eb in std::default_delete::operator()(lucene::store::Directory*) const /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:85:2 #2 0x558665a655eb in std::unique_ptr>::~unique_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:361:4 #3 0x558665f83dfe in std::pair, std::allocator>> const, std::unique_ptr>>::~pair() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_iterator.h:2379:12 #4 0x558665f83dfe in void std::destroy_at, std::allocator>> const, std::unique_ptr>>>(std::pair, std::allocator>> const, std::unique_ptr>>*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15 #5 0x558665f83dfe in void std::allocator_traits, std::allocator>> const, std::unique_ptr>>>>>::destroy, std::allocator>> const, std::unique_ptr>>>(std::allocator, std::allocator>> const, std::unique_ptr>>>>&, std::pair, std::allocator>> const, std::unique_ptr>>*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:533:4 #6 0x558665f83dfe in std::_Rb_tree, std::allocator>>, std::pair, std::allocator>> const, std::unique_ptr>>, std::_Select1st, std::allocator>> const, std::unique_ptr>>>, std::less, std::allocator>>>, std::allocator, std::allocator>> const, std::unique_ptr>>>>::_M_destroy_node(std::_Rb_tree_node, std::allocator>> const, std::unique_ptr>>>*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_tree.h:623:2 #7 0x558665f83dfe in std::_Rb_tree, std::allocator>>, std::pair, std::allocator>> const, std::unique_ptr>>, std::_Select1st, std::allocator>> const, std::unique_ptr>>>, std::less, std::allocator>>>, std::allocator, std::allocator>> const, std::unique_ptr>>>>::_M_drop_node(std::_Rb_tree_node, std::allocator>> const, std::unique_ptr>>>*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_tree.h:631:2 #8 0x558665f9e9b8 in std::pair, std::allocator>> const, std::unique_ptr>>>, bool> std::_Rb_tree, std::allocator>>, std::pair, std::allocator>> const, std::unique_ptr>>, std::_Select1st, std::allocator>> const, std::unique_ptr>>>, std::less, std::allocator>>>, std::allocator, std::allocator>> const, std::unique_ptr>>>>::_M_emplace_unique, std::allocator>>, std::unique_ptr>>(std::pair, std::allocator>>&&, std::unique_ptr>&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_tree.h:2391:6 #9 0x558665f86e29 in std::pair, std::allocator>> const, std::unique_ptr>>>, bool> std::map, std::allocator>>, std::unique_ptr>, std::less, std::allocator>>>, std::allocator, std::allocator>> const, std::unique_ptr>>>>::emplace, std::allocator>>, std::unique_ptr>>(std::pair, std::allocator>>&&, std::unique_ptr>&&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_map.h:577:16 #10 0x558665f86e29 in doris::segment_v2::InvertedIndexFileWriter::open(doris::TabletIndex const*) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_file_writer.cpp:60:19 #11 0x5586663d0423 in doris::segment_v2::InvertedIndexColumnWriterImpl<(doris::FieldType)5>::open_index_directory() /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp:160:16 #12 0x5586663d00f7 in doris::segment_v2::InvertedIndexColumnWriterImpl<(doris::FieldType)5>::init_bkd_index() /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp:146:16 #13 0x5586663cb2d9 in doris::segment_v2::InvertedIndexColumnWriterImpl<(doris::FieldType)5>::init() /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp:107:24 #14 0x5586663b1201 in doris::segment_v2::InvertedIndexColumnWriter::create(doris::Field const*, std::unique_ptr>*, doris::segment_v2::InvertedIndexFileWriter*, doris::TabletIndex const*) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp:697:27 #15 0x55866632cf8d in doris::segment_v2::ScalarColumnWriter::init() /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/column_writer.cpp:483:13 #16 0x5586662a5e52 in doris::segment_v2::SegmentWriter::_create_column_writer(unsigned int, doris::TabletColumn const&, std::shared_ptr const&) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:267:5 #17 0x5586662a71b4 in doris::segment_v2::SegmentWriter::_create_writers(std::shared_ptr const&, std::vector> const&) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:316:9 #18 0x5586662a2be3 in doris::segment_v2::SegmentWriter::init(std::vector> const&, bool) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:285:5 #19 0x5586665333b2 in doris::VerticalBetaRowsetWriter::add_columns(doris::vectorized::Block const*, std::vector> const&, bool, unsigned int) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/vertical_beta_rowset_writer.cpp:96:17 #20 0x558665a8cb9b in doris::Merger::vertical_compact_one_group(std::shared_ptr, doris::ReaderType, doris::TabletSchema const&, bool, std::vector> const&, doris::vectorized::RowSourcesBuffer*, std::vector, std::allocator>> const&, doris::RowsetWriter*, long, doris::Merger::Statistics*, std::vector>, long, doris::CompactionSampleInfo*) /home/zcp/repo_center/doris_master/doris/be/src/olap/merger.cpp:309:9 #21 0x558665a93190 in doris::Merger::vertical_merge_rowsets(std::shared_ptr, doris::ReaderType, doris::TabletSchema const&, std::vector, std::allocator>> const&, doris::RowsetWriter*, long, long, doris::Merger::Statistics*) /home/zcp/repo_center/doris_master/doris/be/src/olap/merger.cpp:471:21 #22 0x558665a00fd7 in doris::Compaction::merge_input_rowsets() /home/zcp/repo_center/doris_master/doris/be/src/olap/compaction.cpp:192:19 #23 0x558665a329a3 in doris::CloudCompactionMixin::execute_compact_impl(long) /home/zcp/repo_center/doris_master/doris/be/src/olap/compaction.cpp:1158:5 #24 0x558665a33381 in doris::CloudCompactionMixin::execute_compact() /home/zcp/repo_center/doris_master/doris/be/src/olap/compaction.cpp:1173:5 #25 0x55869a893d89 in doris::CloudCumulativeCompaction::execute_compact() /home/zcp/repo_center/doris_master/doris/be/src/cloud/cloud_cumulative_compaction.cpp:191:38 #26 0x55869a861fac in doris::CloudStorageEngine::_submit_cumulative_compaction_task(std::shared_ptr const&)::$_1::operator()() const /home/zcp/repo_center/doris_master/doris/be/src/cloud/cloud_storage_engine.cpp:693:31 #27 0x55869a861fac in void std::__invoke_impl const&)::$_1&>(std::__invoke_other, doris::CloudStorageEngine::_submit_cumulative_compaction_task(std::shared_ptr const&)::$_1&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 #28 0x55869a861fac in std::enable_if const&)::$_1&>, void>::type std::__invoke_r const&)::$_1&>(doris::CloudStorageEngine::_submit_cumulative_compaction_task(std::shared_ptr const&)::$_1&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2 #29 0x55869a861fac in std::_Function_handler const&)::$_1>::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 #30 0x5586679033fb in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:543:24 #31 0x5586678db057 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5 #32 0x7f0639d7bac2 in start_thread nptl/pthread_create.c:442:8 previously allocated by thread T490 (CumuCompactionT) here: #0 0x55866325953d in operator new(unsigned long) (/mnt/hdd01/ci/master-deploy/cluster0/be/lib/doris_be+0x337a153d) (BuildId: fa8094411569cb8d) #1 0x558665fb7d7b in doris::segment_v2::DorisFSDirectoryFactory::getDirectory(std::shared_ptr const&, char const*, bool, lucene::store::LockFactory*) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_fs_directory.cpp:797:15 #2 0x558665f86b5c in doris::segment_v2::InvertedIndexFileWriter::open(doris::TabletIndex const*) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_file_writer.cpp:58:17 #3 0x5586663d0423 in doris::segment_v2::InvertedIndexColumnWriterImpl<(doris::FieldType)5>::open_index_directory() /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp:160:16 #4 0x5586663d00f7 in doris::segment_v2::InvertedIndexColumnWriterImpl<(doris::FieldType)5>::init_bkd_index() /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp:146:16 #5 0x5586663cb2d9 in doris::segment_v2::InvertedIndexColumnWriterImpl<(doris::FieldType)5>::init() /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp:107:24 #6 0x5586663b1201 in doris::segment_v2::InvertedIndexColumnWriter::create(doris::Field const*, std::unique_ptr>*, doris::segment_v2::InvertedIndexFileWriter*, doris::TabletIndex const*) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/inverted_index_writer.cpp:697:27 #7 0x55866632cf8d in doris::segment_v2::ScalarColumnWriter::init() /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/column_writer.cpp:483:13 #8 0x5586662a5e52 in doris::segment_v2::SegmentWriter::_create_column_writer(unsigned int, doris::TabletColumn const&, std::shared_ptr const&) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:267:5 #9 0x5586662a71b4 in doris::segment_v2::SegmentWriter::_create_writers(std::shared_ptr const&, std::vector> const&) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:316:9 #10 0x5586662a2be3 in doris::segment_v2::SegmentWriter::init(std::vector> const&, bool) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:285:5 #11 0x5586665333b2 in doris::VerticalBetaRowsetWriter::add_columns(doris::vectorized::Block const*, std::vector> const&, bool, unsigned int) /home/zcp/repo_center/doris_master/doris/be/src/olap/rowset/vertical_beta_rowset_writer.cpp:96:17 #12 0x558665a8cb9b in doris::Merger::vertical_compact_one_group(std::shared_ptr, doris::ReaderType, doris::TabletSchema const&, bool, std::vector> const&, doris::vectorized::RowSourcesBuffer*, std::vector, std::allocator>> const&, doris::RowsetWriter*, long, doris::Merger::Statistics*, std::vector>, long, doris::CompactionSampleInfo*) /home/zcp/repo_center/doris_master/doris/be/src/olap/merger.cpp:309:9 #13 0x558665a93190 in doris::Merger::vertical_merge_rowsets(std::shared_ptr, doris::ReaderType, doris::TabletSchema const&, std::vector, std::allocator>> const&, doris::RowsetWriter*, long, long, doris::Merger::Statistics*) /home/zcp/repo_center/doris_master/doris/be/src/olap/merger.cpp:471:21 #14 0x558665a00fd7 in doris::Compaction::merge_input_rowsets() /home/zcp/repo_center/doris_master/doris/be/src/olap/compaction.cpp:192:19 #15 0x558665a329a3 in doris::CloudCompactionMixin::execute_compact_impl(long) /home/zcp/repo_center/doris_master/doris/be/src/olap/compaction.cpp:1158:5 #16 0x558665a33381 in doris::CloudCompactionMixin::execute_compact() /home/zcp/repo_center/doris_master/doris/be/src/olap/compaction.cpp:1173:5 #17 0x55869a893d89 in doris::CloudCumulativeCompaction::execute_compact() /home/zcp/repo_center/doris_master/doris/be/src/cloud/cloud_cumulative_compaction.cpp:191:38 #18 0x55869a861fac in doris::CloudStorageEngine::_submit_cumulative_compaction_task(std::shared_ptr const&)::$_1::operator()() const /home/zcp/repo_center/doris_master/doris/be/src/cloud/cloud_storage_engine.cpp:693:31 #19 0x55869a861fac in void std::__invoke_impl const&)::$_1&>(std::__invoke_other, doris::CloudStorageEngine::_submit_cumulative_compaction_task(std::shared_ptr const&)::$_1&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 #20 0x55869a861fac in std::enable_if const&)::$_1&>, void>::type std::__invoke_r const&)::$_1&>(doris::CloudStorageEngine::_submit_cumulative_compaction_task(std::shared_ptr const&)::$_1&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2 #21 0x55869a861fac in std::_Function_handler const&)::$_1>::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 #22 0x5586679033fb in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:543:24 #23 0x5586678db057 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:498:5 #24 0x7f0639d7bac2 in start_thread nptl/pthread_create.c:442:8 ```
CalvinKirs
pushed a commit
that referenced
this pull request
Jan 3, 2025
…ache#46296) ### What problem does this PR solve? ```cpp 172.20.50.85 be.out: #8 0x56435579ab5e in lucene::store::BufferedIndexInput::readBytes(unsigned char*, int, bool) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/clucene/src/core/CLucene/store/IndexInput.cpp:192:9 #9 0x564319f7d95d in doris::segment_v2::CSIndexInput::readInternal(unsigned char*, int) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_compound_reader.cpp:107:11 #10 0x56435579bb5b in lucene::store::BufferedIndexInput::refill() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/clucene/src/core/CLucene/store/IndexInput.cpp:285:5 #11 0x56435579b096 in lucene::store::BufferedIndexInput::readBytes(unsigned char*, int, int, bool) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/clucene/src/core/CLucene/store/IndexInput.cpp:217:9 #12 0x56435579a9f6 in lucene::store::BufferedIndexInput::readBytes(unsigned char*, int) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/clucene/src/core/CLucene/store/IndexInput.cpp:186:9 #13 0x564319fe8aee in doris::segment_v2::InvertedIndexReader::read_null_bitmap(doris::io::IOContext const*, doris::OlapReaderStatistics*, doris::segment_v2::InvertedIndexQueryCacheHandle*, lucene::store::Directory*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_reader.cpp:144:29 #14 0x564319febab4 in doris::segment_v2::InvertedIndexReader::handle_searcher_cache(doris::segment_v2::InvertedIndexCacheHandle*, doris::io::IOContext const*, doris::OlapReaderStatistics*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_reader.cpp:194:27 #15 0x564319ff4b0b in doris::segment_v2::StringTypeInvertedIndexReader::query(doris::io::IOContext const*, doris::OlapReaderStatistics*, doris::RuntimeState*, std::__cxx11::basic_string, std::allocator> const&, void const*, doris::segment_v2::InvertedIndexQueryType, std::shared_ptr&) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_reader.cpp:407:5 #16 0x56431a001f00 in doris::segment_v2::InvertedIndexIterator::read_from_inverted_index(std::__cxx11::basic_string, std::allocator> const&, void const*, doris::segment_v2::InvertedIndexQueryType, unsigned int, std::shared_ptr&, bool) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_reader.cpp:1137:5 #17 0x5643191f4a52 in doris::ComparisonPredicateBase<(doris::PrimitiveType)23, (doris::PredicateType)1>::evaluate(std::pair, std::allocator>, std::shared_ptr> const&, doris::segment_v2::InvertedIndexIterator*, unsigned int, roaring::Roaring*) const /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/comparison_predicate.h:107:9 #18 0x56431a249ef7 in doris::segment_v2::SegmentIterator::_apply_inverted_index_on_column_predicate(doris::ColumnPredicate*, std::vector>&, bool*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:871:28 #19 0x56431a23ddc0 in doris::segment_v2::SegmentIterator::_apply_inverted_index() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:951:13 #20 0x56431a22f956 in doris::segment_v2::SegmentIterator::_get_row_ranges_by_column_conditions() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:512:13 #21 0x56431a22b78f in doris::segment_v2::SegmentIterator::_lazy_init() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:383:5 #22 0x56431a26c667 in doris::segment_v2::SegmentIterator::_next_batch_internal(doris::vectorized::Block*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2008:9 #23 0x56431a264fb6 in doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*)::$_0::operator()() const /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1917:9 #24 0x56431a264fb6 in doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1916:19 #25 0x56431a08315c in doris::segment_v2::LazyInitSegmentIterator::next_batch(doris::vectorized::Block*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/lazy_init_segment_iterator.h:44:33 #26 0x564319b95855 in doris::BetaRowsetReader::next_block(doris::vectorized::Block*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/beta_rowset_reader.cpp:366:29 #27 0x56434e3a0c10 in doris::vectorized::VCollectIterator::Level0Iterator::_refresh() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.h #28 0x56434e38ba5b in doris::vectorized::VCollectIterator::Level0Iterator::refresh_current_row() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:509:24 #29 0x56434e38cd76 in doris::vectorized::VCollectIterator::Level0Iterator::ensure_first_row_ref() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:482:14 #30 0x56434e399745 in doris::vectorized::VCollectIterator::Level1Iterator::ensure_first_row_ref() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:696:27 #31 0x56434e37f7f9 in doris::vectorized::VCollectIterator::build_heap(std::vector, std::allocator>>&) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:186:9 #32 0x56434e312fe2 in doris::vectorized::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/block_reader.cpp:140:5 #33 0x56434e3165df in doris::vectorized::BlockReader::init(doris::TabletReader::ReaderParams const&) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/block_reader.cpp:212:19 #34 0x5643515aba04 in doris::vectorized::NewOlapScanner::open(doris::RuntimeState*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/new_olap_scanner.cpp:231:32 #35 0x564336c138c6 in doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr, std::shared_ptr) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:247:5 #36 0x564336c18db0 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()::operator()() const::'lambda0'()::operator()() const /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:180:21 #37 0x564336c18db0 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()::operator()() const /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:179:31 #38 0x564336c18db0 in void std::__invoke_impl, std::shared_ptr)::$_1::operator()() const::'lambda'()&>(std::__invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 #39 0x564336c18db0 in std::enable_if, std::shared_ptr)::$_1::operator()() const::'lambda'()&>, void>::type std::__invoke_r, std::shared_ptr)::$_1::operator()() const::'lambda'()&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2 #40 0x564336c18db0 in std::_Function_handler, std::shared_ptr)::$_1::operator()() const::'lambda'()>::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 #41 0x56431b9d94eb in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/threadpool.cpp:543:24 #42 0x56431b9b18c7 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/thread.cpp:498:5 #43 0x7f31d3b87ac2 in start_thread nptl/pthread_create.c:442:8 #44 0x7f31d3c1984f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81 0x61d000a7f3c8 is located 1352 bytes inside of 2064-byte region [0x61d000a7ee80,0x61d000a7f690) freed by thread T1698 (Pipe_normal [wo) here: #0 0x564317278d9d in operator delete(void*) (/mnt/hdd01/ci/doris-deploy-branch-3.0-cloud/be/lib/doris_be+0x3626bd9d) (BuildId: 9dab40b94b1dc995) #1 0x5643515b192c in std::unique_ptr>::reset(doris::TabletReader*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:456:7 #2 0x5643515b192c in doris::vectorized::NewOlapScanner::close(doris::RuntimeState*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/new_olap_scanner.cpp:534:20 #3 0x564352457463 in doris::vectorized::ScannerDelegate::~ScannerDelegate() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/vscan_node.h:35:31 #4 0x5643172a8324 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:168:6 #5 0x56434e7d7870 in std::__shared_ptr::~__shared_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1149:31 #6 0x56434e7d7870 in void std::destroy_at>(std::shared_ptr*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15 #7 0x56434e7d7870 in void std::allocator_traits>>>::destroy>(std::allocator>>&, std::shared_ptr*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:533:4 #8 0x56434e7d7870 in std::__cxx11::_List_base, std::allocator>>::_M_clear() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/list.tcc:77:4 #9 0x564351db11d2 in std::__cxx11::_List_base, std::allocator>>::~_List_base() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_list.h:499:9 #10 0x564351db11d2 in doris::pipeline::ScanLocalState::close(doris::RuntimeState*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/exec/scan_operator.cpp:1261:5 #11 0x56434e6357e3 in doris::pipeline::OperatorXBase::close(doris::RuntimeState*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/exec/operator.cpp:245:28 #12 0x564352ad00c1 in doris::pipeline::PipelineTask::close(doris::Status) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/pipeline_task.cpp:489:28 #13 0x564352b0701f in doris::pipeline::_close_task(doris::pipeline::PipelineTask*, doris::Status) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/task_scheduler.cpp:91:27 #14 0x564352b095f3 in doris::pipeline::TaskScheduler::_do_work(unsigned long) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/task_scheduler.cpp:181:17 #15 0x56431b9d94eb in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/threadpool.cpp:543:24 #16 0x56431b9b18c7 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/thread.cpp:498:5 #17 0x7f31d3b87ac2 in start_thread nptl/pthread_create.c:442:8 ``` Problem Summary: ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed changes
Issue Number: close #xxx
Further comments
If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...