一个小表1条数据,大表5000W,left join查询为什么会Memory exceed limit,exec_mem_limit我给了18G。 #7672
Unanswered
WesZhang100
asked this question in
Q&A
Replies: 1 comment
-
兄弟,解决了吗, 我也遇到了这个问题 |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
如题,语句很简单: SELECT * FROM Web_Query_ByUpload
LEFT JOIN
SMT_FVS
ON Web_Query_ByUpload.Data
=SMT_FVS
.SerialNumber
Queryproflie如下
Query:
Summary:
- Query ID: 4d40391f69b24ab1-b2986c1496c016bc
- Start Time: 2022-01-07 21:36:31
- End Time: N/A
- Total: 4s400ms
- Query Type: Query
- Query State: RUNNING
- Doris Version: 0.15.1-rc09
- User: root
- Default Db: default_cluster:mdw
- Sql Statement: SELECT * FROM Web_Query_ByUpload
LEFT JOIN
SMT_FVS
ON Web_Query_ByUpload.Data
=SMT_FVS
.SerialNumber
- Is Cached: No
Execution Summary:
- Analysis Time: 591.47us
- Plan Time: 1.543ms
- Schedule Time: 8.843ms
- Wait and Fetch Result Time: N/A
Execution Profile 4d40391f69b24ab1-b2986c1496c016bc:(Active: 4s399ms, % non-child: 100.00%)
Fragment 0:
Instance 4d40391f69b24ab1-b2986c1496c016c0 (host=TNetworkAddress(hostname:172.19.250.160, port:9060)):(Active: 4s391ms, % non-child: 0.00%)
- FragmentCpuTime: 55.703us
- MemoryLimit: 8.00 GB
- PeakMemoryUsage: 16.00 KB
- PeakReservation: 0.00
- PeakUsedReservation: 0.00
- RowsProduced: 0
BlockMgr:
- BlockWritesOutstanding: 0
- BlocksCreated: 0
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0.00
- MaxBlockSize: 8.00 MB
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
DataBufferSender (dst_fragment_instance_id=4d40391f69b24ab1-b2986c1496c016c0):
- AppendBatchTime: 0ns
- ResultSendTime: 0ns
- TupleConvertTime: 0ns
- NumSentRows: 0
EXCHANGE_NODE (id=3):(Active: 4s391ms, % non-child: 99.82%)
- BytesReceived: 0.00
- ConvertRowBatchTime: 0ns
- DataArrivalWaitTime: 0ns
- DeserializeRowBatchTimer: 0ns
- FirstBatchArrivalWaitTime: 0ns
- PeakMemoryUsage: 0.00
- RowsReturned: 0
- RowsReturnedRate: 0
- SendersBlockedTotalTimer(*): 0ns
Fragment 1:
Instance 4d40391f69b24ab1-b2986c1496c016bf (host=TNetworkAddress(hostname:172.16.50.46, port:9060)):(Active: 4s383ms, % non-child: 0.00%)
- FragmentCpuTime: 240.473us
- MemoryLimit: 8.00 GB
- PeakMemoryUsage: 8.00 GB
- PeakReservation: 0.00
- PeakUsedReservation: 0.00
- RowsProduced: 0
BlockMgr:
- BlockWritesOutstanding: 0
- BlocksCreated: 0
- BlocksRecycled: 0
- BufferedPins: 0
- BytesWritten: 0.00
- MaxBlockSize: 8.00 MB
- TotalBufferWaitTime: 0ns
- TotalEncryptionTime: 0ns
- TotalIntegrityCheckTime: 0ns
- TotalReadBlockTime: 0ns
DataStreamSender (dst_id=3, dst_fragments=[4d40391f69b24ab1-b2986c1496c016c0]):(Active: 42.519us, % non-child: 0.00%)
- BytesSent: 0.00
- IgnoreRows: 0
- LocalBytesSent: 0.00
- OverallThroughput: 0.0 /sec
- PeakMemoryUsage: 544.00 B
- SerializeBatchTime: 0ns
- UncompressedRowBatchSize: 0.00
HASH_JOIN_NODE (id=2):(Active: 4s383ms, % non-child: 91.72%)
- ExecOption: Hash Table Built Asynchronously
- BuildBuckets: 4.194304M (4194304)
- BuildRows: 8.048766M (8048766)
- BuildTime: 4s379ms
- HashTableMaxList: 123
- HashTableMinList: 1
- LoadFactor: 4603574526012293100.00
- PeakMemoryUsage: 8.00 GB
- ProbeRows: 0
- ProbeTime: 0ns
- PushDownComputeTime: 0ns
- PushDownTime: 0ns
- RowsReturned: 0
- RowsReturnedRate: 0
OLAP_SCAN_NODE (id=1):(Active: 348.423ms, % non-child: 7.92%)
- BatchQueueWaitTime: 70.830ms
- BytesRead: 2.09 GB
- NumDiskAccess: 1
- NumScanners: 78
- PeakMemoryUsage: 0.00
- RowsRead: 6.176886M (6176886)
- RowsReturned: 8.04979M (8049790)
- RowsReturnedRate: 23.103434M /sec
- ScannerBatchWaitTime: 361.445ms
- ScannerWorkerWaitTime: 22s348ms
- TabletCount : 78
- TotalReadThroughput: 0.0 /sec
OlapScanner(SMT_FVS):
- BlockConvertTime: 5s434ms
- BlockFetchTime: 27s854ms
- ReaderInitTime: 4s933ms
- RowsDelFiltered: 0
- RowsPushedCondFiltered: 0
- ScanCpuTime: 1m18s
- ScanTime: 1m33s
- ShowHintsTime_V1: 0ns
SegmentIterator:
- BitmapIndexFilterTimer: 31.272us
- BlockLoadTime: 11s51ms
- BlockSeekCount: 629
- BlockSeekTime: 1s720ms
- BlocksLoad: 6.063K (6063)
- CachedPagesNum: 829
- CompressedBytesRead: 360.86 MB
- DecompressorTimer: 4s99ms
- IOTimer: 226.342ms
- IndexLoadTime_V1: 0ns
- NumSegmentFiltered: 0
- NumSegmentTotal: 68
- RawRowsRead: 13.29718M (13297180)
- RowsBitmapIndexFiltered: 0
- RowsBloomFilterFiltered: 0
- RowsConditionsFiltered: 0
- RowsKeyRangeFiltered: 0
- RowsStatsFiltered: 0
- RowsVectorPredFiltered: 0
- TotalPagesNum: 12.196K (12196)
- UncompressedBytesRead: 256.79 MB
- VectorPredEvalTime: 3.226ms
OLAP_SCAN_NODE (id=0):(Active: 2.869us, % non-child: 0.00%)
- BatchQueueWaitTime: 0ns
- BytesRead: 0.00
- NumDiskAccess: 0
- NumScanners: 0
- PeakMemoryUsage: 0.00
- RowsRead: 0
- RowsReturned: 0
- RowsReturnedRate: 0
- ScannerBatchWaitTime: 0ns
- ScannerWorkerWaitTime: 0ns
- TabletCount : 3
- TotalReadThroughput: 0.0 /sec
OlapScanner(Web_Query_ByUpload):
- BlockConvertTime: 0ns
- BlockFetchTime: 0ns
- ReaderInitTime: 0ns
- RowsDelFiltered: 0
- RowsPushedCondFiltered: 0
- ScanCpuTime: 0ns
- ScanTime: 0ns
- ShowHintsTime_V1: 0ns
SegmentIterator:
- BitmapIndexFilterTimer: 0ns
- BlockLoadTime: 0ns
- BlockSeekCount: 0
- BlockSeekTime: 0ns
- BlocksLoad: 0
- CachedPagesNum: 0
- CompressedBytesRead: 0.00
- DecompressorTimer: 0ns
- IOTimer: 0ns
- IndexLoadTime_V1: 0ns
- NumSegmentFiltered: 0
- NumSegmentTotal: 0
- RawRowsRead: 0
- RowsBitmapIndexFiltered: 0
- RowsBloomFilterFiltered: 0
- RowsConditionsFiltered: 0
- RowsKeyRangeFiltered: 0
- RowsStatsFiltered: 0
- RowsVectorPredFiltered: 0
- TotalPagesNum: 0
- UncompressedBytesRead: 0.00
- VectorPredEvalTime: 0ns
Instance 4d40391f69b24ab1-b2986c1496c016bd (host=TNetworkAddress(hostname:172.19.250.160, port:9060)):
Instance 4d40391f69b24ab1-b2986c1496c016be (host=TNetworkAddress(hostname:172.19.250.161, port:9060)):
执行计划:
PLAN FRAGMENT 0
OUTPUT EXPRS:
default_cluster:mdw
.Web_Query_ByUpload
.TransTime
| `default_clustPARTITION: UNPARTITIONED
RESULT SINK
3:EXCHANGE
PLAN FRAGMENT 1
OUTPUT EXPRS:
PARTITION: HASH_PARTITIONED:
default_cluster:mdw
.Web_Query_ByUpload
.Data
STREAM DATA SINK
EXCHANGE ID: 03
UNPARTITIONED
2:HASH JOIN
| join op: LEFT OUTER JOIN
| hash predicates:
| colocate: true
| equal join conjunct:
Data
=SerialNumber
| cardinality=0
|
|----1:OlapScanNode
| TABLE: SMT_FVS
| PREAGGREGATION: OFF. Reason: null
| partitions=26/49
| rollup: SMT_FVS
| tabletRatio=260/260
| tabletList=3051640,3051644,3051648,3051652,3051656,3051660,3051664,3051668,
| cardinality=50746706
| avgRowSize=30.245388
| numNodes=3
|
0:OlapScanNode
TABLE: Web_Query_ByUpload
PREAGGREGATION: OFF. Reason: No AggregateInfo
PREDICATES:
default_cluster:mdw.Web_Query_ByUpload
.__DORIS_DELETE_SIGN__
= 0partitions=1/1
rollup: Web_Query_ByUpload
tabletRatio=10/10
tabletList=4915910,4915914,4915918,4915922,4915926,4915930,4915934,4915938,49159
cardinality=0
avgRowSize=81.0
numNodes=1
请求帮助~~
Beta Was this translation helpful? Give feedback.
All reactions