Skip to content

Commit

Permalink
[fix](doc) spell errors fixes for memory tracker docs (apache#27865)
Browse files Browse the repository at this point in the history
  • Loading branch information
Nitin-Kashyap authored Dec 27, 2023
1 parent d3e1afa commit 8035287
Show file tree
Hide file tree
Showing 4 changed files with 7 additions and 7 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ Notice:
2. When the process memory exceeds the limit, BE will trigger memory GC.

```
W1127 17:23:16.372572 19896 mem_tracker_limiter.cpp:214] System Mem Exceed Limit Check Faild, Try Alloc: 1062688
W1127 17:23:16.372572 19896 mem_tracker_limiter.cpp:214] System Mem Exceed Limit Check Failed, Try Alloc: 1062688
Process Memory Summary:
process memory used 2.68 GB limit 2.47 GB, sys mem available 50.95 GB min reserve 3.20 GB, tc/jemalloc allocator cache 51.97 MB
Alloc Stacktrace:
Expand Down Expand Up @@ -116,13 +116,13 @@ The low water mark defaults to a maximum of 1.6G, calculated based on `MemTotal`
## Query or import a single execution memory limit
When the following error is returned, it means that the memory limit of a single execution has been exceeded.
```
ERROR 1105 (HY000): errCode = 2, detailMessage = Memory limit exceeded:<consuming tracker:<Query#Id=f78208b15e064527-a84c5c0b04c04fcf>, failed alloc size 1.03 MB, exceeded tracker:<Query#Id=f78208b15e064527-a84c5c0b04c04fcf>, limit 100.00 MB, peak used 99.29 MB, current used 99.25 MB>, executing msg:<execute:<ExecNode:VHASH_JOIN_NODE (id=4)>>. backend 172.24.47.117 process memory used 1.13 GB, limit ex 98.92 GB cery tracker If , `set exec_mem_limit=8G` to change limit, details mem usage see log/be.INFO.
ERROR 1105 (HY000): errCode = 2, detailMessage = Memory limit exceeded:<consuming tracker:<Query#Id=f78208b15e064527-a84c5c0b04c04fcf>, failed alloc size 1.03 MB, exceeded tracker:<Query#Id=f78208b15e064527-a84c5c0b04c04fcf>, limit 100.00 MB, peak used 99.29 MB, current used 99.25 MB>, executing msg:<execute:<ExecNode:VHASH_JOIN_NODE (id=4)>>. backend 172.24.47.117 process memory used 1.13 GB, limit 98.92 GB. If query tracker exceed, `set exec_mem_limit=8G` to change limit, details mem usage see log/be.INFO
```

### Error message analysis
The error message is divided into three parts:
1. `Memory limit exceeded:<consuming tracker:<Query#Id=f78208b15e064527-a84c5c0b04c04fcf>`: It is found that the memory limit is exceeded during the memory application process of query `f78208b15e064527-a84c5c0b04c04fcf`.
2. `failed alloc size 1.03 MB, exceeded tracker:<Query#Id=f78208b15e064527-a84c5c0b04c04fcf>, limit 100.00 MB, peak used 99.29 MB, current used 99.25 MB`: The memory requested this time is 1.03 MB The current consumption of `f78208b15e064527-a84c5c0b04c04fcf` memory tracker is 99.28MB plus 1.03MB, which exceeds the limit of 100MB. The value of limit comes from `exec_mem_limit` in session veriables, and the default is 4G.
2. `failed alloc size 1.03 MB, exceeded tracker:<Query#Id=f78208b15e064527-a84c5c0b04c04fcf>, limit 100.00 MB, peak used 99.29 MB, current used 99.25 MB`: The memory requested this time is 1.03 MB The current consumption of `f78208b15e064527-a84c5c0b04c04fcf` memory tracker is 99.28MB plus 1.03MB, which exceeds the limit of 100MB. The value of limit comes from `exec_mem_limit` in session variables, and the default is 4G.
3. `executing msg:<execute:<ExecNode:VHASH_JOIN_NODE (id=4)>>. backend 172.24.47.117 process memory used 1.13 GB, limit 98.92 GB. If query tracker exceeds, `set exec_mem_limit=8G` to change limit, details mem usage see be.INFO.`: The location of this memory application is `VHASH_JOIN_NODE (id=4)`, and it prompts that `set exec_mem_limit` can be used to increase the memory limit of a single query.

### Log Analysis
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ For the memory statistics results of historical queries, you can view the `peakM
- StorageEngine:, the memory consumed by the storage engine during loading the data directory, a subset of Orphan.
- SegCompaction: The memory sum of all SegCompaction tasks, a subset of Orphan.
- SegmentMeta: memory use by segment meta data such as footer or index page, a subset of Orphan.
- TabletManager: The memory consumed by the storage engine get, add, and delte Tablet, a subset of Orphan.
- TabletManager: The memory consumed by the storage engine get, add, and delete Tablet, a subset of Orphan.
- BufferAllocator: Only used for memory multiplexing in the non-vectorized Partitioned Agg process, a subset of Orphan.

- DataPageCache: Used to cache data Pages to speed up Scan.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ ERROR 1105 (HY000): errCode = 2, detailMessage = Memory limit exceeded:<consumin
2、当进程内存超限后,BE会触发内存GC。

```
W1127 17:23:16.372572 19896 mem_tracker_limiter.cpp:214] System Mem Exceed Limit Check Faild, Try Alloc: 1062688
W1127 17:23:16.372572 19896 mem_tracker_limiter.cpp:214] System Mem Exceed Limit Check Failed, Try Alloc: 1062688
Process Memory Summary:
process memory used 2.68 GB limit 2.47 GB, sys mem available 50.95 GB min reserve 3.20 GB, tc/jemalloc allocator cache 51.97 MB
Alloc Stacktrace:
Expand Down Expand Up @@ -122,7 +122,7 @@ ERROR 1105 (HY000): errCode = 2, detailMessage = Memory limit exceeded:<consumin
### 错误信息分析
错误信息分为三部分:
1、`Memory limit exceeded:<consuming tracker:<Query#Id=f78208b15e064527-a84c5c0b04c04fcf>`:当前正在执行query `f78208b15e064527-a84c5c0b04c04fcf`的内存申请过程中发现内存超限。
2、`failed alloc size 1.03 MB, exceeded tracker:<Query#Id=f78208b15e064527-a84c5c0b04c04fcf>, limit 100.00 MB, peak used 99.29 MB, current used 99.25 MB`:本次尝试申请 1.03MB 的内存,但此时query `f78208b15e064527-a84c5c0b04c04fcf` memory tracker 的当前 consumption 为 99.28MB 加上 1.03MB 后超过了 100MB 的limit,limit的值来自 session veriables 中的 `exec_mem_limit`,默认4G。
2、`failed alloc size 1.03 MB, exceeded tracker:<Query#Id=f78208b15e064527-a84c5c0b04c04fcf>, limit 100.00 MB, peak used 99.29 MB, current used 99.25 MB`:本次尝试申请 1.03MB 的内存,但此时query `f78208b15e064527-a84c5c0b04c04fcf` memory tracker 的当前 consumption 为 99.28MB 加上 1.03MB 后超过了 100MB 的limit,limit的值来自 session variables 中的 `exec_mem_limit`,默认4G。
3、`executing msg:<execute:<ExecNode:VHASH_JOIN_NODE (id=4)>>. backend 172.24.47.117 process memory used 1.13 GB, limit 98.92 GB. If query tracker exceed, `set exec_mem_limit=8G` to change limit, details mem usage see be.INFO.`:本次内存申请的位置是`VHASH_JOIN_NODE (id=4)`,并提示可通过 `set exec_mem_limit` 来调高单次查询的内存上限。

### 日志分析
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ https://shimo.im/docs/DT6JXDRkdTvdyV3G
- StorageEngine:,存储引擎加载数据目录过程中消耗的内存,Orphan的子集。
- SegCompaction: 所有 SegCompaction 任务的内存总和,Orphan的子集。
- SegmentMeta: memory use by segment meta data such as footer or index page,Orphan的子集。
- TabletManager: 存储引擎 get、add、delte Tablet 过程中消耗的内存,Orphan的子集。
- TabletManager: 存储引擎 get、add、delete Tablet 过程中消耗的内存,Orphan的子集。
- BufferAllocator: 仅用于非向量化Partitioned Agg过程中的内存复用,Orphan的子集。

- DataPageCache: 用于缓存数据 Page,用于加速 Scan。
Expand Down

0 comments on commit 8035287

Please sign in to comment.