From 8553af3de9d22789a720d9b465accbaba1b3b0bd Mon Sep 17 00:00:00 2001 From: winlin Date: Tue, 20 Aug 2024 17:50:59 +0800 Subject: [PATCH] Refine valgrind doc. --- .../current/doc/performance.md | 29 +++++++++++++++++++ .../current/doc/performance.md | 29 ++++++++++++++++++- 2 files changed, 57 insertions(+), 1 deletion(-) diff --git a/i18n/en-us/docusaurus-plugin-content-docs/current/doc/performance.md b/i18n/en-us/docusaurus-plugin-content-docs/current/doc/performance.md index 30a7320a..c6ba8a4b 100644 --- a/i18n/en-us/docusaurus-plugin-content-docs/current/doc/performance.md +++ b/i18n/en-us/docusaurus-plugin-content-docs/current/doc/performance.md @@ -255,6 +255,35 @@ valgrind --leak-check=full ./objs/srs -c conf/console.conf > Remark: For ST to support valgrind, see [state-threads](https://github.com/ossrs/state-threads#usage) and [ST#2](https://github.com/ossrs/state-threads/issues/2). +To use Valgrind to detect memory leaks in SRS, even though Valgrind hooks are supported in ST, there are +still many false positives. A more reasonable approach is to have Valgrind report incremental memory leaks. +This way, global and static variables can be avoided, and detection can be achieved without exiting the +program. Follow these steps: + +1. Compile SRS with Valgrind support: `./configure --valgrind=on && make` +1. Start SRS with memory leak detection enabled: `valgrind --leak-check=full ./objs/srs -c conf/console.conf` +1. Trigger memory detection by using curl to access the API and generate calibration data. There will still be many false positives, but these can be ignored: `curl http://127.0.0.1:1985/api/v1/valgrind?check=added` +1. Perform load testing or test the suspected leaking functionality, such as RTMP streaming: `ffmpeg -re -i doc/source.flv -c copy -f flv rtmp://127.0.0.1/live/livestream` +1. Stop streaming and wait for SRS to clean up the Source memory, approximately 30 seconds. +1. Perform incremental memory leak detection. The reported leaks will be very accurate at this point: `curl http://127.0.0.1:1985/api/v1/valgrind?check=added` + +```text +HTTP #0 11.176.19.95:42162 GET http://9.134.74.169:1985/api/v1/valgrind?check=added, content-length=-1 +query check=added +==1481822== LEAK SUMMARY: +==1481822== definitely lost: 0 (+0) bytes in 0 (+0) blocks +==1481822== indirectly lost: 0 (+0) bytes in 0 (+0) blocks +==1481822== possibly lost: 3,406,847 (+0) bytes in 138 (+0) blocks +==1481822== still reachable: 18,591,709 (+0) bytes in 819 (+0) blocks +==1481822== of which reachable via heuristic: +==1481822== multipleinheritance: 536 (+0) bytes in 4 (+0) blocks +==1481822== suppressed: 0 (+0) bytes in 0 (+0) blocks +==1481822== Reachable blocks (those to which a pointer was found) are not shown. +==1481822== To see them, rerun with: --leak-check=full --show-leak-kinds=all +``` + +> Note: It is recommended to use a browser to access `/api/v1/valgrind` because the browser uses long connections, whereas curl uses short connections, which can generate some memory objects and might cause some interference. + ## Syscall Please use [strace -c -p PID](https://man7.org/linux/man-pages/man1/strace.1.html) for syscal performance issue. diff --git a/i18n/zh-cn/docusaurus-plugin-content-docs/current/doc/performance.md b/i18n/zh-cn/docusaurus-plugin-content-docs/current/doc/performance.md index d2293fcf..e0cc7105 100644 --- a/i18n/zh-cn/docusaurus-plugin-content-docs/current/doc/performance.md +++ b/i18n/zh-cn/docusaurus-plugin-content-docs/current/doc/performance.md @@ -250,7 +250,7 @@ killall -2 srs ## VALGRIND -VALGRIND是大名鼎鼎的C分析工具，SRS3之后支持了。SRS3之前，因为使用了ST，需要给ST打PATCH才能用。 +VALGRIND是大名鼎鼎的内存分析工具，SRS3之后支持了。SRS3之前，因为使用了ST，需要给ST打PATCH才能用。 ``` valgrind --leak-check=full ./objs/srs -c conf/console.conf @@ -258,6 +258,33 @@ valgrind --leak-check=full ./objs/srs -c conf/console.conf > Remark: SRS3之前的版本，可以手动给ST打PATCH支持VALGRIND，参考[state-threads](https://github.com/ossrs/state-threads#usage)，详细的信息可以参考[ST#2](https://github.com/ossrs/state-threads/issues/2)。 +使用valgrind检测SRS内存泄露时，尽管在ST中支持了valgrind的hook，还是有大量的误报信息。比较合理的是让valgrind报告增量的内存泄露， +这样可以避开全局和静态变量，也可以不用退出程序就可以实现检测。操作步骤如下： + +1. 编译SRS支持valgrind：`./configure --valgrind=on && make` +1. 启动SRS，开启内存泄露的检测：`valgrind --leak-check=full ./objs/srs -c conf/console.conf` +1. 触发内存检测，使用curl访问API，形成校准的数据，依然有大量误报，但可以忽略这些数据：`curl http://127.0.0.1:1985/api/v1/valgrind?check=added` +1. 压测，或者针对怀疑泄露的功能测试，比如RTMP推流：`ffmpeg -re -i doc/source.flv -c copy -f flv rtmp://127.0.0.1/live/livestream` +1. 停止推流，等待SRS清理Source内存，大概等待30秒左右。 +1. 增量内存泄露检测，此时报告的泄露情况就非常准确了：`curl http://127.0.0.1:1985/api/v1/valgrind?check=added` + +```text +HTTP #0 11.176.19.95:42162 GET http://9.134.74.169:1985/api/v1/valgrind?check=added, content-length=-1 +query check=added +==1481822== LEAK SUMMARY: +==1481822== definitely lost: 0 (+0) bytes in 0 (+0) blocks +==1481822== indirectly lost: 0 (+0) bytes in 0 (+0) blocks +==1481822== possibly lost: 3,406,847 (+0) bytes in 138 (+0) blocks +==1481822== still reachable: 18,591,709 (+0) bytes in 819 (+0) blocks +==1481822== of which reachable via heuristic: +==1481822== multipleinheritance: 536 (+0) bytes in 4 (+0) blocks +==1481822== suppressed: 0 (+0) bytes in 0 (+0) blocks +==1481822== Reachable blocks (those to which a pointer was found) are not shown. +==1481822== To see them, rerun with: --leak-check=full --show-leak-kinds=all +``` + +> Note: 推荐使用浏览器访问`/api/v1/valgrind`，因为浏览器是长链接，而curl是短链接，本身会产生一些内存对象，可能会有些干扰。 + ## Syscall 系统调用的性能排查，参考[strace -c -p PID](https://man7.org/linux/man-pages/man1/strace.1.html)