Merge remote-tracking branch 'upstream/master' into 6.0/stage #13
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fix compilation warnings.
Several existing kernel version checks are intended to distinguish 5.11+ kernels vs 5.10 and earlier, but as written, 5.10.1+ kernels will take the wrong path. Update the checks to match the intent expressed in their accompanying comments. Signed-off-by: Connor O'Brien <[email protected]>
Remove extra whitespace at the end of a line.
libbpf-tools: fix kernel version checks
Signed-off-by: Sohaib Mohamed <[email protected]>
Add Jammy(Ubuntu 22.04) package dependency guide.
libbpf-tools/execsnoop: fixup: drop unused define
…ll-guide Add Ubuntu 22.04 package dependency guide
Signed-off-by: Hengqi Chen <[email protected]>
tools/tcplife: Remove dead code
/home/rongtao/Git/bcc/src/cc/bpf_module_rw_engine.cc: In member function ‘int ebpf::BPFModule::annotate()’:
/home/rongtao/Git/bcc/src/cc/bpf_module_rw_engine.cc:419:63: warning: ‘llvm::Type* llvm::PointerType::getElementType() const’ is deprecated: Pointer element types are deprecated. You can *temporarily* use Type::getPointerElementType() instead [-Wdeprecated-declarations]
419 | StructType *st = dyn_cast<StructType>(pt->getElementType());
| ~~~~~~~~~~~~~~~~~~^~
In file included from /usr/include/llvm/IR/DataLayout.h:27,
from /usr/include/llvm/ExecutionEngine/ExecutionEngine.h:24,
from /usr/include/llvm/ExecutionEngine/MCJIT.h:17,
from /home/rongtao/Git/bcc/src/cc/bpf_module_rw_engine.cc:20:
/usr/include/llvm/IR/DerivedTypes.h:675:9: note: declared here
675 | Type *getElementType() const {
| ^~~~~~~~~~~~~~
See llvm: [OpaquePtrs] Deprecate PointerType::getElementType()
llvm/llvm-project@184591a
Belongs to release/14.x branch.
Use task->real_parent instead of task->parent (when one process being traced, the tracer becomes its parent, so use task->real_parent is more accurate). Unify PID column width (at most 7 chars) #3915, try to unify PID/PPID/TID column width (at most 7 chars).
Sometimes, I just want to focus on a specified disk rather than all disks or per-disk. Refer to libbpf-tools/biolatency, this patch try to add disk filter support.
Signed-off-by: Hengqi Chen <[email protected]>
sync libbpf repo to 4eb6485c0886 Makefile: add support for cross compilation Signed-off-by: Yonghong Song <[email protected]>
In cross compiling case, we need target toolchain for application code, and host toolchain for bpftool. Signed-off-by: Jie Wang <[email protected]>
When the output is redirected to a file, it'd be better flush the output for each iteration.
Instead of nsec, it can show the time with unit like below:
$ sudo klockstat -n 5
Tracing mutex/sem lock events... Hit Ctrl-C to end
^C
Caller Avg Wait Count Max Wait Total Wait
iwl_mvm_mac_sta_statistics+0x6b 703.9 us 4 2.5 ms 2.8 ms
i915_vma_pin_ww+0x6ff 1.0 us 5224 1.8 ms 5.2 ms
do_epoll_wait+0x1d5 1.9 us 8569 651.3 us 16.0 ms
kernfs_dop_revalidate+0x35 1.1 us 1176 540.2 us 1.3 ms
kernfs_iop_permission+0x2a 1.0 us 1512 528.6 us 1.5 ms
Caller Avg Hold Count Max Hold Total Hold
__fdget_pos+0x42 13.4 ms 201 19.7 ms 2.7 s
genl_rcv+0x15 1.8 ms 8 6.3 ms 14.3 ms
nl80211_pre_doit+0xdb 1.9 ms 4 6.2 ms 7.4 ms
ieee80211_get_station+0x2a 1.8 ms 4 6.2 ms 7.1 ms
bpf_tracing_prog_attach+0x264 2.9 ms 15 3.8 ms 43.4 ms
Exiting trace of mutex/sem locks
The --per-thread option is to aggregate the lock stats per thread
instead of per callstack. The result is like below:
$ sudo klockstat -n 5 -P
Tracing mutex/sem lock events... Hit Ctrl-C to end
^C
Tid Comm Avg Wait Count Max Wait Total Wait
366434 kworker/u17:1 273.3 us 18 3.0 ms 4.9 ms
4286 Chrome_ChildIOT 1.5 us 335 57.4 us 488.3 us
4325 VizCompositorTh 1.1 us 751 20.5 us 817.0 us
4324 Chrome_ChildIOT 1.3 us 332 18.2 us 443.7 us
92900 Web Content 1.5 us 45 16.0 us 67.8 us
Tid Comm Avg Hold Count Max Hold Total Hold
1056 in:imklog 4.0 ms 349 20.3 ms 1.4 s
366519 kworker/u17:3 605.8 us 42 15.3 ms 25.4 ms
368783 klockstat 1.0 ms 180 2.8 ms 184.3 ms
4250 Chrome_IOThread 8.4 us 342 1.5 ms 2.9 ms
2916 gnome-shell 2.6 us 773 383.4 us 2.0 ms
Exiting trace of mutex/sem locks
Signed-off-by: Hengqi Chen <[email protected]>
Total same as [slabratetop(8)](https://github.com/iovisor/bcc/blob/master/tools/slabratetop.py), but add some options params: ```bash $ sudo ./slabratetop -h Usage: slabratetop [OPTION...] Trace slab kmem cache alloc by process. USAGE: slabratetop [-h] [-p PID] [interval] [count] EXAMPLES: slabratetop # slab rate top, refresh every 1s slabratetop -p 181 # only trace PID 181 slabratetop -s count # sort columns by count slabratetop -r 100 # print 100 rows slabratetop 5 10 # 5s summaries, 10 times -C, --noclear Don't clear the screen -p, --pid=PID Process ID to trace -r, --rows=ROWS Maximum rows to print, default 20 -s, --sort=SORT Sort columns, default size [name, count, size] -v, --verbose Verbose debug output -?, --help Give this help list --usage Give a short usage message -V, --version Print program version Mandatory or optional arguments to long options are also mandatory or optional for any corresponding short options. Report bugs to https://github.com/iovisor/bcc/tree/master/libbpf-tools. ``` print like: ```bash 13:29:26 loadavg: 0.06 0.01 0.02 1/556 9961 CACHE ALLOCS BYTES names_cache 86 374272 filp 43 27520 vm_area_struct 87 22968 anon_vma 32 8704 mm_struct 3 5760 anon_vma_chain 55 5280 cred_jar 19 4864 dentry 12 4704 page->ptl 44 4576 UNIX 2 4224 proc_inode_cache 3 3744 xfs_trans 11 3256 skbuff_head_cache 8 2560 seq_file 8 2496 xfs_log_ticket 10 2480 sighand_cache 1 2368 signal_cache 1 1792 xfs_btree_cur 5 1440 sock_inode_cache 1 1408 lsm_file_cache 43 1376 ```
The tools biosnoop and biostacks are broken due to kernel change ([0]).
blk_account_io_{start, done} were renamed to __blk_account_io_{start, done},
and the symbols gone from vmlinux BTF. Fix them by checking symbol existence.
[0]: torvalds/linux@be6bfe3
Signed-off-by: Hengqi Chen <[email protected]>
bcc: add method to close file descriptors
Using hash maps with BPF_F_NO_PREALLOC flag triggers a warning ([0]), and according to kernel [commit 94dacdbd5d2d](torvalds/linux@94dacdbd5d2d), this may cause deadlocks. Remove the flag from libbpf tools. [0]: https://github.com/torvalds/linux/blob/v5.18/kernel/bpf/verifier.c#L11972-L12000 Signed-off-by: Hengqi Chen <[email protected]>
Remove executable permission for 'funcinterval.8'.
There is no conflict in '-P'&'-p PID', fix the commit(508d969) tools/runqslower: add '-P' optional Signed-off-by: zhenwei pi <[email protected]>
…ux/bpf.h When building bcc with a relatively new packaged libbpf (0.8.1) and -DCMAKE_USE_LIBBPF_PACKAGE:BOOL=TRUE, multiple compilation failures are encountered due the fact the system uapi header in /usr/include/linux/bpf.h is not very recent (this is often the case for distros, which sync it via a kernel headers package quite conservatively due to use by glibc). With libbpf built via git submodule, the uapi header included in the libbpf package is used, so here a similar approach is proposed for the external package build. Instead of having to sync another file the already present compat/linux/virtual_bpf.h is used; we copy it to compat/linux/bpf.h (eliminating the string prefix/suffix on first/last lines). From there, we ensure that places that assume the presence of the libbpf git submodule point at compat/ as a location to find the uapi header. Signed-off-by: Alan Maguire <[email protected]>
For package builds, compat/linux/bpf.h is generated from compat/linux/virtual_bpf.h, so add it to .gitignore. Suggested-by: Hengqi Chen <[email protected]> Signed-off-by: Alan Maguire <[email protected]>
We have encountered an issue while running runqslower for long run test, as you can check the log below. The workload of target platform is running auto test script and launching lots of applications periodically on Android TV. There are several abnormal records of run queue latency, ex. 1311763801. We have investgated this issue and found the root cuase of this issue is the hash map to store en-queue start time of each task overflowed, and it seems cpudist also encountered such issue before and fix by following pull request. iovisor/bcc#2568 Apply the similar fix to runqslower, getting the max_pid from /proc/sys/kernel/max_pid and pass to BPF C in kernel to create enough entries of BPF map to store en-queue start time of each task. In the mean time, since we create the BPF map with max_pid entries, use BPF_ARRAY instead of BPF_HASH to prevent collision. ``` (bcc)root@localhost:/# runqslower 100000 -P Tracing run queue latency higher than 100000 us TIME COMM TID LAT(us) PREV COMM PREV TID 23:42:45 glide-source-th 28986 161605 kworker/u9:3 12260 23:42:50 HTTPREQUEST_MAN 26249 107187 OMX_VideoInput 7152 23:42:50 migration/2 24 146799 OmxSideband 7147 23:42:50 rtk_post_worker 128 174145 migration/2 24 23:42:50 Binder:233_4 633 170457 runqslower 7086 23:42:58 GIBBON_SURFACE_ 26233 103377 [email protected] 457 23:43:14 Binder:581_1B 2202 113153 Binder:581_7 833 23:43:24 migration/0 15 107716 writer 7277 .... 05:08:14 dumpLogs.sh 22242 1311763801 NBP_POOL-[1] 24233 05:08:14 dumpLogs.sh 22243 1311752373 appsrc7:src 24281 05:08:14 dumpLogs.sh 22248 1311760825 dumpLogs.sh 22193 ``` Signed-off-by: Ism Hong <[email protected]>
Use bpf_probe_read_user_str() instead of bpf_probe_read_user() in opensnoop.py to avoid empty file names. Co-authored-by: Nurdan <[email protected]>
…for -v option
Add additional information and change format of backtrace
- add symbol base offset, dso name, dso base offset
- symbol and dso info is included if it's available in target binary
- changed format:
INDEX ADDR [SYMBOL+OFFSET] (MODULE+OFFSET)
Print backtrace of ip if it failed to get syms.
Before:
# offcputime -v
psiginfo
vscanf
__snprintf_chk
[unknown]
[unknown]
[unknown]
[unknown]
[unknown]
sd_event_exit
sd_event_dispatch
sd_event_run
[unknown]
__libc_start_main
[unknown]
- systemd-journal (204)
1
xas_load
xas_find
filemap_map_pages
__handle_mm_fault
handle_mm_fault
do_page_fault
do_translation_fault
do_mem_abort
do_el0_ia_bp_hardening
el0_ia
xas_load
--
failed to get syms
- PmLogCtl (138757)
1
After:
# offcputime -v
#0 0xffffffc01018b7e8 __arm64_sys_clock_nanosleep+0x0
#1 0xffffffc01009a93c el0_svc_handler+0x34
#2 0xffffffc010084a08 el0_svc+0x8
#3 0xffffffc01018b7e8 __arm64_sys_clock_nanosleep+0x0
--
#4 0x0000007fa0bffd14 clock_nanosleep+0x94 (/usr/lib/libc-2.31.so+0x9ed14)
#5 0x0000007fa0c0530c nanosleep+0x1c (/usr/lib/libc-2.31.so+0xa430c)
#6 0x0000007fa0c051e4 sleep+0x34 (/usr/lib/libc-2.31.so+0xa41e4)
#7 0x000000558a5a9608 flb_loop+0x28 (/usr/bin/fluent-bit+0x52608)
#8 0x000000558a59f1c4 flb_main+0xa84 (/usr/bin/fluent-bit+0x481c4)
#9 0x0000007fa0b85124 __libc_start_main+0xe4 (/usr/lib/libc-2.31.so+0x24124)
#10 0x000000558a59d828 _start+0x34 (/usr/bin/fluent-bit+0x46828)
- fluent-bit (1238)
1
#0 0xffffffc01027daa4 generic_copy_file_checks+0x334
#1 0xffffffc0102ba634 __handle_mm_fault+0x8dc
#2 0xffffffc0102baa20 handle_mm_fault+0x168
#3 0xffffffc010ad23c0 do_page_fault+0x148
#4 0xffffffc010ad27c0 do_translation_fault+0xb0
#5 0xffffffc0100816b0 do_mem_abort+0x50
#6 0xffffffc0100843b0 el0_da+0x1c
#7 0xffffffc01027daa4 generic_copy_file_checks+0x334
--
#8 0x0000007f8dc12648 [unknown]
#9 0x0000007f8dc0aef8 [unknown]
#10 0x0000007f8dc1c990 [unknown]
#11 0x0000007f8dc08b0c [unknown]
#12 0x0000007f8dc08e48 [unknown]
#13 0x0000007f8dc081c8 [unknown]
- PmLogCtl (2412)
1
Fixed: #3884
Signed-off-by: Eunseon Lee <[email protected]>
Currently, we support the following libc.so path pattern:
7f22eba09000-7f22ebbc5000 r-xp 00000000 fd:01 274076 /usr/lib64/libc-2.28.so
7f22ebbc5000-7f22ebdc4000 ---p 001bc000 fd:01 274076 /usr/lib64/libc-2.28.so
7f22ebdc4000-7f22ebdc8000 r--p 001bb000 fd:01 274076 /usr/lib64/libc-2.28.so
7f22ebdc8000-7f22ebdca000 rw-p 001bf000 fd:01 274076 /usr/lib64/libc-2.28.so
But failed to handle the following case:
ffffff88c19000-ffffff88d3a000 r-xp 00000000 fc:01 258823 /usr/lib/riscv64-linux-gnu/libc.so.6
ffffff88d3a000-ffffff88d3b000 ---p 00121000 fc:01 258823 /usr/lib/riscv64-linux-gnu/libc.so.6
ffffff88d3b000-ffffff88d3e000 r--p 00121000 fc:01 258823 /usr/lib/riscv64-linux-gnu/libc.so.6
ffffff88d3e000-ffffff88d40000 rw-p 00124000 fc:01 258823 /usr/lib/riscv64-linux-gnu/libc.so.6
Add support for it.
Signed-off-by: Hengqi Chen <[email protected]>
ksnoop and offcputime are run under kernel context,
use kernel variants of bpf_probe_read{,_str} instead.
This allows these tools to run on architectures that
lack support of bpf_probe_read{,_str}.
Signed-off-by: Hengqi Chen <[email protected]>
Some architectures don't have open syscall, thus don't have tracepoint for open syscall. Disable the corresponding BPF program so that the tool can run on such architectures. Signed-off-by: Hengqi Chen <[email protected]>
Trace all stat syscall variants that have a filename argument, but enable them only when the corresponding tracepoints exist. Signed-off-by: Hengqi Chen <[email protected]>
With latest llvm16, I got the following build errors:
...
/.../bcc/build/src/cc/libbcc.so: undefined reference to
`llvm::hlsl::FrontendResource::FrontendResource(llvm::GlobalVariable*,
llvm::StringRef, llvm::hlsl::ResourceKind, unsigned int, unsigned int)'
collect2: error: ld returned 1 exit status
make[2]: *** [tests/cc/CMakeFiles/test_libbcc.dir/build.make:371: tests/cc/test_libbcc] Error 1
...
The reason is due to llvm upsteam patch https://reviews.llvm.org/D135110
which added llvm library FrontendHLSL as a dependency to clang CodeGen library.
So we need to add it as well for non-shared library build mode.
Signed-off-by: Yonghong Song <[email protected]>
Since version 15, clang issues error for implicit conversion of
integer to pointer. Several tools are broken. This patch add explicit
pointer cast where needed.
Fixes the following errors:
/virtual/main.c:37:18: error: incompatible integer to pointer conversion initializing 'struct request *' with an expression of type 'unsigned long' [-Wint-conversion]
struct request *req = ctx->di;
^ ~~~~~~~
/virtual/main.c:49:18: error: incompatible integer to pointer conversion initializing 'struct request *' with an expression of type 'unsigned long' [-Wint-conversion]
struct request *req = ctx->di;
^ ~~~~~~~
2 errors generated.
/virtual/main.c:73:19: error: incompatible integer to pointer conversion initializing 'struct pt_regs *' with an expression of type 'unsigned long' [-Wint-conversion]
struct pt_regs * __ctx = ctx->di;
^ ~~~~~~~
/virtual/main.c:100:240: error: incompatible integer to pointer conversion passing 'u64' (aka 'unsigned long long') to parameter of type 'const void *' [-Wint-conversion]
data.ppid = ({ typeof(pid_t) _val; __builtin_memset(&_val, 0, sizeof(_val)); bpf_probe_read(&_val, sizeof(_val), (u64)&({ typeof(struct task_struct *) _val; __builtin_memset(&_val, 0, sizeof(_val)); bpf_probe_read(&_val, sizeof(_val), (u64)&task->real_parent); _val; })->tgid); _val; });
^~~~~~~~~~~~~~~~~~~~~~~
/virtual/main.c:100:118: error: incompatible integer to pointer conversion passing 'u64' (aka 'unsigned long long') to parameter of type 'const void *' [-Wint-conversion]
data.ppid = ({ typeof(pid_t) _val; __builtin_memset(&_val, 0, sizeof(_val)); bpf_probe_read(&_val, sizeof(_val), (u64)&({ typeof(struct task_struct *) _val; __builtin_memset(&_val, 0, sizeof(_val)); bpf_probe_read(&_val, sizeof(_val), (u64)&task->real_parent); _val; })->tgid); _val; });
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Signed-off-by: Jerome Marchand <[email protected]>
libdebuginfod support is currently only configurable by the presence of the library itself. Unfortunately, when using the static libraries the libdebuginfod dependency gets propagated to users of bcc (e.g. bpftrace), who would have to require it as well. Introduce ENABLE_LIBDEBUGINFOD to allow overriding libdebuginfod usage from above. This allows bcc users to decide for themselves if they want this dependency.
Migrating perf based powerpc-hcalls.py to a more enhanced ebpf variant inspired by existing bcc/tools/syscount.py. Additional hcalls updated from Linux kernel source. Signed-off-by: Harsh Prateek Bora ([email protected])
…t or union
Some fields can belong to unnamed struct or union (e.g. rcu and
rcu_users fields of task_struct). In C, they are accessed as if their
belong directly to the parent of the unnamed struct or union but this
is not the case for BTF.
When looking for a field, kernel_struct_has_field should also look
reccursively into unnamed structs or unions. That allows code such as
the following to work as expected:
BPF.kernel_struct_has_field('task_struct', 'rcu')
Signed-off-by: Jerome Marchand <[email protected]>
* Add missing examples files
prakashsurya
force-pushed
the
dlpx/pr/prakashsurya/bbc7dfcd-2343-4602-a8fd-2180e9fa7a35
branch
from
a98a3e3 to
e4af814
Compare
prakashsurya
force-pushed
the
dlpx/pr/prakashsurya/bbc7dfcd-2343-4602-a8fd-2180e9fa7a35
branch
from
e4af814 to
c5f91cd
Compare
delphix-devops-bot
force-pushed
the
develop
branch
from
7aa2829 to
ef9a7c7
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.