This is the mail archive of the
systemtap@sourceware.org
mailing list for the systemtap project.
Can't set statement probe in self-built linux-4.4
- From: Chris Dunlop <chris at onthe dot net dot au>
- To: systemtap at sourceware dot org
- Date: Thu, 30 Jun 2016 17:57:44 +1000
- Subject: Can't set statement probe in self-built linux-4.4
- Authentication-results: sourceware.org; auth=none
Hi,
I can't seem to set statement probes in a self-built linux-4.4. What can
be going wrong?
I'm using stap 3.1, gcc 4.9.2, and have SYSTEMTAP_RELEASE set to my build
dir:
----------------------------------------------------------------------
$ stap --version
Systemtap translator/driver (version 3.1/0.159, commit release-3.0-133-g42b97387ed3f + changes)
Copyright (C) 2005-2016 Red Hat, Inc. and others
This is free software; see the source for copying conditions.
tested kernel versions: 2.6.18 ... 4.6-rc
enabled features: AVAHI BOOST_STRING_REF LIBSQLITE3 LIBXML2 NLS NSS READLINE
$ gcc --version
gcc (Debian 4.9.2-10) 4.9.2
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ env | grep SYSTEMTAP_RELEASE
SYSTEMTAP_RELEASE=/home/chris/git/linux-build/4.4.14-otn-00011-g7bcaa67
----------------------------------------------------------------------
The kernel is configued with CONFIG options per:
https://sourceware.org/systemtap/wiki/SystemTapWithSelfBuiltKernel
...with the addition of CONFIG_DEBUG_INFO_DWARF if that matters:
----------------------------------------------------------------------
$ egrep '^CONFIG_(DEBUG_INFO|KPROBES|RELAY|DEBUG_FS|MODULES|MODULE_UNLOAD|UPROBES|DEBUG_INFO_DWARF4)=' .config
CONFIG_RELAY=y
CONFIG_KPROBES=y
CONFIG_UPROBES=y
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
CONFIG_DEBUG_INFO=y
CONFIG_DEBUG_INFO_DWARF4=y
CONFIG_DEBUG_FS=y
----------------------------------------------------------------------
I can set a kernel function probe, e.g.:
----------------------------------------------------------------------
$ stap -p2 -e 'probe kernel.function("register_shrinker") { printf("foo\n"); }'
# probes
kernel.function("register_shrinker") /* pc=_stext+0x1421b0 */ /* <- kernel.function("register_shrinker") */
----------------------------------------------------------------------
But I can't set a kernel statement probe using either a function offset or
explicit line number, e.g.:
----------------------------------------------------------------------
$ grep -n -A5 '^int register_shrinker' mm/vmscan.c
226:int register_shrinker(struct shrinker *shrinker)
227-{
228- size_t size = sizeof(*shrinker->nr_deferred);
229-
230- /*
231- * If we only have one possible node in the system anyway, save
$ stap -v -p2 -e 'probe kernel.statement("register_shinker@mm/vmscan.c+3") { printf("foo\n"); }'
Pass 1: parsed user script and 116 library scripts using 93372virt/31136res/2988shr/28456data kb, in 180usr/20sys/205real ms.
semantic error: resolution failed in DWARF builder
semantic error: while resolving probe point: identifier 'kernel' at <input>:1:7
source: probe kernel.statement("register_shinker@mm/vmscan.c+3") { printf("foo\n"); }
^
semantic error: no match
Pass 2: analyzed script: 0 probes, 0 functions, 0 embeds, 0 globals using 112444virt/50900res/3736shr/47528data kb, in 330usr/40sys/362real ms.
Pass 2: analysis failed. [man error::pass2]
$ stap -v -p2 -e 'probe kernel.statement("*@*/vmscan.c:229") { printf("foo\n"); }'
Pass 1: parsed user script and 116 library scripts using 93372virt/31132res/2988shr/28456data kb, in 190usr/30sys/204real ms.
semantic error: resolution failed in DWARF builder
semantic error: while resolving probe point: identifier 'kernel' at <input>:1:7
source: probe kernel.statement("*@*/vmscan.c:229") { printf("foo\n"); }
^
semantic error: no match
Pass 2: analyzed script: 0 probes, 0 functions, 0 embeds, 0 globals using 108076virt/46716res/3736shr/43160data kb, in 380usr/30sys/427real ms.
Pass 2: analysis failed. [man error::pass2]
$ stap -vvvvv -p2 -e 'probe kernel.statement("*@*/vmscan.c:229") { printf("foo\n"); }' >& /tmp/stap.out
----------------------------------------------------------------------
The full -vvvvvv output from above is available at:
https://www.dropbox.com/s/dfepzebixd7yb36/stap.out.gz?dl=0
>From here I'm pretty lost, but the image looks to contain line numbers,
e.g.:
----------------------------------------------------------------------
$ readelf --debug-dump=line $SYSTEMTAP_RELEASE/vmlinux | head -45
Raw dump of debug contents of section .debug_line:
Offset: 0x0
Length: 375
DWARF Version: 2
Prologue Length: 86
Minimum Instruction Length: 1
Initial value of 'is_stmt': 1
Line Base: -5
Line Range: 14
Opcode Base: 13
Opcodes:
Opcode 1 has 0 args
Opcode 2 has 1 args
Opcode 3 has 1 args
Opcode 4 has 1 args
Opcode 5 has 1 args
Opcode 6 has 0 args
Opcode 7 has 0 args
Opcode 8 has 0 args
Opcode 9 has 1 args
Opcode 10 has 0 args
Opcode 11 has 0 args
Opcode 12 has 1 args
The Directory Table (offset 0x1b):
1 /home/chris/git/linux/arch/x86/kernel
The File Name Table (offset 0x42):
Entry Dir Time Size Name
1 1 0 0 head_64.S
2 1 0 0 verify_cpu.S
Line Number Statements:
[0x00000060] Extended opcode 2: set Address to 0xffffffff81000000
[0x0000006b] Advance Line by 68 to 69
[0x0000006e] Copy
[0x0000006f] Special opcode 81: advance Address by 5 to 0xffffffff81000005 and Line by 6 to 75
[0x00000070] Special opcode 104: advance Address by 7 to 0xffffffff8100000c and Line by 1 to 76
[0x00000071] Special opcode 106: advance Address by 7 to 0xffffffff81000013 and Line by 3 to 79
[0x00000072] Special opcode 48: advance Address by 3 to 0xffffffff81000016 and Line by 1 to 80
[0x00000073] Special opcode 76: advance Address by 5 to 0xffffffff8100001b and Line by 1 to 81
[0x00000074] Special opcode 34: advance Address by 2 to 0xffffffff8100001d and Line by 1 to 82
[0x00000075] Special opcode 94: advance Address by 6 to 0xffffffff81000023 and Line by 5 to 87
----------------------------------------------------------------------
...and files seem to be in the right places, e.g.:
----------------------------------------------------------------------
$ readelf --debug-dump=info $SYSTEMTAP_RELEASE/vmlinux | less
Contents of the .debug_info section:
...
Compilation Unit @ offset 0x4ed2:
Length: 0x30 (32-bit)
Version: 4
Abbrev Offset: 0x2904
Pointer Size: 8
<0><4edd>: Abbrev Number: 1 (DW_TAG_compile_unit)
<4ede> DW_AT_ranges : 0xcc400
<4ee2> DW_AT_low_pc : 0x0
<4eea> DW_AT_stmt_list : 0x141ef1
<4eee> DW_AT_GNU_dwo_name: (indirect string, offset: 0x2cb1): mm/.tmp_vmscan.dwo
<4ef2> DW_AT_comp_dir : (indirect string, offset: 0x20): /home/chris/git/linux-build/4.4.14-otn-00011-g7bcaa67
<4ef6> DW_AT_GNU_pubnames: 1
<4ef6> DW_AT_GNU_addr_base: 0x11fc98
<4efa> DW_AT_GNU_dwo_id : 0xac451f4dafc46631
<4f02> DW_AT_GNU_ranges_base: 0xc88d0
$ readelf --debug-dump=info /home/chris/git/linux-build/4.4.14-otn-00011-g7bcaa67/mm/.tmp_vmscan.dwo | less
Contents of the .debug_info.dwo section:
Compilation Unit @ offset 0x0:
Length: 0x1fbe1 (32-bit)
Version: 4
Abbrev Offset: 0x0
Pointer Size: 8
<0><b>: Abbrev Number: 1 (DW_TAG_compile_unit)
<c> DW_AT_producer : (indexed string: 0x37): GNU C 4.9.2 -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -m64 -mno-80387 -mno-fp-ret-in-387 -mpreferred-stack-boundary=3 -mtune=generic -mno-red-zone -mcmodel=kernel -maccumulate-outgoing-args -mfentry -march=x86-64 -gsplit-dwarf -gdwarf-4 -O2 -std=gnu90 -p -fno-strict-aliasing -fno-common -falign-jumps=1 -falign-loops=1 -funit-at-a-time -fno-asynchronous-unwind-tables -fno-delete-null-pointer-checks -fno-stack-protector -fno-omit-frame-pointer -fno-optimize-sibling-calls -fno-var-tracking-assignments -fno-strict-overflow -fconserve-stack --param allow-store-data-races=0
<d> DW_AT_language : 1 (ANSI C)
<e> DW_AT_name : (indexed string: 0xd05): /home/chris/git/linux/mm/vmscan.c
<10> DW_AT_comp_dir : (indexed string: 0x4d9): /home/chris/git/linux-build/4.4.14-otn-00011-g7bcaa67
<12> DW_AT_GNU_dwo_id : 0xac451f4dafc46631
$ ls -l /home/chris/git/linux/mm/vmscan.c
-rw-r--r-- 1 chris chris 113618 Jun 24 01:22 /home/chris/git/linux/mm/vmscan.c
----------------------------------------------------------------------
Where to from here?
Cheers,
Chris