This is the mail archive of the
systemtap@sourceware.org
mailing list for the systemtap project.
[Bug kprobes/13108] New: kprobing some paravirt stuff seems unsafe
- From: "mjw at redhat dot com" <sourceware-bugzilla at sourceware dot org>
- To: systemtap at sourceware dot org
- Date: Thu, 18 Aug 2011 20:07:50 +0000
- Subject: [Bug kprobes/13108] New: kprobing some paravirt stuff seems unsafe
- Auto-submitted: auto-generated
http://sourceware.org/bugzilla/show_bug.cgi?id=13108
Bug #: 13108
Summary: kprobing some paravirt stuff seems unsafe
Product: systemtap
Version: unspecified
Status: NEW
Severity: normal
Priority: P2
Component: kprobes
AssignedTo: systemtap@sourceware.org
ReportedBy: mjw@redhat.com
Classification: Unclassified
The following, run inside a kvm guest will often (but not always) crash the kvm
guest:
$ /usr/local/install/systemtap/bin/stap -m clts -e "global c; probe
kernel.function(\"clts\") { if(c++ < 3) log(pp()) else exit() }" -c 'sleep 1;
ls -laR /dev /proc > /tmp/garbage.out 2>&1; sync'
The crashes aren't consistent though:
exhibit 1)
clts: systemtap: 1.7/0.152, base: ffffffffa06f5000, memory:
48data/18text/10ctx/10net/33alloc kb, probes: 7
BUG: unable to handle kernel paging request at fffffffffffffff0
IP: [<ffffffff810155e7>] restore_i387_xstate+0xc7/0x1c0
PGD 1a27067 PUD 1a28067 PMD 0
Oops: 0002 [#1] SMP
last sysfs file: /sys/module/xt_state/sections/__mcount_loc
CPU 3
Modules linked in: clts(U) ebtable_nat ebtables ipt_MASQUERADE iptable_nat
nf_nat xt_CHECKSUM iptable_mangle bridge stp llc autofs4 nfs lockd fscache(T)
nfs_acl auth_rpcgss sunrpc xt_physdev ipt_REJECT nf_conntrack_ipv4
nf_defrag_ipv4 iptable_filter ip_tables ip6t_REJECT nf_conntrack_ipv6
nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 dm_mirror
dm_region_hash dm_log vhost_net macvtap macvlan tun uinput sg microcode
virtio_balloon snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device
snd_pcm snd_timer snd soundcore snd_page_alloc virtio_net i2c_piix4 i2c_core
ext4 mbcache jbd2 virtio_blk sr_mod cdrom virtio_pci virtio_ring virtio
pata_acpi ata_generic ata_piix dm_mod [last unloaded: cls_destroy]
Modules linked in: clts(U) ebtable_nat ebtables ipt_MASQUERADE iptable_nat
nf_nat xt_CHECKSUM iptable_mangle bridge stp llc autofs4 nfs lockd fscache(T)
nfs_acl auth_rpcgss sunrpc xt_physdev ipt_REJECT nf_conntrack_ipv4
nf_defrag_ipv4 iptable_filter ip_tables ip6t_REJECT nf_conntrack_ipv6
nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 dm_mirror
dm_region_hash dm_log vhost_net macvtap macvlan tun uinput sg microcode
virtio_balloon snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device
snd_pcm snd_timer snd soundcore snd_page_alloc virtio_net i2c_piix4 i2c_core
ext4 mbcache jbd2 virtio_blk sr_mod cdrom virtio_pci virtio_ring virtio
pata_acpi ata_generic ata_piix dm_mod [last unloaded: cls_destroy]
Pid: 13482, comm: stapio Tainted: G ---------------- T
2.6.32-131.6.1.el6.x86_64 #1 Bochs
RIP: 0010:[<ffffffff810155e7>] [<ffffffff810155e7>]
restore_i387_xstate+0xc7/0x1c0
RSP: 0018:ffff8800061b7ea8 EFLAGS: 00010346
RAX: ffff8800061b6000 RBX: 00007fff5d53c6c0 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff880037a76800
RBP: ffff8800061b7ef8 R08: 0000000000000000 R09: ffff880037a76600
R10: 00007fff5d53c710 R11: 0000000000000246 R12: ffff880099747540
R13: ffff880099747540 R14: ffff8800061b7fd8 R15: 00007fff5d53c500
FS: 00007f080b574700(0000) GS:ffff880002180000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: fffffffffffffff0 CR3: 00000000061ff000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff4ff0 DR7: 0000000000000400
Process stapio (pid: 13482, threadinfo ffff8800061b6000, task ffff880099747540)
Stack:
ffff8800061b7f48 00007fffffffeffd ffff8800061b7f08 0000000101e1dd39
<0> ffff880099747af8 ffff8800061b7fd8 0000000000402080 00007fff5d53c4f8
<0> ffff8800061b7f58 0000000000000200 ffff8800061b7f48 ffffffff8100adc0
Call Trace:
[<ffffffff8100adc0>] sys_rt_sigreturn+0x200/0x280
[<ffffffff8100b68c>] stub_rt_sigreturn+0x6c/0xa0
Code: e0 ff ff 48 83 de 00 48 85 f6 0f 85 ff 00 00 00 41 f6 44 24 15 20 74 7c
65 4c 8b 2c 25 00 cc 00 00 49 8b 45 08 f6 40 14 01 75 0f <cc> 06 0f 1f 44 00 00
49 8b 45 08 83 48 14 01 b0 00 84 c0 74 44
RIP [<ffffffff810155e7>] restore_i387_xstate+0xc7/0x1c0
RSP <ffff8800061b7ea8>
CR2: fffffffffffffff0
---[ end trace d2747920f0b64285 ]---
Kernel panic - not syncing: Fatal exception
Pid: 13482, comm: stapio Tainted: G D ---------------- T
2.6.32-131.6.1.el6.x86_64 #1
Call Trace:
[<ffffffff814da518>] ? panic+0x78/0x143
[<ffffffff814de564>] ? oops_end+0xe4/0x100
[<ffffffff81040c9b>] ? no_context+0xfb/0x260
[<ffffffff81040f25>] ? __bad_area_nosemaphore+0x125/0x1e0
[<ffffffff81040ff3>] ? bad_area_nosemaphore+0x13/0x20
[<ffffffff810416cd>] ? __do_page_fault+0x31d/0x480
[<ffffffff8107d8ed>] ? __sigqueue_free+0x3d/0x50
[<ffffffff8108120f>] ? __dequeue_signal+0xdf/0x1f0
[<ffffffff810813fa>] ? dequeue_signal+0xda/0x170
[<ffffffff814e054e>] ? do_page_fault+0x3e/0xa0
[<ffffffff814dd8d5>] ? page_fault+0x25/0x30
[<ffffffff810155e7>] ? restore_i387_xstate+0xc7/0x1c0
[<ffffffff81015658>] ? restore_i387_xstate+0x138/0x1c0
[<ffffffff8100adc0>] ? sys_rt_sigreturn+0x200/0x280
[<ffffffff8100b68c>] ? stub_rt_sigreturn+0x6c/0xa0
exhibit 2) [note it ran the same probe first without trouble]
clts: systemtap: 1.7/0.152, base: ffffffffa00f8000, memory:
48data/18text/10ctx/10net/33alloc kb, probes: 7
clts: systemtap: 1.7/0.152, base: ffffffffa02c6000, memory:
48data/18text/10ctx/10net/33alloc kb, probes: 7
invalid opcode: 0000 [#1] SMP
last sysfs file: /sys/module/xt_state/sections/__mcount_loc
CPU 3
Modules linked in: clts(U) ebtable_nat ebtables ipt_MASQUERADE iptable_nat
nf_nat xt_CHECKSUM iptable_mangle bridge stp llc autofs4 nfs lockd fscache(T)
nfs_acl auth_rpcgss sunrpc xt_physdev ipt_REJECT nf_conntrack_ipv4
nf_defrag_ipv4 iptable_filter ip_tables ip6t_REJECT nf_conntrack_ipv6
nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 dm_mirror
dm_region_hash dm_log vhost_net macvtap macvlan tun uinput sg microcode
virtio_balloon snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device
snd_pcm snd_timer snd soundcore snd_page_alloc virtio_net i2c_piix4 i2c_core
ext4 mbcache jbd2 virtio_blk sr_mod cdrom virtio_pci virtio_ring virtio
pata_acpi ata_generic ata_piix dm_mod [last unloaded: clts]
Modules linked in: clts(U) ebtable_nat ebtables ipt_MASQUERADE iptable_nat
nf_nat xt_CHECKSUM iptable_mangle bridge stp llc autofs4 nfs lockd fscache(T)
nfs_acl auth_rpcgss sunrpc xt_physdev ipt_REJECT nf_conntrack_ipv4
nf_defrag_ipv4 iptable_filter ip_tables ip6t_REJECT nf_conntrack_ipv6
nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 dm_mirror
dm_region_hash dm_log vhost_net macvtap macvlan tun uinput sg microcode
virtio_balloon snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device
snd_pcm snd_timer snd soundcore snd_page_alloc virtio_net i2c_piix4 i2c_core
ext4 mbcache jbd2 virtio_blk sr_mod cdrom virtio_pci virtio_ring virtio
pata_acpi ata_generic ata_piix dm_mod [last unloaded: clts]
Pid: 0, comm: swapper Tainted: G ---------------- T
2.6.32-131.6.1.el6.x86_64 #1 Bochs
RIP: 0010:[<ffffffffa001a002>] [<ffffffffa001a002>] 0xffffffffa001a002
RSP: 0018:ffff880099eb7ad8 EFLAGS: 00010102
RAX: ffff88009c046000 RBX: ffff88009b327580 RCX: ffff88009c01eb00
RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff88009b327c20
RBP: ffff880099eb7b28 R08: 0000000000000000 R09: 0000000000000001
R10: 0000002f818d8aa9 R11: 0000000000000001 R12: ffff88009c01eb00
R13: 0000000000000000 R14: 0000000000000003 R15: ffff88009b327c20
FS: 0000000000000000(0000) GS:ffff880002180000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000020ba110 CR3: 000000009a062000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff4ff0 DR7: 0000000000000400
Process swapper (pid: 0, threadinfo ffff88009c046000, task ffff88009c01eb00)
Stack:
0000000000000000 0000000000000000 0000000001000000 ffff880002193b40
<0> 0000000000000001 ffff880002195f80 ffff88009acb5a00 0000000000000003
<0> ffff88009b16a440 00000000ffffffff ffff88009b327580 ffffffff814dabd9
Call Trace:
Code:
BUG: unable to handle kernel paging request at ffffffffa0019fd7
IP: [<ffffffff81009757>] __switch_to+0x157/0x320
PGD 1a27067 PUD 1a2b063 PMD 37b49067 PTE 0
Oops: 0000 [#2] SMP
last sysfs file: /sys/module/xt_state/sections/__mcount_loc
CPU 3
Modules linked in: clts(U) ebtable_nat ebtables ipt_MASQUERADE iptable_nat
nf_nat xt_CHECKSUM iptable_mangle bridge stp llc autofs4 nfs lockd fscache(T)
nfs_acl auth_rpcgss sunrpc xt_physdev ipt_REJECT nf_conntrack_ipv4
nf_defrag_ipv4 iptable_filter ip_tables ip6t_REJECT nf_conntrack_ipv6
nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 dm_mirror
dm_region_hash dm_log vhost_net macvtap macvlan tun uinput sg microcode
virtio_balloon snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device
snd_pcm snd_timer snd soundcore snd_page_alloc virtio_net i2c_piix4 i2c_core
ext4 mbcache jbd2 virtio_blk sr_mod cdrom virtio_pci virtio_ring virtio
pata_acpi ata_generic ata_piix dm_mod [last unloaded: clts]
Modules linked in: clts(U) ebtable_nat ebtables ipt_MASQUERADE iptable_nat
nf_nat xt_CHECKSUM iptable_mangle bridge stp llc autofs4 nfs lockd fscache(T)
nfs_acl auth_rpcgss sunrpc xt_physdev ipt_REJECT nf_conntrack_ipv4
nf_defrag_ipv4 iptable_filter ip_tables ip6t_REJECT nf_conntrack_ipv6
nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 dm_mirror
dm_region_hash dm_log vhost_net macvtap macvlan tun uinput sg microcode
virtio_balloon snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device
snd_pcm snd_timer snd soundcore snd_page_alloc virtio_net i2c_piix4 i2c_core
ext4 mbcache jbd2 virtio_blk sr_mod cdrom virtio_pci virtio_ring virtio
pata_acpi ata_generic ata_piix dm_mod [last unloaded: clts]
Pid: 0, comm: swapper Tainted: G ---------------- T
2.6.32-131.6.1.el6.x86_64 #1 Bochs
RIP: 0010:[<ffffffff81009757>] [<ffffffff81009757>] __switch_to+0x157/0x320
RSP: 0018:ffff880099eb7850 EFLAGS: 00010097
RAX: ffff880099eb7887 RBX: ffff880099eb7a28 RCX: 0000000000000001
RDX: 0000000000000001 RSI: ffffffffa0019fd7 RDI: ffff880099eb7887
RBP: ffff880099eb78b8 R08: ffffffff81b9e300 R09: 0000000000000000
R10: 000000000000000f R11: 0000000000000000 R12: ffffffffa0019fd7
R13: ffff88009c047fd8 R14: ffff88009c046000 R15: 000000000000002b
FS: 0000000000000000(0000) GS:ffff880002180000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: ffffffffa0019fd7 CR3: 000000009a062000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff4ff0 DR7: 0000000000000400
Process swapper (pid: 0, threadinfo ffff88009c046000, task ffff88009c01eb00)
Stack:
ffffffff8100e3cf ffffffff81773d83 ffffffffffffffff 0000000000000000
<0> 0000004000000006 ffff880099eb7888 ffffffff814e066a ffff880099eb78b8
<0> 0000000000000000 ffffffff81773d83 ffff880099eb7a28 0000000000000000
Call Trace:
Code: cb 02 00 66 90 48 89 c7 48 83 cf 08 e8 83 cb 02 00 66 90 eb 10 0f 1f 80
00 00 00 00 41 c6 84 24 10 02 00 00 00 80 7d c3 00 74 07 <0f> 06 0f 1f 44 00 00
48 89 df 0f 1f 80 00 00 00 00 45 85 ed 0f
RIP [<ffffffff81009757>] __switch_to+0x157/0x320
RSP <ffff880099eb7850>
CR2: ffffffffa0019fd7
---[ end trace 07cc9d4c6df5c545 ]---
Kernel panic - not syncing: Fatal exception
Pid: 0, comm: swapper Tainted: G D ---------------- T
2.6.32-131.6.1.el6.x86_64 #1
Call Trace:
A lot of the paravirt stuff (at least that inside
arch/x86/include/asm/paravirt.h and arch/x86/kernel/paravirt*.c) looks somewhat
problematic/tricky to handle through kprobes.
Trying the following patch:
diff --git a/dwflpp.cxx b/dwflpp.cxx
index 7da8a72..36a4a3c 100644
--- a/dwflpp.cxx
+++ b/dwflpp.cxx
@@ -2963,6 +2963,9 @@ dwflpp::build_blacklist()
blfile += "|arch/.*/include/asm/io\\.h";
blfile += "|arch/.*/include/asm/bitops\\.h";
blfile += "|drivers/ide/ide-iops\\.c";
+ // paravirt ops
+ blfile += "|arch/.*/kernel/paravirt.*c";
+ blfile += "|arch/.*/include/asm/paravirt\\.h";
// XXX: it would be nice if these blacklisted functions were pulled
// in dynamically, instead of being statically defined here.
Might be overkill?
--
Configure bugmail: http://sourceware.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.