This is the mail archive of the
systemtap@sourceware.org
mailing list for the systemtap project.
Re: Kernel Deadlock while using uprobes
I disabled lockdep since it wasn't helping much.
This resulted in a different looking deadlock(see below).
From the code, I can't find a way so that task_utrace_lock is held
twice by the same task unless STAPCONF_UTRACE_VIA_FTRACE has
something to do with it.
I'm thinking maybe ftrace is tracing the tracing code in systemtap, thus
leading to a deadlock if the systemtap callback is called by ftrace.
The deadlock doesn't happen when using STAPCONF_UTRACE_VIA_TRACEPOINTS .
[ 52.207772] =============================================
[ 52.214703] [ INFO: possible recursive locking detected ]
[ 52.220090] 3.0.22-mid26-00008-g48ae378-dirty #14
[ 52.224813] ---------------------------------------------
[ 52.230305] stapio/3210 is trying to acquire lock:
[ 52.235116] (task_utrace_lock){-.-...}, at: [<f6680ebd>]
task_utrace_struct+0x16/0x33 [test]
[ 52.243808]
[ 52.243811] but task is already holding lock:
[ 52.249754] (task_utrace_lock){-.-...}, at: [<f6680ebd>]
task_utrace_struct+0x16/0x33 [test]
[ 52.258380]
[ 52.258383] other info that might help us debug this:
[ 52.265017] Possible unsafe locking scenario:
[ 52.265020]
[ 52.270972] CPU0
[ 52.273511] ----
[ 52.275968] lock(task_utrace_lock);
[ 52.279679] lock(task_utrace_lock);
On Mon, Apr 9, 2012 at 8:48 PM, Josh Stone <jistone@redhat.com> wrote:
> On 04/09/2012 05:30 AM, Negreanu Marius wrote:
>> I managed to get a 3.0 kernel patched with uprobes from here[1].
>> The kernel probing works fine as far as disktop.stp can test that.
>>
>> But, when I'm trying to insert a uprobe, the kernel detects a deadlock.
>> The script I'm using is:
>> echo 'probe process("/bin/ping").function("main") { print($$parms) }'
>>> /tmp/cfg/test.stp
>> and the deadlock is the one attached.
>
>> [ Â173.733192] Â Â Â ÂCPU0
>> [ Â173.735655] Â Â Â Â----
>> [ Â173.738192] Â lock(task_utrace_lock);
>> [ Â173.741912] Â <Interrupt>
>> [ Â173.744550] Â Â lock(task_utrace_lock);
>> [ Â173.748440]
>> [ Â173.748443] Â*** DEADLOCK ***
>
> Hmm, looks like we have inconsistent interrupt state while taking that
> lock. ÂWe either need to disable interrupts while holding it, or make
> sure that it's never attempted within any interrupt. ÂThis is dsmith's
> utrace port, hopefully he can take a closer look.
>
>> 1: Âhttp://repos.fedorapeople.org/repos/jistone/kernel-uprobes/fedora-15/SRPMS/kernel-2.6.40.4-6.uprobes2.fc15.src.rpm
>
> Do note that this new uprobes is still under heavy development, and I
> haven't been keeping that repo up to date. ÂAnton Arapov has much newer
> builds available here:
> http://repos.fedorapeople.org/repos/aarapov/kernel-uprobes/
--
Regards!
http://groleo.wordpress.com