Re: better SMP support for kprobes

[Prasanna S Panchamukhi <prasanna at in dot ibm dot com>]

Hi Will,

Below is my analysis.

kprobes_table is declared static and is accessed for reading during the probe handling time
and for writing during only kprobes registration/unregistration. But kprobes structure is
modified during registration and during disarm of kprobe within kprobe_handler.

kprobes lock is taken during the int3 handling and released after the single stepping is done.
Other data structure which are defined static are:

current_kprobe;
kprobe_status, kprobe_old_eflags, kprobe_saved_eflags;

There are three alternatives to solve this performance degradation problem on SMP.

1. Use read write lock for kprobes_table. written during insertion/removal of kprobe and 
	read during probe handling. Define a per kprobe lock which is taken during handling of probe and during 
	insertion/removal. Per kprobe lock will serialize the handling of same probe on different CPU, but allows
	different probes to handled parallelly.

2. Using Read-Copy-Update (RCU) instead of read write lock. Not sure if RCU can be used here.

3. Defining kprobe_table as per cpu datastructure and using per kprobe lock.Here we need more memory space
to allocate per cpu kprobe_table than in above methods.

Please provide your comments. I will provide the complete algorithm soon to use read write lock and per kprobe lock.

-Prasanna

>Martin Hunt wrote:
>> Currently kprobe_handler grabs kprobe_lock and keeps it the entire time
>> the probe is executing.  On SMP systems this could cause significant
>> performance degradation because only one cpu can be executing a kprobe;
>> all others will be blocked.
>> 
>> Has anyone looked at fixing this?  Based on a quick read of the code, I
>> see the kprobe_lock is needed to protect access to the kprobe_table.
>> There are also a bunch of globals that could be implemented as per_cpu
>> data. Anyone want to volunteer to take a closer look?
>> 
>> Martin
>> 
>> 

>I have looked a little at this and looked into per cpu data to avoid 
>this problem. I can volunteer for it.


-- 

Prasanna S Panchamukhi
Linux Technology Center
India Software Labs, IBM Bangalore
Ph: 91-80-25044636
<prasanna@in.ibm.com>