This is the mail archive of the systemtap@sourceware.org mailing list for the systemtap project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Unable to retrieve pthread mutex symbols


Mark,

Although I showed you how to set up the tests using the RPMs, I actually ran them with a development build compiled with the '-g' option, so that's not the problem. This was true of the agent, the libraries, and the test program.

I think Frank nailed it when he noticed that the futexes I seek are actually in the shared memory region, so won't have any symbols available.

There are futexes used within the Berkeley DB library as well, so I would probably need to add it's debuginfo as well. I probably won't get to it this weekend due to other commitments, but I'll try it earlier next week. And the uprobe as well.

Thanks for the help guys!

- Dave

On May 29, 2009, at 11:56 AM, Mark Wielaard wrote:

Hi David,

On Thu, 2009-05-28 at 12:43 -0230, David Carter wrote:
Basically, I'm trying to debug mutex use in my program. I use the
pthreads version, which in turn uses futexes. Using the attached
systemtap script, I get the addresses of the futexes under contention,
but not the symbols associated with them. This makes debugging
difficult.


There are a few steps involved in setting up the test.

1. Using either FC10 or FC11, install the arm4 and arm4-devel
packages. These don't have SELinux support yet, so you'll either have
to disable it or make it permissive.

You also want the arm4-debuginfo package and dependencies, which you can
get through debuginfo-install arm4.


2. Add your user to the arm4 group

# usermod -a -G arm4 username

You'll have to log out and log in again for this to take effect.

3. Start the arm4 service

4. Compile the arm4 sample program

$ cc -o arm4test_hello_throughput arm4test_hello_throughput.c -larm4

Again add debug symbols by adding -g. Currently all symbolic information
(backtraces, symbol names, etc) are fetched from the debuginfo.


5. Run the systemtap script while running the arm4 sample program. I
recommend starting the systemtap script before starting the sample.
The sample has an infinite loop so stop it after 30 seconds or so.
The output I get shows the hex addresses of the futexes rather than
the symbols. This makes it difficult to debug.

Yes, then it still doesn't work :(
I am not sure how futexes work though, the futex_uaddr given seems to
point in the middle of the address range that has /var/lib/arm4/__db. 001
mapped for me (see with /proc/<pid>/maps). Obviously there isn't an user
symbol there. But the address doesn't really make sense to me, so maybe
I am misinterpreting something.


BTW. For user space symbol resolving to work properly systemtap needs to
have the symbol table for the application and needs to know which
executables to track mmapping of code for (the symbol tables it keeps
are relative to the start of the address that a code segments gets
mapped to into the process space). This is automatically done if you
have a uprobe on the process. So in this case I would add something like
probe process("arm4_daemon").function("main") { log("daemon started"); }
just to take care of that.


Cheers,

Mark



---- Custom Software Solutions http://www.davidcarter.ca http://davidcarterca.wordpress.com




Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]