Re: [RFC PATCH v1 0/2] perf: Support for SDT markers

[Hemant Kumar <hkshaw at linux dot vnet dot ibm dot com>]

On 02/25/2014 05:14 PM, Masami Hiramatsu wrote:
> (2014/02/24 18:14), Hemant Kumar wrote:
> > This patchset helps in listing dtrace style markers(SDT) present in user space
> > applications through perf.
> > Notes/markers are placed at important places by the
> > developers. They have a negligible overhead when not enabled.
> > We can enable them and probe at these places and find some important information
> > like the arguments' values, etc.
> >
> > We have lots of applications which use SDT markers today, like:
> > Postgresql, MySql, Mozilla, Perl, Python, Java, Ruby, libvirt, QEMU, glib
> >
> > To add SDT markers into user applications:
> > We need to have this header sys/sdt.h present.
> > sys/sdt.h used is version 3.
> > If not present, install systemtap-sdt-devel package (for fedora-18).
> >
> > Please refer to the Documentation patch to see how the SDT markers are added into
> > a program.
> >
> > With this patchset,
> > - Use perf to list the markers in the app:
> > # perf list sdt ./user_app
> >
> > ./user_app :
> > %user_app:foo_start
> > %user_app:fun_start
> >
> > - Also, we can see the SDT markers present in our system in the usual binaries.
> > These usual binaries are libraries (dsos) listed by ldconfig --print-cache and some
> > binaries present in PATH environment variable.
> >
> > First, scan the binaries using :
> > # perf list sdt --scan
> >
> > Creating a cache of SDT markers...
> > perf sdt cache created!
> >   Use : "perf list sdt"
> >   to see the SDT markers
> Hmm, in that case, I think you'd better introduce perf-sdt for scanning.
> e.g.
>
>   # perf sdt --scan app

Hmm, this seems a better idea :)

>   then you can add app to sdt cache, without app,
>
>   # perf sdt --scan
>
>   will just scans all binaries on the PATH and the libraries which listed
> by `ldconfig --print-caceh`
>
> And perf-list shows only the SDTs in the cache.

Well, what will be better? perf-list or perf-sdt or perf-list sdt??
If perf-list, then wouldn't it be a huge list!!

> If there is no SDTs in the cache, `perf-list sdt` will warn you to
> run the `perf sdt --scan`.

Right.

> > After the sdt cache file is created, use perf list to view the markers :
> > # perf list sdt
> >
> > %rtld : init_start
> > %rtld : init_complete
> > %rtld : map_failed
> > %rtld : map_start
> > %rtld : lll_futex_wake
> > ...
> > ...
> > %libgcc : unwind
> > %libvirt : rpc_server_client_auth_allow
> > %libvirt : rpc_server_client_auth_fail
> > %libvirt : rpc_server_client_auth_deny
> >
> > Alternatively, one can view the /var/cache/perf/perf-sdt.cache directly.
> >
> > This link shows an example of marker probing with Systemtap:
> > https://sourceware.org/systemtap/wiki/AddingUserSpaceProbingToApps
> >
> > Also, this link provides important info regarding SDT notes:
> > http://sourceware.org/systemtap/wiki/UserSpaceProbeImplementation
> >
> > - Markers in binaries :
> > These SDT markers are present in the ELF in the section named
> > ".note.stapsdt".
> > Here, the name of the marker, its provider, type, location, base
> > address, semaphore address.
> > We can retrieve these values using the members name_off and desc_off in
> > Nhdr structure. If these are not enabled, they are present in the ELF as nop.
> >
> > Changes since last series :
> > - Made the SDT markers present in a system more visible through perf list
> >    as suggested by Pekka Enberg.
> > - Less complex command line interface to access the SDT markers.
> > - Added a cache file to store most of the SDT markers of a system.
> >
> > TODO:
> > - Add support to probe these SDT markers and integrate with a previous patch
> >    (support to perf to probe SDT markers) posted in lkml.
> >    https://lkml.org/lkml/2013/10/23/10
> Yeah, but I think we'd better choose another way to integrate it.
> Since SDT is like markers(static events), setting each of them via perf-probe is
> not intuitive. :)  I'd like to use it as an event, e.g.
>
>   # perf top -e "%libgcc:unwind"
>
> And perf top internally calls perf-probe to add new uprobe event, and
> clean the new event at exit.

Yeah! Right :) Makes sense.

Will implement the suggestions in the next version asap!

--
Thanks
Hemant Kumar