Re: Using stapdyn to probe child processes of the target?

[Serhei Makarov <serhei dot etc at gmail dot com>]

Ok, I've found the code (in mutator::post_fork_callback) which is
meant to instrument the child process, but that code is not being
triggered when I run any of my scripts (I enabled the logging just to
check). I'll have time to investigate further on Monday. Most likely
there is a PEBKAC of some sort going on [1].

[1] https://en.wiktionary.org/wiki/PEBCAK

On Fri, Jun 16, 2017 at 12:22 AM, Arkady <arkady.miasnikov@gmail.com> wrote:
> I see that there is pre-fork and post-fork hooks in the Dyninst code.
>
> On Fri, Jun 16, 2017 at 7:17 AM, Arkady <arkady.miasnikov@gmail.com> wrote:
>> If the timing is important you can add some code to the script. For
>> example you can call to processAttach() in the script when a certain
>> function in the GCC starts a new process.
>>
>> On Fri, Jun 16, 2017 at 7:05 AM, Arkady <arkady.miasnikov@gmail.com> wrote:
>>> I have never tried user space hooking. My immediate idea is to run gcc
>>> under strace, filter all fork-exec pairs (a Python script here?) and
>>> attach stap to all child PIDs
>>>
>>> On Fri, Jun 16, 2017 at 12:47 AM, Serhei Makarov <serhei.etc@gmail.com> wrote:
>>>> Hello all,
>>>>
>>>> I've been trying to observe gcc's execution using Stapdyn. Most of the
>>>> actual work is done by a child process (cc1) forked from gcc. (Below
>>>> I've attached a script, proctree.stp, for listing processes forked by
>>>> a target command.) If I'm using kernel systemtap, I can just target
>>>> cc1 directly, for example:
>>>>
>>>> stap -ve 'probe
>>>> process("/usr/libexec/gcc/x86_64-redhat-linux/6.3.1/cc1").insn {
>>>> counter<<<1 } global counter probe end { printf("%d calls\n",
>>>> @count(counter)) }' -c "gcc test/widget3.c"
>>>>
>>>> Instrumenting using kernel uprobes introduces quite a large
>>>> performance overhead, so I would prefer Dyninst for this. But the same
>>>> script doesn't work with --dyninst option, because the target process
>>>> selected using -c is not cc1.
>>>>
>>>> As far as I can tell, my only options for observing cc1 with Stapdyn are:
>>>>
>>>> (a) use -c option : invoke cc1 directly. Feasible in this case, may
>>>> not be feasible in other software where the parent/child processes
>>>> coordinate using more complex IPC. This also doesn't give a profile of
>>>> the full GCC execution (unlike, say, "valgrind --trace-children=yes").
>>>> (b) use -x option : attach to cc1 by PID after it starts running. Not
>>>> feasible in this case since cc1 is a batch program, not a long-running
>>>> daemon.
>>>>
>>>> Am I missing any possibilities? Is it currently possible to use
>>>> Stapdyn to attach to processes other than a single target?
>>>>
>>>> All the best,
>>>>      Serhei Makarov
>>>>
>>>> PS Here's the proctree.stp script:
>>>>
>>>> global pids
>>>>
>>>> probe begin {
>>>>   pids[target()] = 0
>>>> }
>>>>
>>>> probe process.begin {
>>>>   child = pid(); parent = ppid()
>>>>   if (parent in pids) {
>>>>     printf("fork %s(%d) -> %s(%d)\n", pexecname(), parent, execname(), child)
>>>>     pids[child] = parent
>>>>   }
>>>> }
>>>>
>>>> This outputs things like:
>>>>
>>>> [serhei@beatrice e-tracer]$ stap proctree.stp -c "gcc test/widget3.c"
>>>> Missing separate debuginfos, use: debuginfo-install
>>>> kernel-core-4.10.10-200.fc25.x86_64
>>>> fork gcc(2725) -> gcc(2728)
>>>> fork gcc(2725) -> cc1(2728)
>>>> fork gcc(2725) -> gcc(2729)
>>>> fork gcc(2725) -> as(2729)
>>>> fork gcc(2725) -> gcc(2730)
>>>> fork gcc(2725) -> collect2(2730)
>>>> fork collect2(2730) -> collect2(2731)
>>>> fork collect2(2730) -> ld(2731)
>>>>
>>>> Or, here's a Firefox browsing session:
>>>>
>>>> [serhei@beatrice e-tracer]$ stap proctree.stp -c "firefox --new-instance"
>>>> fork firefox(3179) -> firefox(3182)
>>>> fork firefox(3179) -> basename(3182)
>>>> fork firefox(3179) -> firefox(3183)
>>>> fork firefox(3179) -> uname(3183)
>>>> fork firefox(3179) -> firefox(3184)
>>>> fork firefox(3179) -> pidof(3184)
>>>> fork firefox(3179) -> firefox(3185)
>>>> fork firefox(3179) -> mkdir(3185)
>>>> fork firefox(3179) -> firefox(3186)
>>>> fork firefox(3186) -> firefox(3187)
>>>> fork firefox(3186) -> firefox(3188)
>>>> fork firefox(3186) -> sed(3188)
>>>> fork firefox(3179) -> firefox(3189)
>>>> fork firefox(3189) -> firefox(3190)
>>>> fork firefox(3189) -> firefox(3191)
>>>> fork firefox(3189) -> sed(3191)
>>>> fork firefox(3179) -> firefox(3192)
>>>> fork firefox(3179) -> expr(3192)
>>>> fork run-mozilla.sh(3179) -> run-mozilla.sh(3193)
>>>> fork run-mozilla.sh(3179) -> basename(3193)
>>>> fork run-mozilla.sh(3179) -> run-mozilla.sh(3194)
>>>> fork run-mozilla.sh(3179) -> dirname(3194)
>>>> fork run-mozilla.sh(3179) -> run-mozilla.sh(3195)
>>>> fork run-mozilla.sh(3179) -> uname(3195)
>>>> fork firefox(3179) -> firefox(3196)
>>>> fork firefox(3179) -> firefox(3198)
>>>> fork Gecko_IOThread(3179) -> Gecko_IOThread(3266)
>>>> fork Gecko_IOThread(3179) -> firefox(3266)