workqueue testsuite - optional probes

["Turgis, Frederic" <f-turgis at ti dot com>]

Hi,

In previous thread solved by Josh stone (http://sourceware.org/ml/systemtap/2011-q3/msg00413.html), I had promised to share an example script to correlate workqueue execution, process name, function executed and workqueue name. It is shared below and tested on a x86 Ubuntu 11.04 and Android Gingerbread ARM (kernel 3.0)

However, I am a bit late because I recently realized that we have a tapset for that in irq.stp, "probe workqueue.execute" relying on kernel.trace probe. I don't know why I didn't look for it first, as I usually do !

- the example can eventually still be of some interest for the workqueue name (which we don't need in the tapset, I do that for my own convenience). I let you judge that. But I would rewrite it using the tapset probe

- I met some issues with workqueue tapset:
   * tracepoints are too old but test testsuite/buildok/irq-detailed.stp succeeds:
      + testsuite uses "probe workqueue.execute ?" so optional probe. Naïve question: is it really expected ? Not sure we are then testing much
      + tapset also uses "?" in definition of the probe. But if I remove the "?" in the testsuite, test fails. And the tapset probe itself in my script fails. It is optional, it shall not fail ? I didn't find known issue in bug list

   * there are now tracepoints for execution start/end, activation and insertion. There is then no equivalent for creation and destruction. I guess we port then to functions, not tracepoints ?

   * trace point "trace_workqueue_execute_start(work)" seems to be enabled correctly (I compared with irq_handler_entry for generated functions in System.map). Still I have no match while resolving the probe. Log states:
Pass 2: getting a tracepoint query for 1 headers:
  xxx/omap//include/trace/events/workqueue.h
...
include/trace/events/workqueue.h:37: error: 'struct cpu_workqueue_struct' declared inside parameter list
include/trace/events/workqueue.h:37: error: its scope is only this definition or declaration, which is probably not what you want

I digged a bit further and found that trace/events/workqueue.h uses "struct cpu_workqueue_struct", which is defined in workqueue.c (oups !). Kernel solves that by including trace/events/workqueue.h in workqueue.c at line 263 and not at start. Confirmed by commit 97bd234701b2b39a0e749c1fe0e44f1d14c94292.

Any thought ? Wait for clean tracepoints ? Temporarily port to function probing ?




#! /usr/bin/env stap

# Copyright (C) 2011 Texas Instruments
# by Frederic Turgis <f-turgis@ti.com>
#
# c34056a3fdde777c079cc8a70785c2602f2586cb renames all workqueue workers
# as kworker/CPUID:WORKERID. This prepares 7e11629d0efec829cbf62366143ba1081944a70e,
# which pools all workers per cpu. As there is no longer strict association
# between a cwq and its worker, it is interesting to trace function called
# and workqueue name when a work is executed.
# Note that a single kworker execution can even execute several works from
# different workqueues !
# It is also interesting to mix this script with any kind of scheduler
# monitoring script, add timestamp, ...

probe kernel.function("process_one_work") {
        /* we don't test cpu_workqueue==NULL because kernel code does not */
        cpu_workqueue = atomic_long_read(& $work->data)
        cpu_workqueue &= %{ WORK_STRUCT_WQ_DATA_MASK %};
        name = @cast(cpu_workqueue, "struct cpu_workqueue_struct")->wq->name;
        printf("Workqueue %s calls %p in %s:%u\n", kernel_string(name), $work->func, execname(), tid());
}

probe kernel.function("process_one_work").return {
        printf("Workqueue in %s:%u ends\n", execname(), tid());
}



Regards
Fred

Frederic Turgis
OMAP Platform Business Unit - OMAP System Engineering - Platform Enablement - System Multimedia


Texas Instruments France SA, 821 Avenue Jack Kilby, 06270 Villeneuve Loubet. 036 420 040 R.C.S Antibes. Capital de EUR 753.920