This is the mail archive of the
systemtap@sourceware.org
mailing list for the systemtap project.
Re:
- From: Arkady <arkady dot miasnikov at gmail dot com>
- To: Daniel Doron <danielmeirdoron at gmail dot com>
- Cc: Eyal Yehuda <eyal_yehuda at yahoo dot com>, "systemtap at sourceware dot org" <systemtap at sourceware dot org>
- Date: Sun, 30 Jul 2017 10:30:12 +0300
- Subject: Re:
- Authentication-results: sourceware.org; auth=none
- References: <457412720.592599.1501166820895.ref@mail.yahoo.com> <457412720.592599.1501166820895@mail.yahoo.com> <CANA-60pWhi1wLcrAz5_w90D0aG_aWD9LrwdVA2X+Q_=g_H_=OQ@mail.gmail.com> <CANA-60o4-1eGegpMk-dWerDaz02RJDYVrQQ-7ek9g3H85zz9zA@mail.gmail.com> <CANA-60qX-Jo2RnykRu4+BMkqr-G1zmCMkOE9mMqCz3OfEn=0tQ@mail.gmail.com> <CAFwN=+zeAQiNRviAsdDCJ4b-c9V6ubEzcJfV+-haLvZQt=8Ppg@mail.gmail.com> <CANA-60qFUiEiOH-vVDNT-FH_4qdz7Qu6Qwy-8+X8A4ziLgU5Dg@mail.gmail.com> <CANA-60pBswvHdsGeMvvcn7niBWTN1j7nTGD=i=H_8DM4jWC8Dw@mail.gmail.com> <CANA-60pUeo978E1ax9Y1t_oB+k1TGCkKwxhZ4=tV=Kx9-TMApA@mail.gmail.com>
STAP runtime flag "-d MODULE" can potentially expose the external
symbols in the module. I have never tried to use it.
On Sun, Jul 30, 2017 at 10:26 AM, Arkady <arkady.miasnikov@gmail.com> wrote:
> See 6.3 in https://www.kernel.org/doc/Documentation/kbuild/modules.txt
> for intricate details. The bottom line is that STAP builds modules in
> a specific environment which can not be easily (or at least I do not
> know how to) controlled.
>
> On Sun, Jul 30, 2017 at 10:17 AM, Arkady <arkady.miasnikov@gmail.com> wrote:
>> This link can be relevant
>> https://stackoverflow.com/questions/33718358/insmod-fails-with-unknown-symbol-in-module-for-a-symbol-defined-in-another-mod
>>
>> On Sun, Jul 30, 2017 at 10:09 AM, Arkady <arkady.miasnikov@gmail.com> wrote:
>>> Daniel,
>>>
>>> STAP attempts to compile the kernel module against the kernel source
>>> tree installed in the system. One way to resolve the issue is to
>>> update the kernel source tree installed in the system. Without
>>> updating the kernel sources the compilation will fail.
>>>
>>> On Sun, Jul 30, 2017 at 9:56 AM, Daniel Doron <danielmeirdoron@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> How is systemtap's kernel module different from other modules?
>>>> Given module A which exports a function and module B using that exported
>>>> function and A is insmodded before B, I have not seen any problem. Why is
>>>> Systemtap's KO different? Is it not the kernel's job to resolve the exported
>>>> symbol?
>>>>
>>>>
>>>> Daniel.
>>>>
>>>> On Thu, 27 Jul 2017 at 19:29 Arkady <arkady.miasnikov@gmail.com> wrote:
>>>>>
>>>>> P.S.1 There is yet another way - very cool way - to to add a symbol to
>>>>> the stap script - via /sys/kernel/debug/systemtap/<module name>/.cmd
>>>>> (CTL_CHANNEL_NAME below)
>>>>>
>>>>> This is a Python sample
>>>>>
>>>>> def write_relocate_string(f, s, size):
>>>>> f.write(s)
>>>>> padding = size - len(s)
>>>>> for _ in range(padding):
>>>>> f.write('\x00')
>>>>>
>>>>> def write_relocate(module_name, symbol_name, address):
>>>>> STP_MODULE_NAME_LEN = 128
>>>>> STP_SYMBOL_NAME_LEN = 128
>>>>>
>>>>> with open(CTL_CHANNEL_NAME, "wb") as f:
>>>>> # STAP_RELOCATE (9)
>>>>> f.write(struct.pack('<I', 9))
>>>>> write_relocate_string(f, module_name, STP_MODULE_NAME_LEN)
>>>>> write_relocate_string(f, symbol_name, STP_SYMBOL_NAME_LEN)
>>>>> f.write(struct.pack('<Q', address))
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Jul 27, 2017 at 7:20 PM, Arkady <arkady.miasnikov@gmail.com>
>>>>> wrote:
>>>>> > P.S. Another idea is to use kernel IPC instead of function calls.
>>>>> > Files, named pipes, sockets, memory mapped files - just about anything
>>>>> > can do the trick.
>>>>> >
>>>>> > On Thu, Jul 27, 2017 at 7:03 PM, Arkady <arkady.miasnikov@gmail.com>
>>>>> > wrote:
>>>>> >> Eyal,
>>>>> >>
>>>>> >> Utility stap employs the kernel debug information installed in the
>>>>> >> system. When you load a module command insmod does not update the
>>>>> >> kernel debug info. There are simple two ways to resolve the issue
>>>>> >> * Load the module, get the address of the function foo using
>>>>> >> /proc/kallsyms, provide the address of the foo as an argument to the
>>>>> >> STAP script.
>>>>> >> * Link the module statically with the kernel and update the
>>>>> >> kernel-debuginfo
>>>>> >>
>>>>> >> Arkady
>>>>> >>
>>>>> >> On Thu, Jul 27, 2017 at 5:47 PM, Eyal Yehuda via systemtap
>>>>> >> <systemtap@sourceware.org> wrote:
>>>>> >>>
>>>>> >>>
>>>>> >>> hello
>>>>> >>>
>>>>> >>> The
>>>>> >>> issue I am facing is of undefined symbols in system tap kernel module
>>>>> >>> when trying to link to another kernel module with exported symbol
>>>>> >>>
>>>>> >>> please note environment setup:
>>>>> >>> os: debian 9 stretch 64bit
>>>>> >>> kernel version: 4.9.0-3-amd64
>>>>> >>>
>>>>> >>>
>>>>> >>> I created the following simple systemtap script using guru
>>>>> >>> mode
>>>>> >>>
>>>>> >>> system tap script: temp.stp
>>>>> >>>
>>>>> >>>
>>>>> >>> %{
>>>>> >>> extern int foo(void);
>>>>> >>> %}
>>>>> >>>
>>>>> >>> function call_foo:long() %{/* pure */
>>>>> >>> STAP_RETVALUE = foo();
>>>>> >>> %}
>>>>> >>>
>>>>> >>> probe syscall.open {
>>>>> >>> printf("foo() : %d\n", call_foo())
>>>>> >>> }
>>>>> >>>
>>>>> >>>
>>>>> >>>
>>>>> >>>
>>>>> >>> In addition to that I created the following simple kernel module with
>>>>> >>> an exported symbol function to be used from inside system tap
>>>>> >>> script
>>>>> >>>
>>>>> >>> kernel module: xxx.c
>>>>> >>>
>>>>> >>> #include <linux/kernel.h>
>>>>> >>> #include <linux/module.h>
>>>>> >>> #include <linux/printk.h>
>>>>> >>>
>>>>> >>> static int foo(void){
>>>>> >>> pr_info("foo()\n"); return 0;
>>>>> >>> }
>>>>> >>> EXPORT_SYMBOL(foo);
>>>>> >>>
>>>>> >>> static int __init xxx_init(void){
>>>>> >>> pr_info("xxx_init()\n");
>>>>> >>> return 0;
>>>>> >>> }
>>>>> >>>
>>>>> >>> static void __exit xxx_exit(void){
>>>>> >>> pr_info("xxx_exit() \n");
>>>>> >>> }
>>>>> >>>
>>>>> >>> module_init(xxx_init);
>>>>> >>> module_exit(xxx_exit);
>>>>> >>> MODULE_LICENSE("GPL");
>>>>> >>>
>>>>> >>>
>>>>> >>> After compiling the kernel module and insmoding it, I run the
>>>>> >>> command to compile systemtap script as follow:
>>>>> >>> stap -gk -v temp.stp
>>>>> >>>
>>>>> >>> the compilation command result with the following error:
>>>>> >>>
>>>>> >>>
>>>>> >>> Pass 1: parsed user script and 465 library scripts using
>>>>> >>> 113912virt/46752res/6476shr/40564data kb, in 110usr/20sys/124real ms.
>>>>> >>>
>>>>> >>> Pass 2: analyzed script: 2 probes, 2 functions, 99 embeds, 0 globals
>>>>> >>> using 165912virt/100212res/7716shr/92564data kb, in
>>>>> >>> 680usr/50sys/731real
>>>>> >>> ms.
>>>>> >>>
>>>>> >>> Pass 3: translated to C into "/tmp/stapzKUZ11/stap_1837_src.c" using
>>>>> >>> 165912virt/100404res/7908shr/92564data kb, in 10usr/10sys/9real ms.
>>>>> >>> WARNING: "foo" [/tmp/stapzKUZ11/stap_1837.ko] undefined!
>>>>> >>> Pass 4: compiled C into "stap_1837.ko" in
>>>>> >>> 1410usr/120sys/1963real ms.
>>>>> >>> Pass 5: starting run.
>>>>> >>> ERROR: Couldn't insert module '/tmp/stapzKUZ11/stap_1837.ko':
>>>>> >>> Invalid parameters
>>>>> >>> WARNING: /usr/bin/staprun exited with status: 1
>>>>> >>> Pass 5: run completed in 0usr/0sys/16real ms.
>>>>> >>> Pass 5: run failed. [man error::pass5]
>>>>> >>> Tip: /usr/share/doc/systemtap/README.Debian should help you
>>>>> >>> get started.
>>>>> >>> Keeping temporary directory "/tmp/stapzKUZ11"
>>>>> >>>
>>>>> >>>
>>>>> >>> in dmesg I can see the following lines
>>>>> >>>
>>>>> >>> Jul 27 03:56:14 debian9 kernel: [ 3938.166906] stap_2230: no
>>>>> >>> symbol version for foo
>>>>> >>> Jul 27 03:56:14 debian9 kernel: [ 3938.166908] stap_2230:
>>>>> >>> Unknown symbol foo (err -22)
>>>>> >>>
>>>>> >>>
>>>>> >>> I also checked /proc/kallsyms
>>>>> >>>
>>>>> >>>
>>>>> >>> cat /proc/kallsyms | grep xxx
>>>>> >>> ffffffffc0584000 T foo [xxx]
>>>>> >>> ffffffffc0584014 t xxx_exit [xxx]
>>>>> >>> ffffffffc0585030 r __ksymtab_foo [xxx]
>>>>> >>> ffffffffc058506e r __kstrtab_foo [xxx]
>>>>> >>> ffffffffc0585040 r __kcrctab_foo [xxx]
>>>>> >>> ffffffffc0586000 d __this_module [xxx]
>>>>> >>> ffffffffc0584014 t cleanup_module [xxx]
>>>>> >>>
>>>>> >>>
>>>>> >>> Please advice
>>>>> >>> 1.
>>>>> >>> what should i do in addition to solve this issue as I must create an
>>>>> >>> interaction between systemtap script and another kernel module
>>>>> >>> 2.
>>>>> >>> In case this is not feasible can u please offer another alternative
>>>>> >>> to
>>>>> >>> make systemtap script use another kernel module services ?
>>>>> >>>
>>>>> >>> Thanks eyal yehuda
>>>>>
>>>>