This is the mail archive of the
systemtap@sourceware.org
mailing list for the systemtap project.
Re: No output from trace file after a few hours
- From: Arkady <arkady dot miasnikov at gmail dot com>
- To: Daniel Doron <danielmeirdoron at gmail dot com>
- Cc: systemtap at sourceware dot org
- Date: Thu, 15 Jun 2017 14:32:04 +0300
- Subject: Re: No output from trace file after a few hours
- Authentication-results: sourceware.org; auth=none
- References: <CAFwN=+ywma7CvB6AajmEb99LKZkTg9fWP724to6r=FrKrB0stQ@mail.gmail.com> <CANA-60qKnB_x4UuPfej_H2ebkHNXepmqpE+dgnOT=kOV6MYbqw@mail.gmail.com> <CANA-60oP_i=0oCQ1e73B_96juT1gW10d2Q7yps_r-jPyo=AJKQ@mail.gmail.com> <CAFwN=+xs21q38vf_ZNN-QAr5-EOfSkoKUx-e3u8PaZ=GKOkkdA@mail.gmail.com> <CAFwN=+yn+TGsHKNmoCJBaqeJakRG=XS8uVGbN8a16a0z3UdXmQ@mail.gmail.com> <CAFwN=+xMd=iXUw79A0WkpjHY1Vii3tFzWh4=5vS7f9qdtfdSAQ@mail.gmail.com> <CAFwN=+x34nb0ZYBW1wYB=-4m-Ctyrd-tkT4wXS0bcDTdBNaOtg@mail.gmail.com> <CANA-60pBnzZrAkg_WJ1bLr+e1atAGeFbUcbU_ffrW=hTj0Q+dw@mail.gmail.com> <CAFwN=+w2u+QpZyPavfxf2i+QsrhaHbSnRcGzYLPOpLwK4TA4vA@mail.gmail.com> <CANA-60o1VWpywY6-YZCuyjHYVXxakRoZ4f=JnD9rRERH-rpjQA@mail.gmail.com> <CAFwN=+yAGLbX+a8fQQ2h2doXrEd2-OxLWKPO11xb8G96UifYOw@mail.gmail.com> <CANA-60r=U8tmEfiQJxfQ-u+vdDBgwHFzZHthXbL3uQbusrYNew@mail.gmail.com> <CANA-60oB-kzF0YQhJjht-Hhrv73O-0j-PtJdUF7HQg+F7Hbv8A@mail.gmail.com> <CANA-60qX659-jTbKu75wga6oSLktTEya-2fdeS1Jb-4=xFM1gQ@mail.gmail.com> <CAFwN=+xJc3nZg8U6pMCakrmnDS87t=LH2OboTDb4PwdZeJJF8g@mail.gmail.com>
One possible reason is a system update which caused lot of probes to trigger.
On Thu, Jun 15, 2017 at 2:13 PM, Daniel Doron <danielmeirdoron@gmail.com> wrote:
> I will try that.
> BTW, this also just happened after almost 24 hours on kernel 4.11.3-1-ARCH
>
> On Thu, Jun 15, 2017 at 2:05 PM, Arkady <arkady.miasnikov@gmail.com> wrote:
>> This works fine
>>
>> sudo stap -D MAXSKIPPED=0 -D MAXTRYLOCK=1000 -D TRYLOCKDELAY=100 -F
>> -o ./test.log -S 1,2 test.stp
>>
>> On Thu, Jun 15, 2017 at 2:03 PM, Arkady <arkady.miasnikov@gmail.com> wrote:
>>> STAP simply quits in a couple of seconds
>>>
>>> sudo stap -F -o ./test.log -S 1,2 test.stp
>>> 94089
>>> ps -ef | grep -v "grep" | grep 94089
>>>
>>> On Thu, Jun 15, 2017 at 1:53 PM, Arkady <arkady.miasnikov@gmail.com> wrote:
>>>> Interesting. I have reproduced the problem by running two tight bash
>>>> loops simultaneously
>>>>
>>>> sudo stap -F -o ./test.log -S 1,2 test.stp
>>>> while [ 1 ];do ls > /dev/null;done &
>>>> while [ 1 ];do echo -en "\033[0;0H";ls -l | grep "test.log";echo -en
>>>> "\033[0;0H";done;
>>>>
>>>> Linux centos6 2.6.32-642.el6.x86_64 #1 SMP Tue May 10 17:27:01 UTC
>>>> 2016 x86_64 x86_64 x86_64 GNU/Linux
>>>>
>>>>
>>>>
>>>> On Thu, Jun 15, 2017 at 11:59 AM, Daniel Doron
>>>> <danielmeirdoron@gmail.com> wrote:
>>>>> On that specific kernel about 8-10 hours.
>>>>> On my machine with latest kernel, been running for 24 hours.
>>>>> I need to check other distros.
>>>>>
>>>>> sudo stap -v -F -m mymodule test.stp
>>>>>
>>>>> On Thu, Jun 15, 2017 at 11:26 AM, Arkady <arkady.miasnikov@gmail.com> wrote:
>>>>>> Do you do something like
>>>>>>
>>>>>> sudo stap -F -o ./test.log -S 1,2 test.stp ?
>>>>>>
>>>>>> On Thu, Jun 15, 2017 at 11:25 AM, Daniel Doron
>>>>>> <danielmeirdoron@gmail.com> wrote:
>>>>>>> On that specific kernel about 8-10 hours.
>>>>>>> On my machine with latest kernel, been running for 24 hours.
>>>>>>> I need to check other distros.
>>>>>>>
>>>>>>> On Jun 15, 2017 11:18, "Arkady" <arkady.miasnikov@gmail.com> wrote:
>>>>>>>>
>>>>>>>> How long time it runs for you?
>>>>>>>>
>>>>>>>> On Thu, Jun 15, 2017 at 10:38 AM, Daniel Doron
>>>>>>>> <danielmeirdoron@gmail.com> wrote:
>>>>>>>> > Systemtap translator/driver (version 2.9/0.164, rpm 2.9-7.el6)
>>>>>>>> > Copyright (C) 2005-2015 Red Hat, Inc. and others
>>>>>>>> > This is free software; see the source for copying conditions.
>>>>>>>> > enabled features: AVAHI LIBRPM LIBSQLITE3 NLS NSS TR1_UNORDERED_MAP
>>>>>>>> >
>>>>>>>> > On Thu, Jun 15, 2017 at 10:37 AM, Daniel Doron
>>>>>>>> > <danielmeirdoron@gmail.com> wrote:
>>>>>>>> >> BTW kernel version: 2.6.32-431.el6.x86_64
>>>>>>>> >>
>>>>>>>> >> On Thu, Jun 15, 2017 at 10:36 AM, Daniel Doron
>>>>>>>> >> <danielmeirdoron@gmail.com> wrote:
>>>>>>>> >>> if you mean that the kernel version does not get upgraded, then yes.
>>>>>>>> >>>
>>>>>>>> >>> On Thu, Jun 15, 2017 at 10:34 AM, Daniel Doron
>>>>>>>> >>> <danielmeirdoron@gmail.com> wrote:
>>>>>>>> >>>> Sorry, I do not know what you mean by that...
>>>>>>>> >>>>
>>>>>>>> >>>> On Thu, Jun 15, 2017 at 10:32 AM, Arkady <arkady.miasnikov@gmail.com>
>>>>>>>> >>>> wrote:
>>>>>>>> >>>>> Do you have the kernel version pinned?
>>>>>>>> >>>>>
>>>>>>>> >>>>> On Thu, Jun 15, 2017 at 10:12 AM, Arkady
>>>>>>>> >>>>> <arkady.miasnikov@gmail.com> wrote:
>>>>>>>> >>>>>> Please send the stap script
>>>>>>>> >>>>>>
>>>>>>>> >>>>>> On Thu, Jun 15, 2017 at 10:10 AM, Daniel Doron
>>>>>>>> >>>>>> <danielmeirdoron@gmail.com> wrote:
>>>>>>>> >>>>>>> Hi,
>>>>>>>> >>>>>>>
>>>>>>>> >>>>>>> I built my module to probe a few system calls and functions.
>>>>>>>> >>>>>>> Nothing
>>>>>>>> >>>>>>> fancy, no big work in it, just prints some data. Then I wrote a
>>>>>>>> >>>>>>> simple
>>>>>>>> >>>>>>> python script to read data directly off the
>>>>>>>> >>>>>>> /sys/kernel/debug/systemtap/mymodule/trace0 file (of course
>>>>>>>> >>>>>>> running as
>>>>>>>> >>>>>>> sudo)
>>>>>>>> >>>>>>> I run stap in Flight recorder mode.
>>>>>>>> >>>>>>> After a few hours (varies from 8 to 20) I stop getting any output
>>>>>>>> >>>>>>> from
>>>>>>>> >>>>>>> the module...
>>>>>>>> >>>>>>> Any idea how to check why that is?
>>>>>>>> >>>>>>> dmesg does not report anything unusual...
>>>>>>>> >>>>>>>
>>>>>>>> >>>>>>> Thanks.
>>>>>>>> >>>>>>> Daniel.