This is the mail archive of the
systemtap@sourceware.org
mailing list for the systemtap project.
Re: No output from trace file after a few hours
- From: Daniel Doron <danielmeirdoron at gmail dot com>
- To: Arkady <arkady dot miasnikov at gmail dot com>
- Cc: systemtap at sourceware dot org
- Date: Thu, 15 Jun 2017 15:01:14 +0300
- Subject: Re: No output from trace file after a few hours
- Authentication-results: sourceware.org; auth=none
- References: <CAFwN=+ywma7CvB6AajmEb99LKZkTg9fWP724to6r=FrKrB0stQ@mail.gmail.com> <CANA-60qKnB_x4UuPfej_H2ebkHNXepmqpE+dgnOT=kOV6MYbqw@mail.gmail.com> <CANA-60oP_i=0oCQ1e73B_96juT1gW10d2Q7yps_r-jPyo=AJKQ@mail.gmail.com> <CAFwN=+xs21q38vf_ZNN-QAr5-EOfSkoKUx-e3u8PaZ=GKOkkdA@mail.gmail.com> <CAFwN=+yn+TGsHKNmoCJBaqeJakRG=XS8uVGbN8a16a0z3UdXmQ@mail.gmail.com> <CAFwN=+xMd=iXUw79A0WkpjHY1Vii3tFzWh4=5vS7f9qdtfdSAQ@mail.gmail.com> <CAFwN=+x34nb0ZYBW1wYB=-4m-Ctyrd-tkT4wXS0bcDTdBNaOtg@mail.gmail.com> <CANA-60pBnzZrAkg_WJ1bLr+e1atAGeFbUcbU_ffrW=hTj0Q+dw@mail.gmail.com> <CAFwN=+w2u+QpZyPavfxf2i+QsrhaHbSnRcGzYLPOpLwK4TA4vA@mail.gmail.com> <CANA-60o1VWpywY6-YZCuyjHYVXxakRoZ4f=JnD9rRERH-rpjQA@mail.gmail.com> <CAFwN=+yAGLbX+a8fQQ2h2doXrEd2-OxLWKPO11xb8G96UifYOw@mail.gmail.com> <CANA-60r=U8tmEfiQJxfQ-u+vdDBgwHFzZHthXbL3uQbusrYNew@mail.gmail.com> <CANA-60oB-kzF0YQhJjht-Hhrv73O-0j-PtJdUF7HQg+F7Hbv8A@mail.gmail.com> <CANA-60qX659-jTbKu75wga6oSLktTEya-2fdeS1Jb-4=xFM1gQ@mail.gmail.com> <CAFwN=+xJc3nZg8U6pMCakrmnDS87t=LH2OboTDb4PwdZeJJF8g@mail.gmail.com> <CANA-60q0Y+nAJ2+gkTe=3KcqQH8xRbowUZMB-VUazXh-e4AoTw@mail.gmail.com> <CANA-60oaD=MEa1QWksyVrcFsKDc9tOcxDJAZEPYWRb_5=SOG6Q@mail.gmail.com>
ok
On Thu, Jun 15, 2017 at 2:59 PM, Arkady <arkady.miasnikov@gmail.com> wrote:
> you can also try --suppress-time-limits
>
> On Thu, Jun 15, 2017 at 2:32 PM, Arkady <arkady.miasnikov@gmail.com> wrote:
>> One possible reason is a system update which caused lot of probes to trigger.
>>
>> On Thu, Jun 15, 2017 at 2:13 PM, Daniel Doron <danielmeirdoron@gmail.com> wrote:
>>> I will try that.
>>> BTW, this also just happened after almost 24 hours on kernel 4.11.3-1-ARCH
>>>
>>> On Thu, Jun 15, 2017 at 2:05 PM, Arkady <arkady.miasnikov@gmail.com> wrote:
>>>> This works fine
>>>>
>>>> sudo stap -D MAXSKIPPED=0 -D MAXTRYLOCK=1000 -D TRYLOCKDELAY=100 -F
>>>> -o ./test.log -S 1,2 test.stp
>>>>
>>>> On Thu, Jun 15, 2017 at 2:03 PM, Arkady <arkady.miasnikov@gmail.com> wrote:
>>>>> STAP simply quits in a couple of seconds
>>>>>
>>>>> sudo stap -F -o ./test.log -S 1,2 test.stp
>>>>> 94089
>>>>> ps -ef | grep -v "grep" | grep 94089
>>>>>
>>>>> On Thu, Jun 15, 2017 at 1:53 PM, Arkady <arkady.miasnikov@gmail.com> wrote:
>>>>>> Interesting. I have reproduced the problem by running two tight bash
>>>>>> loops simultaneously
>>>>>>
>>>>>> sudo stap -F -o ./test.log -S 1,2 test.stp
>>>>>> while [ 1 ];do ls > /dev/null;done &
>>>>>> while [ 1 ];do echo -en "\033[0;0H";ls -l | grep "test.log";echo -en
>>>>>> "\033[0;0H";done;
>>>>>>
>>>>>> Linux centos6 2.6.32-642.el6.x86_64 #1 SMP Tue May 10 17:27:01 UTC
>>>>>> 2016 x86_64 x86_64 x86_64 GNU/Linux
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Jun 15, 2017 at 11:59 AM, Daniel Doron
>>>>>> <danielmeirdoron@gmail.com> wrote:
>>>>>>> On that specific kernel about 8-10 hours.
>>>>>>> On my machine with latest kernel, been running for 24 hours.
>>>>>>> I need to check other distros.
>>>>>>>
>>>>>>> sudo stap -v -F -m mymodule test.stp
>>>>>>>
>>>>>>> On Thu, Jun 15, 2017 at 11:26 AM, Arkady <arkady.miasnikov@gmail.com> wrote:
>>>>>>>> Do you do something like
>>>>>>>>
>>>>>>>> sudo stap -F -o ./test.log -S 1,2 test.stp ?
>>>>>>>>
>>>>>>>> On Thu, Jun 15, 2017 at 11:25 AM, Daniel Doron
>>>>>>>> <danielmeirdoron@gmail.com> wrote:
>>>>>>>>> On that specific kernel about 8-10 hours.
>>>>>>>>> On my machine with latest kernel, been running for 24 hours.
>>>>>>>>> I need to check other distros.
>>>>>>>>>
>>>>>>>>> On Jun 15, 2017 11:18, "Arkady" <arkady.miasnikov@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>> How long time it runs for you?
>>>>>>>>>>
>>>>>>>>>> On Thu, Jun 15, 2017 at 10:38 AM, Daniel Doron
>>>>>>>>>> <danielmeirdoron@gmail.com> wrote:
>>>>>>>>>> > Systemtap translator/driver (version 2.9/0.164, rpm 2.9-7.el6)
>>>>>>>>>> > Copyright (C) 2005-2015 Red Hat, Inc. and others
>>>>>>>>>> > This is free software; see the source for copying conditions.
>>>>>>>>>> > enabled features: AVAHI LIBRPM LIBSQLITE3 NLS NSS TR1_UNORDERED_MAP
>>>>>>>>>> >
>>>>>>>>>> > On Thu, Jun 15, 2017 at 10:37 AM, Daniel Doron
>>>>>>>>>> > <danielmeirdoron@gmail.com> wrote:
>>>>>>>>>> >> BTW kernel version: 2.6.32-431.el6.x86_64
>>>>>>>>>> >>
>>>>>>>>>> >> On Thu, Jun 15, 2017 at 10:36 AM, Daniel Doron
>>>>>>>>>> >> <danielmeirdoron@gmail.com> wrote:
>>>>>>>>>> >>> if you mean that the kernel version does not get upgraded, then yes.
>>>>>>>>>> >>>
>>>>>>>>>> >>> On Thu, Jun 15, 2017 at 10:34 AM, Daniel Doron
>>>>>>>>>> >>> <danielmeirdoron@gmail.com> wrote:
>>>>>>>>>> >>>> Sorry, I do not know what you mean by that...
>>>>>>>>>> >>>>
>>>>>>>>>> >>>> On Thu, Jun 15, 2017 at 10:32 AM, Arkady <arkady.miasnikov@gmail.com>
>>>>>>>>>> >>>> wrote:
>>>>>>>>>> >>>>> Do you have the kernel version pinned?
>>>>>>>>>> >>>>>
>>>>>>>>>> >>>>> On Thu, Jun 15, 2017 at 10:12 AM, Arkady
>>>>>>>>>> >>>>> <arkady.miasnikov@gmail.com> wrote:
>>>>>>>>>> >>>>>> Please send the stap script
>>>>>>>>>> >>>>>>
>>>>>>>>>> >>>>>> On Thu, Jun 15, 2017 at 10:10 AM, Daniel Doron
>>>>>>>>>> >>>>>> <danielmeirdoron@gmail.com> wrote:
>>>>>>>>>> >>>>>>> Hi,
>>>>>>>>>> >>>>>>>
>>>>>>>>>> >>>>>>> I built my module to probe a few system calls and functions.
>>>>>>>>>> >>>>>>> Nothing
>>>>>>>>>> >>>>>>> fancy, no big work in it, just prints some data. Then I wrote a
>>>>>>>>>> >>>>>>> simple
>>>>>>>>>> >>>>>>> python script to read data directly off the
>>>>>>>>>> >>>>>>> /sys/kernel/debug/systemtap/mymodule/trace0 file (of course
>>>>>>>>>> >>>>>>> running as
>>>>>>>>>> >>>>>>> sudo)
>>>>>>>>>> >>>>>>> I run stap in Flight recorder mode.
>>>>>>>>>> >>>>>>> After a few hours (varies from 8 to 20) I stop getting any output
>>>>>>>>>> >>>>>>> from
>>>>>>>>>> >>>>>>> the module...
>>>>>>>>>> >>>>>>> Any idea how to check why that is?
>>>>>>>>>> >>>>>>> dmesg does not report anything unusual...
>>>>>>>>>> >>>>>>>
>>>>>>>>>> >>>>>>> Thanks.
>>>>>>>>>> >>>>>>> Daniel.