This is the mail archive of the systemtap@sourceware.org mailing list for the systemtap project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: [pcp] suitability of PCP for event tracing


On 13/09/2010 11:29 PM, Max Matveev wrote:
On Mon, 13 Sep 2010 02:43:03 +1000, Ken McDonell wrote:

kenj> Some of the suggestions to date include ...

  kenj>  + data filtering predicates pushed from a client to pmcd and then on to
  kenj>  a pmda to enable or restrict the types of events or conditions on event
  kenj>  parameters that would be evaluated before asynchronously sending
  kenj>  matching events to the client

How would that work if multiple clients request mutually exclusive
predicates?

I wonder about this also. But my initial guess would be that predicates from different clients could be combined with a boolean OR to register the union of the events of interest, and then applying individual predicates to the stream to produce the events for each client. If the underlying event mechanism cannot support this, e.g. some process based cpu event registers where only one process at a time can be traced, then "first in best dressed" is probably the only protocol that will work.



  kenj>  + additional per-client state data being held in pmcd to allow rate
  kenj>  aggregation (and similar temporal averaging) to be done at pmcd, rather
  kenj>  than the client [note I have a long-standing objection to this approach
  kenj>  based on the original design criteria that pmcd needs to be mean and
  kenj>  lean to reduce impact, and data reduction and analysis should be pushed
  kenj>  out to the clients where the computation can be done without impacting
  kenj>  the system being monitored ... but maybe it is time to revist this, as
  kenj>  the current environments where PCP is being used may differ from those
  kenj>  we were concerned with in 1994]

Recent (2010) experience on 3rd rate platform suggests that this is
still an issue - doing too much calculation in pmda is adding to the
time it takes to fetch the data, unless pmcd can magically hide delays
induced by calculations it has to make or calculations made by pmda
ALL clients suffer.

Yep that is always a risk ... and as a general rule I'd like to see collection being restricted at the pmcd site (e.g. don't probe for stats that no one is asking for) and aggregated processing like rate calculations moved out to the clients.


  kenj>  Depending on the set of goals we agree on, there may even be a place to
  kenj>  consider maintaining the poll-based mechanism, but the export data is a
  kenj>  variable length buffer of all event records (each aggregated and
  kenj>  self-identifying as below) seen since the last poll-based sample.

How will this work with multiple clients? Will the clients get a "snap
time" to indicate when pmda updated its buffer or will pmda need to
remember the state of each client (which would mean dragging client
information into pmda, maintaining that information and somehow
retiring per-client state without affecting the clients).


I think it would need per-client state in the pmda (which assumes a protocol change between pmcd and the pmdas) ... the implementation might involve a dynamic ring buffer, sequence numbers and last fetched sequence number per client.


The advantage is we maintain polled and synchronous protocols between the clients and pmcd ... it is exactly these sort of pros and cons that I'd like to see us discussing.



Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]