Re: runtime committed to cvs

["Frank Ch. Eigler" <fche at redhat dot com>]

Hi -

hunt wrote:
> [...]
> > > Currently only small amounts of memory (the length of a string or two)
> > > are allocated using GFP_ATOMIC.  
> > 
> > Really?  I see lots of chunks like
> >      m->key1.str = _stp_alloc(strlen(map->c_key1.str) + 1);
> >      strcpy(m->key1.str, map->c_key1.str);
> > throughout the map code.
> 
> That allocates memory for a string key.

And we get the same thing elsewhere for string values.  In other
words, yes, strings passed to/from maps are individually dynamically
allocated using GFP_ATOMIC.  Bad juju.


> [...]
> Alternatives?
> 
> 1. Allocate a big chunk of memory to use for string storage. A malloc
> call just grabs some memory from that and advances a pointer by the
> length of the string.  Free does nothing.  [...]

That's a possibility, though note that a long-running probe can
manipulate many strings during its lifetime.

> 2. Some kind of memory pool that is enlarged as needed by a work queue.
> Isn't this like the memory pool for GFP_ATOMIC?
> 3. Allocate a big chunk of memory and write our own malloc/free to
> manage it.

These are possibilities, made simpler if the sizes of the
dynamically-allocated objects are fixed, and even simpler if the
maximum potential *count* is fixed ahead of time too.

> 4 ???

What I prototyped: fixed-size string objects that are copied by value,
just like integers and statistics buffers.  This was another reason
for the #define templating mechanism: to pay for string buffers only
in those hash tables that needed them.


> > I posit that is simply because the code hasn't run much in adverse
> > enough environments.  Consider trying a module that allocates and
> > deallocates a hundred thousand stringy map elements.
> 
> We will, of course, do stress testing and performance monitoring.  But
> is that scenario realistic?  Hash tables are not going to give good
> performance with a hundred thousand keys.

Those keys don't have to be concurrently active - think fragmentation.
Consider a long-running probe that accumulates statistics on many
processes, and periodically purges the list.  Depending on how kmalloc
works, how much it rounds up your allocations, you might run out much
sooner than 100k entries.


- FChE

Attachment: pgp00000.pgp
Description: PGP signature