Re: malloc() performance and pthreads

[Wolfram Gloger <Wolfram dot Gloger at dent dot med dot uni-muenchen dot de>]

Hello,

>  I've recently discovered that KDE3 applications runs slower compared to 
> their KDE2 versions, even if the source is almost the same. The reason for 
> this turned out to be the fact that KDE3 libraries link to threaded version 
> of Qt (and hence also -lpthread), unlike KDE2. Linking to -lpthread causes 
> malloc() to do locking, and since dynamic memory allocation is very 
> extensively used in Qt/KDE for various reasons, this causes noticeable 
> performance decrease. The "right" way of fixing this problem might be making 
> Qt/KDE not to use malloc() so extensively,

I disagree, cutting down on dynamic memory allocation is (in C++ at
least) not an option :-).

However, if you have _mainly_ non-threaded apps, the "right" way to
fix this might be to have a version of your shared libraries _not_
linked to libpthread, IMHO.  (If that means having two sets of libs,
well there is a tradeoff but I wouldn't discount the possibility
straight away.)

>  The first question is, if you can do something about the malloc() 
> implementation in glibc (e.g. using spinlocks instead of mutexes, as that 
> seems to improve the performance noticeably).

I've already noticed that, too.

This is being worked on right now.  If you checkout the current
glibc-2.3 from CVS, you will see in malloc/thread-m.h that spinlocks
are already being used on ix86 and x86_64.

> And since I'm assuming you 
> cannot do as much optimizations in malloc() in glibc as I did in mine, I'm 
> thinking about making KDE use its own version,

I'd discourage that -- people won't be able to substitute malloc
easily, or use debugging hooks.  Future versions of malloc will use
madvise() on Linux which might give memory savings.  The version in
glibc-2.3 is based on malloc-2.7.0 from Doug Lea, which has been
reported to be very memory-efficient in mozilla, for example.  If you
use the malloc in glibc, you will always profit from such changes
automatically.

>  Could somebody please tell me which parts from LinuxThreads should I use? In 
> the sources I linked above, I just used testandset() with sched_yield() for 
> locking, and just reseting the value for unlocking (search for KLM_THR). Is 
> that sufficient for malloc(), or what's missing?

If you're speaking about ptmalloc, then yes, that should be
sufficient, but is probably not optimal for multiple threads (anyway
you said you don't have many multi-threaded apps yet so it very
probably doesn't matter).

Regards,
Wolfram.