Re: setting of initial bottom of stack variable in pthread.c (again)

[Jay <j2list468 at yahoo dot com>]

--- Daniel Jacobowitz <drow@false.org> wrote:

> Sorry for not getting back to you sooner.  This
> stuff gives me
> a headache.
> 
> On Tue, Aug 08, 2006 at 08:09:58AM -0700, Jay wrote:
> > I posted this on the alpha mailing list, but this
> > might be a better place for it.  From what I
> > understand, when __pthread_initial_thread_bos is
> set
> > in pthread.c, it must be set to cover both the
> initial
> > thread and manager thread.  Looking at the
> > STACK_GROWS_DOWN case, it looks like this variable
> is
> > set to STACK_SIZE beyond the space it needs to to
> > cover the initial and manager thread. 
> Graphically:
> > -------------------   top of initial stack
> > | Initial thread  |
> > |                 |
> > -------------------
> > | Manager thread  |
> > |                 |
> > -------------------
> > | Free space      |
> > |                 |
> > -------------------   __pthread_initial_thread_bos
> 
> No.
> 
> The tops and bottoms of stacks must be aligned to
> STACK_SIZE in this
> model.  The initial stack is determined by the OS,
> and can end up just
> about anywhere.  Also, it looks to me like the 2* is
> unnecessary;
> this is not covering the manager thread.  So the
> initial thread stack
> is a bit of alignment padding (up to STACK_SIZE)
> plus 2*STACK_SIZE.

I understand that the top and bottom of the stacks
must be aligned to STACK_SIZE.  I'm having trouble
with following the last statement.  If the 2* is
uncessary, why the alignment padding (up to
STACK_SIZE) plus 2*STACK_SIZE?  I'm really not
grasping the plus 2*STACK_SIZE at all.

> 
> The manager thread stack is allocated later:
> 
>   __pthread_manager_thread_bos =
> malloc(THREAD_MANAGER_STACK_SIZE);
> 
> That's on the heap somewhere.  Does that end up
> above or below the
> initial thread for you?

It looks like the manager thread is definetly well
below the stack.  I put some printfs in pthread.c
(yes, I know it's very dirty), so you can see the
output below.

> 
> > When the very first worker thread gets assigned
> its
> > stack space it is getting assigned the space in
> free
> > space which is above __pthread_initial_thread_bos
> > (instead of something below
> > __pthread_initial_thread_bos as it should since it
> is
> > not part of the initial thread), so when
> thread_self
> > is called from the first worker thread,
> thread_self
> > returns the initial thread, not the worker thread.
> 
> > All other threads created after the first worker
> > thread are assigned memory below
> > __pthread_initial_thread_bos.  The line that sets
> > __pthread_initial_thread_bos is
> >   __pthread_initial_thread_bos =
> >     (char *)(((long)CURRENT_STACK_FRAME - 2 *
> > STACK_SIZE) & ~(STACK_SIZE - 1));
> > 
> > Changing the 2 in the above equation to a 1 would
> > solve the problem of __pthread_initial_thread_bos
> > being set beyond the free space depicted above. 
> Is
> > what is depicted above correct and it's a problem
> with
> > the memory the kernel is handing out, or is this a
> > bug, and the 2 should be a 1 since the stack is
> > growing down?
> 
> The kernel is responsible for setting the stack size
> of the initial
> VMA.  The heap used to be way down at the other end
> of memory and grow
> up; but there've been a lot of changes in that
> respect lately
> and it may be that big mmaps on your platform are
> being placed
> directly below the stack.
> 
> I think you need to look at the /proc/PID/maps file
> for a program that
> shows this problem.  How big is the stack, really? 
> And what ulimits
> do you have set (ulimit -a) - are they changing the
> stack size?
> 
> -- 
> Daniel Jacobowitz
> CodeSourcery
> 

I haven't found anywhere were the limit is changed
yet, but something does not look right.  The output
(see below) from ulimit says that the stack size is
8192kB.  The STACK_SIZE variable defined in
internals.h and used in pthread.c is set to
1024*1024*2 (2097152).  Could this be causing a
problem with these two sizes not being set right?

Among the stuff I printed out was the stack pointer
when the worker threads (both use the same function
which only has a while loop that delays them exiting)
exit.  As you can see, the sp for the first thread is
within the bounds for the initial thread (I also
printed out the tos & bos for the initial thread).

# pthread_test &
[1] 306
# initial thread bos = BFA00000
initial thread tos = C0000000
stack size 00200000
manager thread bos = 00010650
manager thread tos = 00012630
manager thread stack size 00001FE0
ps
  PID  Uid     VmSize Stat Command
  306 root        472 R   pthread_test
  307 root        472 S   pthread_test
  308 root        472 R   pthread_test
  309 root        472 R   pthread_test
  312 root        600 R   ps
  314 root        352 S   usleep 500000
# cat /proc/306/maps
00008000-00009000 r-xp 00000000 01:00 2258      
/usr/bin/pthread_test
00010000-00011000 rw-p 00000000 01:00 2258      
/usr/bin/pthread_test
00011000-00013000 rwxp 00000000 00:00 0
40000000-40014000 r-xp 00000000 01:00 2176      
/lib/ld-linux.so.2
40014000-40016000 rw-p 00000000 00:00 0
4001b000-4001d000 rw-p 00013000 01:00 2176      
/lib/ld-linux.so.2
4001d000-4002b000 r-xp 00000000 01:00 3208      
/lib/libpthread.so.0
4002b000-4002d000 ---p 0000e000 01:00 3208      
/lib/libpthread.so.0
4002d000-40038000 rw-p 00008000 01:00 3208      
/lib/libpthread.so.0
40038000-4003a000 rw-p 00000000 00:00 0
4003a000-4013a000 r-xp 00000000 01:00 3222      
/lib/libc.so.6
4013a000-40146000 rw-p 000f8000 01:00 3222      
/lib/libc.so.6
40146000-4014b000 rw-p 00000000 00:00 0
bf800000-bf801000 ---p 00000000 00:00 0
bf801000-bfa00000 rwxp 00001000 00:00 0
bfa00000-bfa01000 ---p 00000000 00:00 0
bfa01000-bfc00000 rwxp 00001000 00:00 0
bffff000-c0000000 rwxp 00000000 00:00 0
# thread exiting
exit CURRENT_STACK_FRAME BFBFFAA4
thread exiting
exit CURRENT_STACK_FRAME BF9FFAA4
ps
  PID  Uid     VmSize Stat Command
  306 root        476 R   pthread_test
  307 root        476 S   pthread_test
  308 root        476 S   pthread_test
  383 root        352 S   usleep 500000
  384 root        600 R   ps
# cat /proc/306/maps
00008000-00009000 r-xp 00000000 01:00 2258      
/usr/bin/pthread_test
00010000-00011000 rw-p 00000000 01:00 2258      
/usr/bin/pthread_test
00011000-00013000 rwxp 00000000 00:00 0
40000000-40014000 r-xp 00000000 01:00 2176      
/lib/ld-linux.so.2
40014000-40016000 rw-p 00000000 00:00 0
4001b000-4001d000 rw-p 00013000 01:00 2176      
/lib/ld-linux.so.2
4001d000-4002b000 r-xp 00000000 01:00 3208      
/lib/libpthread.so.0
4002b000-4002d000 ---p 0000e000 01:00 3208      
/lib/libpthread.so.0
4002d000-40038000 rw-p 00008000 01:00 3208      
/lib/libpthread.so.0
40038000-4003a000 rw-p 00000000 00:00 0
4003a000-4013a000 r-xp 00000000 01:00 3222      
/lib/libc.so.6
4013a000-40146000 rw-p 000f8000 01:00 3222      
/lib/libc.so.6
40146000-4014b000 rw-p 00000000 00:00 0
bf800000-bf801000 ---p 00000000 00:00 0
bf801000-bfa00000 rwxp 00001000 00:00 0
bfa00000-bfa01000 ---p 00000000 00:00 0
bfa01000-bfc00000 rwxp 00001000 00:00 0
bffff000-c0000000 rwxp 00000000 00:00 0
# ulimit -a
core file size (blocks)     0
data seg size (kbytes)      unlimited
file size (blocks)          unlimited
max locked memory (kbytes)  unlimited
max memory size (kbytes)    unlimited
open files                  1024
pipe size (512 bytes)       8
stack size (kbytes)         8192
cpu time (seconds)          unlimited
max user processes          512
virtual memory (kbytes)     unlimited



Thanks

Jay

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com