Re: [RFA] Skip the "red zone" on AMD64

[Andrew Cagney <ac131313 at redhat dot com>]

> This is pretty obvious patch that doesn't harm anything, but anyway - 
OK to apply?

Hmm,

> Andreas Schwab told me that:
> Michal Ludvig <mludvig@suse.cz> writes:
>
> |> 2003-08-07  Michal Ludvig  <mludvig@suse.cz>
> |>     * x86-64-tdep.c (x86_64_push_arguments): Skip the red zone.
>
> Please add an empty line after the heading.
>
> Done.
>
> Otherwise OK.
>
> Thanks, committed to 6.0 and head.

The PowerOpen (a.k.a. AIX) and (as I only just discovered) PPC64 ABIs 
make use of the stack beyond the top-of-stack (SP) address.  A very long 
standing bug is that GDB needs to skip that area before allocating stack 
space.

This architecture's `red zone' sounds like the same [mis-]feature.  If 
it is then the posted change won't fix the problem :-(  The return 
structure, already allocated on the stack, will smash that stack area.

I think this code:

  /* Ensure that the initial SP is correctly aligned.  */
  {
    CORE_ADDR old_sp = read_sp ();
    if (gdbarch_frame_align_p (current_gdbarch))
      {
        /* NOTE: cagney/2002-09-18:

           On a RISC architecture, a void parameterless generic dummy
           frame (i.e., no parameters, no result) typically does not
           need to push anything the stack and hence can leave SP and
           FP.  Similarly, a frameless (possibly leaf) function does
           not push anything on the stack and, hence, that too can
           leave FP and SP unchanged.  As a consequence, a sequence of
           void parameterless generic dummy frame calls to frameless
           functions will create a sequence of effectively identical
           frames (SP, FP and TOS and PC the same).  This, not
           suprisingly, results in what appears to be a stack in an
           infinite loop --- when GDB tries to find a generic dummy
           frame on the internal dummy frame stack, it will always
           find the first one.

           To avoid this problem, the code below always grows the
           stack.  That way, two dummy frames can never be identical.
           It does burn a few bytes of stack but that is a small price
           to pay :-).  */
        sp = gdbarch_frame_align (current_gdbarch, old_sp);
        if (sp == old_sp)
          {
            if (INNER_THAN (1, 2))
              /* Stack grows down.  */
              sp = gdbarch_frame_align (current_gdbarch, old_sp - 1);
            else
              /* Stack grows up.  */
              sp = gdbarch_frame_align (current_gdbarch, old_sp + 1);
          }
        gdb_assert ((INNER_THAN (1, 2) && sp <= old_sp)
                    || (INNER_THAN (2, 1) && sp >= old_sp));
      }

so that it uses a new architecture method (size_of_frame_red_zone? 
default 1, assert > 0, doco) to pad out the stack before using it.

If this is the case, can you please revert the patch and post something 
more along the above lines.  Is there code in the testsuite that 
fails/passes with the fix?

Andrew

PS: In case you're wondering.  The above code was only added a few 
months ago.  Prior to that there wasn't anywhere to put the fix.