[Bug python/19927] New: Python unwinder API should be resilient to recursion and frame cache reinitialization

["palves at redhat dot com" <sourceware-bugzilla at sourceware dot org>]

https://sourceware.org/bugzilla/show_bug.cgi?id=19927

            Bug ID: 19927
           Summary: Python unwinder API should be resilient to recursion
                    and frame cache reinitialization
           Product: gdb
           Version: HEAD
            Status: NEW
          Severity: normal
          Priority: P2
         Component: python
          Assignee: unassigned at sourceware dot org
          Reporter: palves at redhat dot com
  Target Milestone: ---

(Split from Bug 19288.  Text mostly Andrew's, slightly edited.)

It would be good if the gdb Python unwinder API could:

i) Detect recursive entry into an unwinder and;

ii) Detect that reinit_frame_stack (and, indeed, any other code which might
inadvertently leave a dangling pointer) has been called below a frame sniffer.
It should really throw an exception so the sniffer can back out rather than
leaving potential for a gdb crash.

I observed gdb crashes and also Python stack overflows in my initial
implementation.

The stack overflow only happens for some unwind attempts, in particular it
appears when the base frame for the pending frame has level -1. when the
unwinder calls back in to certain gdb routines gdb tries to re-establish the
frame stack. so the unwinder gets called again and makes the same callout and
so on.

The crash seems to be able to happen with or without the recursion and again
only happens for certain starting frames and only when the unwinder calls back
in to certain gdb routines.  The problem is that in these circumstances the gdb
routine decides to call method reinit_frame_stack.  Even if the recursion is
avoided this still ends up freeing the block that contains the base frame which
backs the pending_frame passed to the unwinder's __call__ method i.e. the code
which invoked __call__ is left with a dangling pointer.  By the time the
unwinder returns the block has usually been reallocated and scribbled over.
invariably, gdb asserts on return from __call__.

I bypassed the recursion problem by detecting recursive calls to my unwinder's
__call__ method (using an association list keyed by thread to detect an
outstanding call) and backing out of any call which results in recursion to
return None.  This appears to bypass the check of the invalid base frame and
leaves the problematic frames for some other sniffer to deal with.

I bypassed cases where a crash was caused without recursive entry simply by
tweaking my python code until it went away.  This is clearly a bit inadequate,
especially the latter 'fix' which is really a time-bomb.

-- 
You are receiving this mail because:
You are on the CC list for the bug.