Watchpoint task roundup

[Phil Muldoon <pmuldoon at redhat dot com>]

Before I send this out to frysk@sourceware.org, I'd like people to have
a look at this:


Recently I created a tracker bug for Frysk hardware data watchpoints.
This can be found here:

http://sourceware.org/bugzilla/show_bug.cgi?id=1625

This tracks the tasks detailed below. They have not yet been implemented:

http://sourceware.org/bugzilla/show_bug.cgi?id=5557
http://sourceware.org/bugzilla/show_bug.cgi?id=5558
http://sourceware.org/bugzilla/show_bug.cgi?id=5559
http://sourceware.org/bugzilla/show_bug.cgi?id=5560
http://sourceware.org/bugzilla/show_bug.cgi?id=5561
http://sourceware.org/bugzilla/show_bug.cgi?id=5562
http://sourceware.org/bugzilla/show_bug.cgi?id=5563
http://sourceware.org/bugzilla/show_bug.cgi?id=5564
http://sourceware.org/bugzilla/show_bug.cgi?id=5565
http://sourceware.org/bugzilla/show_bug.cgi?id=5566
http://sourceware.org/bugzilla/show_bug.cgi?id=5567
http://sourceware.org/bugzilla/show_bug.cgi?id=5568
http://sourceware.org/bugzilla/show_bug.cgi?id=5569
http://sourceware.org/bugzilla/show_bug.cgi?id=5570
http://sourceware.org/bugzilla/show_bug.cgi?id=5571

I'll explain a little bit about the rational for each tracking issue
below with the hope that if I have missed something,  under or over
simplified or generally have holes in the tasks people may comment.

Supported ISAs .

Unless I am mistaken, PPC32 and 64 do not have any hardware watchpoints
(exposed via the kernel) at this point in time. Jan has tracked the
issue more diligently than I:

http://ozlabs.org/pipermail/linuxppc-dev/2007-November/046951.html

and in any case:

https://bugzilla.redhat.com/show_bug.cgi?id=253117

So pending resolution here, the ISAs I talk about are IA32/X86_64. These
are supported by Frysk. I have attempted to keep an eye out for "future"
ISAs and their own peculiar hardware watchpoint implementation where
possible. Tests for currently implemented ISAs that do not support
watchpoints will be marked as unsupported

Basic Testcases.

The first three tracked issues deal with fundamental test-cases. These
are low-level test cases. This means that they are not sourced from a
higher level client like fhpd, or derived from
source-code->symbol->expression->address/range. While I have tracked
these test cases as single tasks, I expect each tracked issue to spawn
several test scenarios. (i.e. watch a byte on a map boundary).

Testcase: watch 1 byte address
(http://sourceware.org/bugzilla/show_bug.cgi?id=5557)

This is the basic low-level test-case. Watch a one byte address. No
range. This facilitates a basic, sanity check. Are watchpoints working
in their most basic implementation? This test will run on the ISAs:
IA32, X8664,  and 32on64. This is the case for all test cases.

Test case: watch address and range
(http://sourceware.org/bugzilla/show_bug.cgi?id=5558)

This is a basic low-level test-case that builds on 5557. Again this is
not concerned with the translation of symbols into
address/range/expressions. It watches a physical address with a range.
Again this task will comprise of several  tests (i.e. testing address
within a map, with a range that stretches outside it, and so on).

Test case: failure to watch address
(http://sourceware.org/bugzilla/show_bug.cgi?id=5559)

This test will build on 5557. It tests the failure conditions of
attempting to watch an address that is:

- not within a virtual address range as specified in maps;
- marked at ---p, or other no watchable map states.

This test will analyze how the watch-point manager will cope with
failure conditions on setting a watchpoint.

The following tracked tasks highlight "code-addition" tasks in Frysk for
watchpoints. As a lot of the code for setting registers, dealing with
sigtraps, ptrace and so on is already written, a lot of this code calls
existing mechanisms. It presumes that code works, test cases for that
code pass,  and it will call that code. However, as identified in
several of the tracked tasks, some of the code will have to be
refactored to deal with the added complications that hardware
watchpoints brings along with its functionality.

Task: Add frysk.proc.TaskObserver.Watch interface
(http://sourceware.org/bugzilla/show_bug.cgi?id=5560)

This task adds the api level observers code that allows high level code
to set a thread specific watchpoint.  At the implementation level
watchpoints are thread based hardware resources. The watchpoint manager
will note the address, range of the watchpoint, and what hardware
register it was placed in. The watchpoint manager is detailed further in
the email.

Task: Add frysk.proc.ProcObserver.Watch interface
(http://sourceware.org/bugzilla/show_bug.cgi?id=5561)

This task adds the api level observers code to allow high level code to
set a process-wide watchpoint.  At the implementation level watchpoints
are thread based hardware resources, but for convenience these
interfaces will allow the application of watchpoints on a per process
level (pending adequate hardware resources on a per task basis).
Similar to 5560, the watchpoint manager is notified of the address,
range of each task in the process.

If the application of a process-wide watchpoint fails on one task,
should the process watchpoint application fail completely?

Task: Write Debug Status Register Builder
(http://sourceware.org/bugzilla/show_bug.cgi?id=5562)

The Debug status register (I'm using IA32/x8664 vernacular here, but
making the broad assumption that each current and future ISA has a debug
control register.) controls the registration of watchpoints, and
subsequent status of a debug status register on a ptrace sigtrap. One
can assume this register to be ISA specific, and the manipulation should
be abstracted away to the ISA specific sections of the Frysk framework.

The top-side of the builder pattern defines its parameters as: byte,
range, register (from watchpoint manager) for setting, and on the read
side, what event and which register the event was registered. The bottom
end (which is an interface) is built in the ISA specific code. This
bottom end translates this to the requisite pattern (per ISA) that the
debug register should written, read, and cleared.

This task is split into three sections:

- Top end of the builder pattern, which constitutes the API (set/read/write)
- IA32 Bottom end implementation
- X8664 Bottom end implementation.

{Other ISAs to follow, contributed by ISA specific authors}.

Task: Write Debug Register Set/Read Proxy.
((http://sourceware.org/bugzilla/show_bug.cgi?id=5563)

Building on above, this interface allows watchpoints to be set in a
non-ISA specific manner. The current allocation of watchpoints on a per
thread basis is stored in the Watchpoint manager. How to write
address/range to each individual debug register is sourced through a
common task based API. The bottom end of this pattern sits in each
specific ISA that take the api (i.e. watch address 0xf00, range 0xf, new
available watchpoint) and translate it to the actual register write.
(Similarly for read).

Task: Sigtrap handling in Frysk needs to be refactored
(http://sourceware.org/bugzilla/show_bug.cgi?id=5564)

Frysk handles sigtraps generated by ptrace in a very linear method.
There is also a presumption that one sigtrap == one event from ptrace.
Currently it forms a simple daisy-chain approach to sigtraps. In a
linear method, each consumer of a sigtrap  gets a "tap" to ask if this
is their sigtrap event. If not, it passes on the event (sigtrap), and it
gets passed down the chain. For example,  single step will source the
relevant register to determine whether a single step occurred when it
receives a sigtrap event. If a single step had just occurred, then it
"consumes" the sigtrap and the sequence ends. However, with the
introduction of watchpoints, it seems to occur that one sigtrap can
signify multiple events. (i.e. single step through a watched address).
With the code as it stands right now, the single step would receive the
notification, consume it, but the watchpoint manager, further down the
line would not receive any notification.  As an aside, further testing
of this assumption is needed with peers; I do not see multiple sigtraps
received for one instruction step/watchpoint trigger, but that does not
mean that multiple events were not generated. However the overriding
fact here is POSIX does not allow nesting of signals.

Similar to LinuxWaitBuilder that already exists in the Frysk codebase, a
"SigTrap" Factory is required to synthesize events and pass each cloned
copy to each registered listener, and allow them to act independently
and determine if the sigtrap even is for them.

Task: Task and Proc State need new watchpoint state transitions.
(http://sourceware.org/bugzilla/show_bug.cgi?id=5565)

Similar to all other task state transitions that a task can undertake,
there are several watchpoints relevant states that need to be included
in both state machines. This model facilities the underlying mechanics
of what the tasks/proc  transitions from, to, and the means to complete
that transition.

Task:Write watchpoint manager.
(http://sourceware.org/bugzilla/show_bug.cgi?id=5567)

The heart of all these watchpoint operations in the watchpoint manager.
This is a central coordinating singleton that records each watchpoint
register used, address, range on a per task basis. It abstracts the
management of watchpoints from the hardware to software (in Frysk's
case) and builds upon the Debug Status Register Builder detailed above
(5562) and the set/read/write proxies (5563).

A deeper more complex issue beyond the straight management of
watchpoints, is optimization of the hardware resources it manages. On
applying  a watchpoint it should scans the address range of existing
watchpoints and determines whether an actual hardware watchpoint needs
to be used, or whether existing watchpoints in the registers are
sufficient to cover a given case. Because hardware watchpoints are so
very scarce,  deletion of a watchpoint also has considerable effect.
Especially in the former case when a new watchpoint was "optimized" to
rely on an existing watchpoint covering similar address and range, and
that underlying watchpoint was deleted.

Risk:

There is considerable work and test cases to build here. I'll add
further bz's to the tracker as the challenges become clearer.  Things
that come to mind immediately:

- Should this manager register a Task terminated observer to all tasks
that register a watchpoint observer? If not, the task based metadata in
the manager will live on forever.

- How should it handle maps being unmapped and free those watchpoint
resources, if needed?

Task: Add fhpd command watch.
(http://sourceware.org/bugzilla/show_bug.cgi?id=5568)

The HPD specification pages 137-141 define the high level watch command.
In summary:

"watch - Define an unconditional watchpoint
            watch variable [-length byte-count] [-stop stop-set]

variable
   A symbol name corresponding to a program scalar or aggregate
identifier, an element of an
   aggregate, or a de-referenced pointer.

byte-count
   The number of bytes to watch. The default is the length of the named
object, as determined
   by its data type.

stop-set
   The processes that should be suspended once the watchpoint has
triggered, expressed using
   standard process set notation. Defaults to the process where the
watchpoint triggered.

[Not available for core-file debugging]"

In fhpd this is WatchCommand.java. The watch command works on the
current defined set that the user has defined in fhpd.

Task: Write high level (fhpd) test case, watch a variable.
(http://sourceware.org/bugzilla/show_bug.cgi?id=5568)

High level tests. Is based on the 5569 task above. This is a high level
testcase. The fhpd specification defines a set of examples that also
make good test cases:

watch *p

Watches the address stored in the pointer p at the time the watchpoint
is defined for the tasks in the defined set. Only tasks in the defined
set will be stopped. Note that the watchpoint location does not change
when  the value of p changes.

watch arrayx

Watches all elements in the array arrayx for  accesses by tasks defined
in the current set. If arrayx is local to a procedure, the watchpoint is
re-installed automatically every time the procedure is entered.

Task: Add UI hardware watch command to source window.
(http://sourceware.org/bugzilla/show_bug.cgi?id=5570)

Similar to 5568 this adds functionality to the source window. As the
source window does not have the concept of sets, it works on a task and
a process level.

Task: Write dogtail test case to test for correct watch addition
(http://sourceware.org/bugzilla/show_bug.cgi?id=5571)

Similar to 5569 will test for similar watch conditions.