Automated Doc Extraction: A Premature Retrospective

[Maciej Stachowiak <mstachow@mit.edu>]

So Greg and I got the automated doc extraction system working and have
documented all of the approximately 200 C-level primitives in Scwm, as
well as adding documentation for hooks and special variables, and
higher-level concept documentation. The HTML (generated from DocBook)
that we have so far can be seen at:

http://www.cs.washington.edu/research/constraints/cassowary/scwm-doc/


Before I describe what I think went well with this process, let me
point out some shortcomings that we have yet to address:

* The concept and hook/variable documentation is not yet incorporated
into the manual

* Documentation is not yet interactively available from within the
interpreter. Sam Steingold is working on writing a Scheme-level
interface which will extract the docstrings from a plaintext docstring
file, and which will also integrate with the `scwm.el' emacs
interface.

* The manual has no proper organization of any kind yet. You only get
an alphabetical list of all the procedures, or a list by file where
they are defined. It should be possible to impose a better organization

* The manual needs some improvements to the formatting, and the
addition of hyperlinks for cross-references.

* We need to figure out a way to generate texinfo (or perhaps info
directly - texinfo is less necessary since DocBook can generate nice
printed output). A general texinfo or info back end to DocBook may be
the best solution here (perhaps it is time for me to learn DSSSL).


That being said, I would like to point out the ways in which the
doc-extraction system was a great success. First, I was able to
document nearly all of the scwm interface (Greg had already done one
file as an example) in just 3 days, working about 4 hours a day. I do
have fairly intimate knowledge of the code, but I believe the doc
extraction system helped in the following psychological and material
ways:

* I was more motivated to document parts of the code at a time because
I had a real sense of incremental improvement. Formerly, starting at a
huge manual that needed to be organized and constantly checked for
which portions were incomplete was a great psychological barrier; it
was hard to make myself check if something was documented whenever I
changed it, and if so,

* In a related development, I did not have to worry about how to
organize the manual all at once. I was able to write point
documentation for everything, and be confident that it would be easy
to incorporate this detail work into whatever final hierarchy emerged.

* I was less worried about documenting facilities I knew would change
greatly. Since the documentation was right next to the code, I was
more confident that the manual would not bitrot, because when those
facilities changed, it would be necessary

* It was actually concretely easier to write documentation when
looking directly at the code I was documenting, and the comments that
were already there.

The doc extractor also proved to have benefits beyond just the
documentation. Greg added features to it to warn if the C and Scheme
names of a function did not match according to the usual conventions,
or if the C functions formals did not match the number of different
types of arguments as specified in SCM_PROC. This helped locate a
number of latent bugs.

I also think the extraction system will be helpful in the future as a
reminder to people to document their changes and additions, and will
potentially serve as a stick for us to get outside contributors to
document their changes.


Given our success so far, I think our doc extraction system should be
seriously looked at for other Guile apps and extensions, and perhaps
even for the Guile core, once we address it's current problems and
limitations, including those listed above.


In fact, Guile's lack of good documentation has been a serious
impediment to my attempts to evangelize Guile for all sorts of uses,
so much so that I would be highly motivated to personally document a
lot of the Guile core using this system, once it is ready, even if
this documentation were only to be used for docstrings (although I
think a docstring listing is a nice pice of documentation to have even
if you have a separate, handwritten reference manual).

Guile has about 600 primitives vs. scwm's 200, but many of these
already have some form of documentation, either as comments, in the
existing texinfo docs, or in R[45]RS.

So I think 1-2 man-weeks of labor could possibly lead to a full set of
detail docs.

Any feedback on this subject, or on the docs we have so far, is
welcome.

 - Maciej Stachowiak