Re: [PATCH 3/3] manual: Add new header and standards annotations.

[Rical Jasan <ricaljasan at pacific dot net>]

On 11/25/2016 06:53 AM, Joseph Myers wrote:
> On Fri, 25 Nov 2016, Rical Jasan wrote:
> 
>> creature.texi could just be updated, but I don't think it needs to link
>> anywhere, otherwise it would be referencing everything that used it,
>> wouldn't it?  I may be misunderstanding what you meant by "<summary>"
>> above.  Generally, I would expect anything talking about feature test
>> macros to link back there, for detailed information.
> 
> What I mean is e.g. in creature.texi:
> 
> Functions from this standard are listed as POSIX.1-2008 in @ref{Summary of 
> Library Facilities}.
> 
> Or:
> 
> Functions from this standard are listed as POSIX.1-2008 in @ref{Summary of
> Library Facilities}, and in the Standards information in the individual 
> function description.
> 
> (Or maybe use quotes, ``POSIX.1-2008'', in this text.)
> 
> Where the function description would then have:
> 
> Preliminary: | MT-Safe | AS-Safe | AC-Safe | *Note POSIX Safety 
> Concepts::.
> 
> Standards: POSIX.1-2008 (unistd.h). *Note Feature Test Macros::.

I like this.  This is one of the avenues I was going to suggest, though
I wasn't sure how amenable everybody would be to more boilerplate in
*every single* description.

Don't forget that standards annotations aren't only for functions.
Variables, macros, and types, even down to structs and their members can
all have standards annotations, so they're going to show up all over the
place.

I think the only place I found struct members worth annotating was in
glob_t, by the way, since there are GNU extensions, but there is a case
for it; generally I only annotated structs and not their members.  Case
in point, though, there are exceptions to everything.  Even after this
patchset, there are still a few things deserving annotation that aren't
(but they aren't detectable as such, so they're omitted).  Dealing with
those cases requires either significant reworking of the text or
additional macros (I used @vitem and @titem myself, which were very
useful, but I didn't want to introduce new macros in this patchset, and
there was only one case for each anyway).

>> One question I have is how are standards annotations embedded in the
>> manual going to look?  Currently, where I showed the FTMs in the
>> Summary, the standard @comment is used verbatim.
>>
>> The evolution I see the annotations taking is moving from @comments to
>> something like @headers{} and @standards{}.  If we use "POSIX.1-2008",
> 
> I'd imagine a macro taking both header and standard arguments, so multiple 
> instances of it can be used in the case where there are different headers, 
> with different standards associated.

That makes sense.  Your response to my question about header context
below helps here too.  I had been so focused on the functions, types,
etc., that I missed the obvious fact headers themselves are parts of the
standards too.

> I'd also imagine that the macro goes inside the body of the definition of 
> the relevant functions etc., so that only one instance is needed where you 
> have
> 
> @deffoo fun
> @deffoox fun2
> @deffoox fun3
> @standards{POSIX.1-2008, unistd.h}

If we're going to render it in the manual's text, as part of foo's
description, that's the only way.

> for all functions in the same standard / header.  And a macro variant that 
> also takes the name / names of functions, for cases where there is a group 
> documented together, some in different standards from others (e.g.
> 
> @deftypefun double log (double @var{x})
> @deftypefunx float logf (float @var{x})
> @deftypefunx {long double} logl (long double @var{x})
> @fstandards{log, C90, math.h}
> @fstandards{logf logl, C99, math.h}

What about @standardsx??  It's like, convention!  ;)

> which might expand to:
> 
> Standards (log): C90 (math.h). *Note Feature Test Macros::.
> Standards (logf logl): C99 (math.h). *Note Feature Test Macros::.
> 
> This is illustrative of the sort of thing I'd hope to see in the manual, 
> not a precise recipe for what the macros should look like.

I think it looks pretty good, honestly.  I had played with the idea a
bit, but wasn't combining the headers and standards, which I think makes
this work.  I'll probably use this approach when I start on the
conversion and see what crazy sorts of special cases crop up.  Like...

>> but here is the conditional in fnmatch.h:
>>
>> #if !defined _POSIX_C_SOURCE || _POSIX_C_SOURCE < 2 || defined _GNU_SOURCE
> 
> This conditional is fundamentally broken in glibc terms.  It would have 

I'm glad this should have given me pause, then, because it did.

<...>

> Any such fundamentally broken conditionals should not be used as a basis 
> for designing documentation infrastructure.  Rather, the header should be 
> fixed so that in glibc it includes <features.h> and has glibc-appropriate 
> conditionals.  That may require coordination with gnulib.

<...>

>> I had a question of header context I think is related to this.
>> Sometimes a function is annotated as being per a given standard, but
>> doesn't appear conditional on anything in the header.  I would have
>> assumed MISC or DEFAULT were implied, but sometimes things check
>> explicitly for MISC.  If MISC is still something to test for, what do we
>> assume is defined when there are no tests within a header?
> 
> Standard names are strictly more expressive than feature test macros 
> because they cover the case where a function was present in the first 
> standard that defined a given header.
> 
> For example, unistd.h comes from POSIX.1-1990.  So any function that is 
> declared in unistd.h unconditionally should be listed as POSIX.1-1990.  

I get this, but..

> That is completely separate from MISC, which is for *nonstandard* 
> functions that are declared in the absence of feature test macros (or for 
> XSI functions declared in the absence of feature test macros, since the 
> default standards don't include XSI; MISC tests should never be combined 
> with tests for non-XSI POSIX, as they are redundant in that case).

I don't think I'm completely following this.  Is this a redundant test,
then:

./math/bits/math-finite.h:#if defined __USE_MISC && defined __USE_ISOC99

My practical question is, if MISC is nonstandard, what would we document
in @standards?  Just MISC?  I see it replaces deprecated feature test
macros _BSD_SOURCE and _SVID_SOURCE (or rather _DEFAULT_SOURCE does,
which is the only thing that sets __USE_MISC).  Would we not want to
document that something came from BSD, for example?  (Also see comment
below about handling deprecation.)

> As another example: error and signal numbers are defined unconditionally 
> in the headers, but are mostly not in C90.  You can document the standards 
> individual numbers are defined in, which gives portability information 
> beyond what "no feature test macro needed with glibc" would give.
> 
>> So if we were to document FTMs, would these look like "-std=c99",
>> instead of "_ISOC99_SOURCE"?  And how to document "C95", which doesn't
>> have an "_ISOC95_SOURCE" or "-std=c95" (in my gcc(1))?
> 
> I don't think you should document feature test macros (certainly not 
> without a very careful review that the conditionals actually make sense).  
> As noted, these standards don't have feature test macros.  I don't think 
> listing compiler options for individual functions is appropriate either.  
> Most people reading the manual don't want to use feature test macros at 
> all for functions declared by default, and if they want to use feature 
> test macros for other functions _GNU_SOURCE will probably suffice.  
> Documenting the standards provides sufficient information for people 
> concerned about portability or about when the functions are declared.

That could make things easier, I suppose.  I was using the conditionals
mainly as hints, for later work in more fully vetting the standards,
admittedly banking on the fact there seemed to be little to no
established convention for what was already present in the standard
@comments.

> You use -std=iso9899:199409 for C95 (it was published 1995-04-01; 199409 
> is the version from __STDC_VERSION__).
> 
> Note that each POSIX version is based on some ISO C version.  So for 
> POSIX.1-2008 appropriate options are actually -std=c99 
> -D_POSIX_C_SOURCE=200809L.

OK, thank you.

>> Even if we were to accept textual standards, the combining expressions
>> are wont to get ugly, and we would have to parse the logic into a nicely
>> formatted sentence, which is the main reason I considered @ftm{} in
>> addition to @standards{}.
> 
> I wouldn't anticipate sentences, just Standards: whatever as above.

What do you think about handling the common, "This function is a GNU
extension.", throughout the descriptions?  Would you remove those in
lieu of "Standards: GNU (...)"?  It's one of the few consistent and
widespread phrases throughout the manual.

Also, what about deprecation?  I saw the exceptional case of gets
mentioned recently.  There are quite a few BSD things the manual warns
about being deprecated.  Maybe anything that can't be coloured properly
in @standards just deserves a nice paragraph somewhere in the description.


Thank you for all the help!  It is greatly appreciated.

Rical