Re: [PATCH 3/3] manual: Add new header and standards annotations.

[Rical Jasan <ricaljasan at pacific dot net>]

On 11/24/2016 05:37 AM, Joseph Myers wrote:
> On Thu, 24 Nov 2016, Rical Jasan wrote:
> 
>>> XOPEN2K is generically an unhelpful name.  In the headers it actually 
>>> means POSIX.1-2001; the X/Open version is __USE_XOPEN2KXSI.  Likewise 
>>> __USE_XOPEN2K8 means POSIX.1-2008 and the X/Open version is 
>>> __USE_XOPEN2K8XSI.  So anywhere you say XOPEN2K because the headers use 
>>> __USE_XOPEN2K, say POSIX.1-2001 in the manual instead; likewise 
>>> POSIX.1-2008 for XOPEN2K8.
>>
>> I'm glad you mention this.  Have you seen the related discussion on
>> libc-help? [1]  I'm interested in your thoughts on converting standards
>> to feature test macros (or vice versa).
> 
> I'm not on libc-help.  libc-help is for user questions.  It's not an 
> appropriate place for development discussions and is not relevant to 
> establishing consensus on development directions.

Duly noted.

>> Also, if we want long-form standards in the manual, where would we put
>> the feature test macros?  It seems more appropriate to document the
>> feature test macros in the Summary, and the current framework uses the
>> standard @comment verbatim there.  (There is also the issue that, e.g.,
>> XOPEN2K or _USE_XOPEN2K aren't the actual feature test macros one would
>> even use...)
> 
> My assumption is that creature.texi (which is very out of date regarding 
> what's enabled by default, values of _POSIX_C_SOURCE, etc.) would, where 
> it describes support for a standard and any associated feature test 
> macros, say something like: "Functions enabled with this feature test 
> macro / from this standard are listed as POSIX.1-2008 in <summary>.".  
> Where <summary> could be a link to the existing summary.  Or it could say 
> "in the Standards information in the individual function description", if 
> we use macros to generate explicit Standards information in each function 
> description alongside the Safety information.  Or both.
> 
> That is, creature.texi would explain the notation just like intro.texi has 
> an explanation of the safety annotations.  The summary currently says "the 
> standard or other source from which each facility is derived", not "the 
> feature test macro".

I'd like to see the Summary (of Library Facilities) have the header and
feature test macros, since it's something of a quick reference.  For
functions, you get to see the API, and for everything, which header it
comes from and what FTM you need (and what it should be defined to is
also a worthy goal, I think).  So we would have entries like:

int foo (struct bar *bp, size_t len)
  fubar.h (_POSIX_C_SOURCE >= 200809L): <reference/link>

or:

char *program_invocation_short_name
  errno.h (_GNU_SOURCE): <reference/link>

creature.texi could just be updated, but I don't think it needs to link
anywhere, otherwise it would be referencing everything that used it,
wouldn't it?  I may be misunderstanding what you meant by "<summary>"
above.  Generally, I would expect anything talking about feature test
macros to link back there, for detailed information.

One question I have is how are standards annotations embedded in the
manual going to look?  Currently, where I showed the FTMs in the
Summary, the standard @comment is used verbatim.

The evolution I see the annotations taking is moving from @comments to
something like @headers{} and @standards{}.  If we use "POSIX.1-2008",
we could assume "_POSIX_C_SOURCE >= 200809L", but expressions become
difficult.  I've considered having both @ftms{} and @standards{}, but
I'm also hoping for a solution that can benefit the manpages as well,
which would involve some kind of translation between the two (standards
and FTMs).

Consider:

@comment fnmatch.h
@comment GNU
@item FNM_CASEFOLD

which results in:

FNM_CASEFOLD
  fnmatch.h (GNU): <reference/link>

This could be:

@headers{fnmatch.h}
@standards{GNU}
@item FNM_CASEFOLD

but here is the conditional in fnmatch.h:

#if !defined _POSIX_C_SOURCE || _POSIX_C_SOURCE < 2 || defined _GNU_SOURCE

I'm not sure we can translate "GNU" to that, which means accurate
documentation of FTMs from standards names isn't possible.
Acknowledging the fact there is no way to automatically keep this
information up-to-date, we could document both:

@headers{fnmatch.h}
@standards{GNU}
@ftms{!_POSIX_C_SOURCE || _POSIX_C_SOURCE < 2 || _GNU_SOURCE}
@item FNM_CASEFOLD

Then there is also the question of whether we would like to move any of
those annotations into the textual description below.

> There's an open question of whether we wish to continue to document BSD / 
> SVID / ... sources not corresponding to feature test macros, or just make 
> the annotations say MISC or DEFAULT and leave information about BSD etc. 
> origins to free-text descriptions.

I had a question of header context I think is related to this.
Sometimes a function is annotated as being per a given standard, but
doesn't appear conditional on anything in the header.  I would have
assumed MISC or DEFAULT were implied, but sometimes things check
explicitly for MISC.  If MISC is still something to test for, what do we
assume is defined when there are no tests within a header?

> As for names corresponding to standards / feature test macros, I suggest 
> one possibility:
> 
> C90 (everything is a superset of this apart from gets obsoletion)
> C95
> C99
> C11
>   (note that these four are normally selected with -std, not with feature 
>    test macros, though glibc has _ISOC99_SOURCE and _ISOC11_SOURCE)

So if we were to document FTMs, would these look like "-std=c99",
instead of "_ISOC99_SOURCE"?  And how to document "C95", which doesn't
have an "_ISOC95_SOURCE" or "-std=c95" (in my gcc(1))?

> TR 27431-2:2010
> TS 18661-1:2014
> TS 18661-4:2015
> POSIX.1 (= 1990 edition)
> POSIX.2
> POSIX.1-1993
> POSIX.1-1995
> POSIX.1-2001
> XSI POSIX.1-2001
> POSIX.1-2008
> XSI POSIX.1-2008
> DEFAULT
> GNU
> XOPEN (= __USE_XOPEN; listed as XPG3 in conform/ tests; corresponds to 
>        functions in C435 that are not UX-shaded)
> XPG4 (= __USE_XOPEN_EXTENDED; corresponds to everything in C435)
> UNIX98
> LFS (= __USE_LARGEFILE64, i.e. *64 functions)
> 
> (which doesn't cover _LARGEFILE_SOURCE / _ATFILE_SOURCE / _REENTRANT, but 
> should be enough anyway to provide a sufficient indication of where 
> functions come from, at least if you can handle "standard 1 || standard 
> 2", "standard 1 && ! standard 2", or similar).

Even if we were to accept textual standards, the combining expressions
are wont to get ugly, and we would have to parse the logic into a nicely
formatted sentence, which is the main reason I considered @ftm{} in
addition to @standards{}.

I would like to know how everybody feels about adding automated snippets
to the documentation.  The header and standards @comments aren't
currently rendered in the manual in any way, aside from inclusion in the
Summary of Library Facilities.  Would we like to see a standardized
layout, by mimicking the @safety{} framework?

Rical