Re: [MTASCsft PATCH WIP5 01/33] Multi Thread, Async Signal and Async Cancel safety documentation: intro

[Torvald Riegel <triegel at redhat dot com>]

On Tue, 2013-11-26 at 18:44 -0200, Alexandre Oliva wrote:
> On Nov 25, 2013, Torvald Riegel <triegel@redhat.com> wrote:
> 
> > On Sat, 2013-11-23 at 11:15 -0200, Alexandre Oliva wrote:
> >> On Nov 20, 2013, Torvald Riegel <triegel@redhat.com> wrote:
> >> 
> >> > What about linking statically and using LTO?
> >> 
> >> I've added the following to the definition of MT-Safe.  Does it address
> >> satisfactorily the issues you brought up in this regard?
> >> 
> >> A function being MT-Safe does not imply any form of synchronization,
> 
> > I guess I know what you want to point out, but I don't think that it's
> > clear enough.  Also, this depends on how MT-Safe is defined,
> 
> But, but...  The above is proposd as part of our definition of MT-Safe!

Right, sorry.  But I still think that this isn't clear enough.

> > including whether there it is sequentially consistent or not.
> 
> This is not even part of the upstream (posix) definition of MT-Safe
> AFAICT.

Well, it's not stated that way, but there's just no sufficiently
complete definition of it in POSIX.  Sequential consistency could be a
reasonable intent (e.g., both C and C++ think that this should be the
default), but it could also be something weaker.  We just don't know,
which is why at least we need to define what we mean.

The fact that POSIX at least specifies mutex lock/unlock to have a
global total order (even though no implementation would guarantee that,
it's too strong) suggests to me that they would want the same for
MT-Safe too.  But that's guessing, of course.

> > saying that there might not be any
> > synchronization (it's also unclear what you consider to be
> > synchronization) is just confusing, I believe.
> 
> Upstream (posix) defines what (memory) synchronization is, and how it
> can be accomplished.  I'm trying to say that there's no guarantee that
> an MT-Safe function will performany such form of memory synchronization.
> After all, many MT-Safe functions don't even operate on shared state
> that might benefit from synchronization, and a few of those that do
> operate no shared state can do without synchronization because they work
> fine on âstaleâ data.

The latter will at least use the equivalent of atomic accesses with
relaxed memory order.  Which can be considered synchronization; even if
they might not establish inter-thread happens-before orders, they will
certainly *respect* any externally established happens-before order as
well as sequenced-before and the coherency rules.

Does the above fit the mental model you had in mind for MT-Safe while
doing the review?  Should I work on something more detailed?  It would
be based on the C11 model's notation, given that that is more useful
than the fuzzy POSIX definition.

Thus, we could base the MT-Safe definition on relaxed-memory-order
atomic accesses.  If some MT-Safe function does not operate virtually
atomically on the state that it accesses (IOW, it can "step out of" the
defined behavior for a sequential execution), we need to document this.
One way would be to split it into logical parts, each of them being
MT-Safe, and with or without additional ordering guarantees between
them.

> The quotes around stale are there to indicate
> that it's not really stale when there's no synchronization, because
> without synchronization, there's no happens-before relationship.

It doesn't work that way.  There is a happens-before relation even
without synchronization (e.g., created by sequenced-before), but there
may not be any additional inter-thread-happens-before.  Also, if you
modify memory atomically, then the modifications of a single memory
location form a total order too in the C11 model (and I believe in most
hardware models as well).

> >> nor that any combination of calls of MT-Safe functions would be
> >> regarded as MT-Safe.  For example, having a thread call two MT-Safe
> >> functions one right after the other does not guarantee behavior
> >> equivalent to atomic execution of a combination of both functions,
> >> since concurrent calls in other threads may cause behavior to diverge.
> 
> > The example is true and makes sense, but the second part of the first
> > sentence isn't clear enough (ie, the words prior to the example); some
> > combination could be safe depending on how MT-Safety is defined...
> 
> I didn't write âno combination of callsâ, I wrote âany ...â.  This is
> not supposed to rule out the possibility that some combinations could be
> safe.  Isn't that what the construct I used mean?  Do I need to add an
> âarbitraryâ, or make it âall combinationsâ to make it clear?

What you wrote is technically correct; I just think that for clarity and
to give guidance to users, we might want to point out which common
combinations would be safe.  But that's not urgent, I suppose.

> >> In some cases, even expanding MT-Safe functions inline could cause
> >> combinations to become MT-Unsafe due to reordering that this could
> >> enable; this is never the case of @glibcadj{} functions defined in
> >> user-visible headers, because these are meant to be inlined, but it
> >> may be for functions that are exported by @glibcadj{} shared
> >> libraries; whole-program optimizations that might inline functions
> >> across library interfaces might expose this sort of problem, so
> >> performing inlining across the @glibcadj{} interface is not
> >> recommended, nor is the documented MT-Safety status guaranteed under
> >> such arrangements.
> 
> > I think the critical point here is that "documented MT-Safety status" is
> > not "guaranteed under such arrangements".
> 
> I think I understand why you think so, but I differ; to me, that's just
> one of the manu consequences of the previous statements.

And I don't disagree with that.  But to me, from a presentation
perspective, it's better to first say what the thing is that affects the
user, and then give the details why that's the case.  Users don't need
to understand why we have a problem, as long as they remember what not
to do to avoid the problem.

Which other consequences would these causes have for the user? (With the
assumption that they can't use LTO if they want to rely on the MT-Safe
annotations.)