Summary: | Handle [a-z] and [A-Z] in consistent portable fashion regardless of locale. | ||
---|---|---|---|
Product: | glibc | Reporter: | Florian Weimer <fweimer> |
Component: | regex | Assignee: | Carlos O'Donell <carlos> |
Status: | NEW --- | ||
Severity: | normal | CC: | anssi.hannula, aurelien, bugdal, carlos, drepper.fsp, eblake, jakob_sellgren, johannes, maiku.fabian, mark4th, petersen, psmith, rjones, zlang |
Priority: | P2 | Flags: | fweimer:
security-
|
Version: | 2.28 | ||
Target Milestone: | --- | ||
See Also: |
https://sourceware.org/bugzilla/show_bug.cgi?id=23420 https://bugzilla.redhat.com/show_bug.cgi?id=1607286 https://bugzilla.redhat.com/show_bug.cgi?id=1631472 https://sourceware.org/bugzilla/show_bug.cgi?id=12045 https://sourceware.org/bugzilla/show_bug.cgi?id=12051 |
||
Host: | Target: | ||
Build: | Last reconfirmed: | ||
Attachments: | reorder.c |
Description
Florian Weimer
2018-07-10 15:59:25 UTC
I think we should go back to 0123456789 if possible because I expect that this is used a lot for data validation. [[:digit:]] still produces 0-9, and [[:xdigit:]] gives the expected result as well, so perhaps we should treat this as invalid. It's not a POSIX conformance issue because ranges are undefined in non-C locales. (In reply to Florian Weimer from comment #2) > [[:digit:]] still produces 0-9, and [[:xdigit:]] gives the expected result > as well, so perhaps we should treat this as invalid. > > It's not a POSIX conformance issue because ranges are undefined in non-C > locales. That's exactly what I was going to say. Please see commit ac3a3b4b0d561d776b60317d6a926050c8541655 where we discuss exactly this issue. posix/tst-fnmatch.input: +# +# See: +# +# http://pubs.opengroup.org/onlinepubs/7908799/xbd/re.html +# +# > A range expression represents the set of collating elements that fall +# > between two elements in the current collation sequence, +# > inclusively. It is expressed as the starting point and the ending +# > point separated by a hyphen (-). +# > +# > Range expressions must not be used in portable applications because +# > their behaviour is dependent on the collating sequence. Ranges will be +# > treated according to the current collating sequence, and include such +# > characters that fall within the range based on that collating +# > sequence, regardless of character values. This, however, means that +# > the interpretation will differ depending on collating sequence. If, +# > for instance, one collating sequence defines ä as a variant of a, +# > while another defines it as a letter following z, then the expression +# > [ä-z] is valid in the first language and invalid in the second. +# +# Therefore, using [a-z] does not make much sense except in the C/POSIX locale. +# The new iso14651_t1_common lists upper case and lower case Latin characters +# in a different order than the old one which causes surprising results +# for example in the de_DE locale: [a-z] now includes A because A comes +# after a in iso14651_t1_common but does not include Z because that comes +# after z in iso14651_t1_common. Thus such innocuous regexp's as [0-9a-f] for hex are going to potentially accept a whole range of characters that are entirely based on the locale and may be generally unsafe for processing. I fully expect there may be security issues in this area as more applications adopt the broader set of language data available in Unicode upstream. For now I think this is RESOLVED/WONTFIX until we have more real-world data on these problems. As you say we can and should strongly recommend digit and xdigit. I still think that it's very hard to make the case that the fact that [0-9] matches 8 but not 9 is the right behavior. It also seems that grep has a workaround for this and interprets the pattern differently. On the other hand, in the old tr_TR.ISO-8859-9 locale, [0-9] did match 0123456789¼½¾, and not just 0123456789. (de_DE.ISO-8859-1 had 0123456789 though, as another single-byte locale.) This issue breaks makefiles which assume that the glob pattern [a-z]* does not match the file name Makefile. See bug 23420. (In reply to Florian Weimer from comment #6) > This issue breaks makefiles which assume that the glob pattern [a-z]* does > not match the file name Makefile. See bug 23420. Is the solution to rewrite using [:lower:]? (In reply to Florian Weimer from comment #4) > I still think that it's very hard to make the case that the fact that [0-9] > matches 8 but not 9 is the right behavior. Doesn’t your first comment show that 9 is included in the match? expected: "0123456789" actual: "0123456789²³¹¼ ... and lots of other stuff but 9 was there ... (In reply to Mike FABIAN from comment #8) > (In reply to Florian Weimer from comment #4) > > I still think that it's very hard to make the case that the fact that [0-9] > > matches 8 but not 9 is the right behavior. > > Doesn’t your first comment show that 9 is included in the match? Florian is talking about having included FULLWIDTH DIGIT EIGHT but *not* including FULLWIDTH DIGIT NINE, for en_US.UTF-8. Why doesn't it include FULLWIDTH DIGIT NINE? And the answer is because the regex says to stop at "9", and there are lots more nine's after 9 that don't get included. To capture all the nine's you'd have to stop at 10 (which doesn't exist) or the last known "9" (which is locale dependent). Therefore I think it's still the right behaviour. We really should be using [:lower:]. > expected: "0123456789" > actual: "0123456789²³¹¼ ... and lots of other stuff but 9 was there ... Yes, that's fine, and that includes the 9, and so is OK. My opinion is that this issue of matching [0-9] should be closed since it matches 0-9 properly, but also matches other things. This may or may not be a problem in the future, but it is unlike bug 23420 which can cause real issues in a build system that uses characters in the ASCII range and gets odd sorting. In addition to the new behaviour being consistent with the Unicode collating order, as it reasonably should, you should be able to set LC_COLLATE if you want some other collation to be used. I find it very dubious that the current implementation of ranges is useful for anything at all, exception implementation convenience (as it's what we have today). Two possible improvements come to my mind: (a) If the both ranges are ASCII, match only ASCII characters. (b) Ranges include all characters with the same primary collation weight as the endpoints. It's possible to implement both, with (a) superseding (b). I'm not sure if today, range expressions can match collating elements consisting of multiple characters, in which case the following variant might be less surprising: (b') Ranges include all collating elements with the same primary weight as the endpoints. Both approaches are conforming to POSIX because ranges in other locales are undefined anyway. As far as I can see, available user feedback suggests that (a) is the expected behavior. I think some tools actually implement (a) already because we went through this fifteen years ago or something like that, but I can't find the historic discussion. I think bracket range expressions should just always be codepoint ranges. They can't reliably be used for anything else (since the behavior isn't specified), but anything else severely violates the principle of least surprise. (In reply to Rich Felker from comment #13) > I think bracket range expressions should just always be codepoint ranges. > They can't reliably be used for anything else (since the behavior isn't > specified), but anything else severely violates the principle of least > surprise. Codepoint-based ranges would address my concerns. I was just trying to come up with something that would open a backdoor to get access to collation-order-based ranges in case they are needed (because some people seem to like them). But I can't see a useful application for them, either. (In reply to Florian Weimer from comment #14) > (In reply to Rich Felker from comment #13) > > I think bracket range expressions should just always be codepoint ranges. > > They can't reliably be used for anything else (since the behavior isn't > > specified), but anything else severely violates the principle of least > > surprise. > > Codepoint-based ranges would address my concerns. > > I was just trying to come up with something that would open a backdoor to > get access to collation-order-based ranges in case they are needed (because > some people seem to like them). But I can't see a useful application for > them, either. Any generic changes, particularly to use codepoing-based ranges will break existing sortings in many other languages, including en_US.UTF-8. The existing iso_14651_common has ~300+ symbols in the range and this proposal would exclude it down to 26. For example we have supported ñ in Spanish (second most native speakers on the planet after Chinese) within [a-z], and removing that could possibly break a huge number of regexp's used by Spanish developers with file names that contain ñ. The key issue at hand here is the cognitive dissonance with what [a-z] means and that it includes uppercase characters. Basically we've allowed [a-z] to mean the equivalent of ':lower:' and [A-Z] to mean the equivalent of ':upper:'. Anecdotally the cat is out of the bag in my opinion, this change was already made to locales in the glibc 2.12 (RHEL 6 era), and today we have Arabic, Czech, Estonian, Croatian, Sorbian, Icelandic, Khmer, Lao, Lithuanian, Latvian, Odia (India), Polish, Slovenian, Thai, and Turkish, all with the mixed-case aA-zZ collation. Note that Japanese and Korean explicitly sort the lower and upper case ascii ranges without interleaving and so is excluded from this issue. With the ISO 14651 update (derived from Unicode 9.0) we have this issue for all languages that use ISO 14651 as the basis for their collation, and this includes en_US. My opinion is that if we really want to make a change to preserve backwards compatibility it should be in regex and it should be to treat a-z explicitly as :lower: and A-z explicitly as :upper: and in the case of the existing 15 locales, they will have to adjust all of their regexp's to match upper/lower-case expectations. The nominal notion of case is a far more compelling argument than code-points, or equivalence classes. *** Bug 23420 has been marked as a duplicate of this bug. *** Proposal (c) - Handle the range a-z as an alias for :lower:. - Handle the range A-Z as an alias for :upper:. - Handle the range 0-9 as an alias for :digit:. This would bring compatibility for regex's and still allow all other languages to include lowe-case alphabetic symbols like ñ in Spanish without breaking Spanish developer scripts that depend on that. We would be breaking scripts for the 15 locales that have mixed aA-zZ collation if they expect a-z to include A-Y, but the potential for breakage in all the other languages is worse. (In reply to Carlos O'Donell from comment #17) > Proposal (c) > - Handle the range a-z as an alias for :lower:. > - Handle the range A-Z as an alias for :upper:. > - Handle the range 0-9 as an alias for :digit:. > > This would bring compatibility for regex's and still allow all other > languages to include lowe-case alphabetic symbols like ñ in Spanish without > breaking Spanish developer scripts that depend on that. > > We would be breaking scripts for the 15 locales that have mixed aA-zZ > collation if they expect a-z to include A-Y, but the potential for breakage > in all the other languages is worse. From an implementation perspective I believe we can add this in whenever we call __collseq_table_lookup in regcomp.c, regexec.c, and fnmatch_loop.c, we need to do the substitution if we are looking for one of the above ranges and use a fixed range. What about ranges like [0-9a-f]? (In reply to Carlos O'Donell from comment #15) > (In reply to Florian Weimer from comment #14) > > (In reply to Rich Felker from comment #13) > > > I think bracket range expressions should just always be codepoint ranges. > > > They can't reliably be used for anything else (since the behavior isn't > > > specified), but anything else severely violates the principle of least > > > surprise. > > > > Codepoint-based ranges would address my concerns. > > > > I was just trying to come up with something that would open a backdoor to > > get access to collation-order-based ranges in case they are needed (because > > some people seem to like them). But I can't see a useful application for > > them, either. > > Any generic changes, particularly to use codepoing-based ranges will break > existing sortings in many other languages, including en_US.UTF-8. The > existing iso_14651_common has ~300+ symbols in the range and this proposal > would exclude it down to 26. The point Rich and I are making is that there is no requirement in POSIX to have ranges following collation sorting. Our current implementations do this, but it's not required by POSIX. We can change the code (and not the data). > With the ISO 14651 update (derived from Unicode 9.0) we have this issue for > all languages that use ISO 14651 as the basis for their collation, and this > includes en_US. We aren't proposing changes to the collation rules. > My opinion is that if we really want to make a change to preserve backwards > compatibility it should be in regex and it should be to treat a-z explicitly > as :lower: and A-z explicitly as :upper: and in the case of the existing 15 > locales, they will have to adjust all of their regexp's to match > upper/lower-case expectations. The nominal notion of case is a far more > compelling argument than code-points, or equivalence classes. This still fixes only a subset of the problematic cases. For example, using [0-7] for an octal digit or [0-9a-f] for a lower-case hexadecimal digit would still not work, and [a-zA-Z/.] would not match base64 digits only, either. (In reply to Andreas Schwab from comment #19) > What about ranges like [0-9a-f]? I would ignore it. Since aAbBcCdDeEf are all valid for parsing hex, unless you are specifically excluding uppercase. It gets much more complicated to process this if you have to convert a canonical form of [0-9a-f] into :xdigits:. Today I think actually the simplest thing to do for 2.28 is this: - Deinterlace the ISO 14651 data e.g. aAbBcCdDeEfF...zZ => abcd..zA..Z, using the islower / isupper data. This will restore the original ordering, keep the new characters, deviate slightly from ISO 14651, and allow us time to consider our options. Thoughts? (In reply to Florian Weimer from comment #20) > The point Rich and I are making is that there is no requirement in POSIX to > have ranges following collation sorting. Our current implementations do > this, but it's not required by POSIX. We can change the code (and not the > data). This is not my interpretation. http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html ~~~ 7. In the POSIX locale, a range expression represents the set of collating elements that fall between two elements in the collation sequence, inclusive. ~~~ We would not meet that rule if we used code points? > > With the ISO 14651 update (derived from Unicode 9.0) we have this issue for > > all languages that use ISO 14651 as the basis for their collation, and this > > includes en_US. > > We aren't proposing changes to the collation rules. For 2.28 I'm going to propose we de-interlace a-zA-z to solve the issues we have seen so far since we're not ready to make a decision. This is no worse than what we had before and users that already fixed scripts to portable use :upper: :lower: still work too. The 15 locales that use interleaved aA-zZ will remain as-is and need not change. > > My opinion is that if we really want to make a change to preserve backwards > > compatibility it should be in regex and it should be to treat a-z explicitly > > as :lower: and A-z explicitly as :upper: and in the case of the existing 15 > > locales, they will have to adjust all of their regexp's to match > > upper/lower-case expectations. The nominal notion of case is a far more > > compelling argument than code-points, or equivalence classes. > > This still fixes only a subset of the problematic cases. For example, using > [0-7] for an octal digit or [0-9a-f] for a lower-case hexadecimal digit > would still not work, and [a-zA-Z/.] would not match base64 digits only, > either. These are based on an erroneous understanding of POSIX regular expressions. Either way for 2.28 I'm suggesting we revert the lower/upper interleaving in localedata/locales/iso14651_t1_common for now. Thoughts? (In reply to Carlos O'Donell from comment #22) > (In reply to Florian Weimer from comment #20) > > The point Rich and I are making is that there is no requirement in POSIX to > > have ranges following collation sorting. Our current implementations do > > this, but it's not required by POSIX. We can change the code (and not the > > data). > > This is not my interpretation. > > http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html > > ~~~ > 7. In the POSIX locale, a range expression represents the set of collating > elements that fall between two elements in the collation sequence, inclusive. > ~~~ > > We would not meet that rule if we used code points? Sorry, meant to quote the line under that one: ~~~ 7. In the POSIX locale, a range expression represents the set of collating elements that fall between two elements in the collation sequence, inclusive. In other locales, a range expression has unspecified behavior: strictly conforming applications shall not rely on whether the range expression is valid, or on the set of collating elements matched ~~~ You argue that the "unspecified behaviour" (not undefined), would be changed? (In reply to Carlos O'Donell from comment #22) > (In reply to Florian Weimer from comment #20) > > The point Rich and I are making is that there is no requirement in POSIX to > > have ranges following collation sorting. Our current implementations do > > this, but it's not required by POSIX. We can change the code (and not the > > data). > > This is not my interpretation. > > http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html > > ~~~ > 7. In the POSIX locale, a range expression represents the set of collating > elements that fall between two elements in the collation sequence, inclusive. > ~~~ > > We would not meet that rule if we used code points? For ASCII-based implementations, the order is the same. From “LC_COLLATE Category in the POSIX Locale”: # This is the minimum input for the POSIX locale definition for the # LC_COLLATE category. Characters in this list are in the same order # as in the ASCII codeset. And a cursory glance at the definition suggests that the comment is accurate. > You argue that the "unspecified behaviour" (not undefined), would be changed? Yes, or not be changed, for the en_US locale and many common range expressions. Regarding Carlos' proposal in comment 17, it's not clear to me that users want ñ to be matched by [a-z] in en locales or similar, even though letters not used in English are in class [:lower:]. I'm unsure whether they are included in [a-z] now; it probably depends on LC_COLLATE, which many users explicitly set to C. Changing things so that even with LC_COLLATE=C, you get additional characters in [a-z] due to LC_CTYPE, seems undesirable, and like it would prompt users to set LC_CTYPE=C, breaking all Unicode support. I finally found a summary of the old discussion, regarding the Rational Range Interpretation: https://www.gnu.org/software/gawk/manual/html_node/Ranges-and-Locales.html I have patches to fix this along with one new test case and some test case readjustments. I also have an implementation of [a-z][A-Z] and [0-9] in glibc, and I'll post that patch once 2.29 opens and we can discuss. Created attachment 11145 [details]
reorder.c
The following program takes as input the LATIN script range and reorder it based on rules that allow [a-z] and [A-Z] to work as expected.
glibc implements ISO POSIX-2:1993 requirements and so uses collation element order (CEO) as the basis for range expressions. Therefore we can re-arrange the range elements while still keeping the ISO 14651 sorting.
This means we can meet the range requirements we want and the sorting we want. I don't see any downside to this. Further we can create 3 distinct ranges in iso14651_t1_common wihtout breaking collation and those ranges would only contain a-z, A-Z, and 0-9 and implement what Florian and Rich are suggesting for all locales that use iso14651_t1_common collation.
The 15 locales that don't use iso14651_t1_common will not be changed since that's the most conservative solution.
Posted solution for 2.28: https://www.sourceware.org/ml/libc-alpha/2018-07/msg00620.html (In reply to Florian Weimer from comment #12) > I find it very dubious that the current implementation of ranges is useful > for anything at all, exception implementation convenience (as it's what we > have today). > > Two possible improvements come to my mind: > > (a) If the both ranges are ASCII, match only ASCII characters. > > (b) Ranges include all characters with the same primary collation weight as > the endpoints. > > It's possible to implement both, with (a) superseding (b). I'm not sure if > today, range expressions can match collating elements consisting of multiple > characters, in which case the following variant might be less surprising: > > (b') Ranges include all collating elements with the same primary weight as > the endpoints. > > Both approaches are conforming to POSIX because ranges in other locales are > undefined anyway. As far as I can see, available user feedback suggests > that (a) is the expected behavior. Well, close to (a), at any rate. You're looking for Rational Range Interpretation, which has been picked up by several GNU tools already (awk, coreutils, sed, bash, ...) > > I think some tools actually implement (a) already because we went through > this fifteen years ago or something like that, but I can't find the historic > discussion. GNU awk has a whole section on this: https://www.gnu.org/software/gawk/manual/html_node/Ranges-and-Locales.html http://austingroupbugs.net/view.php?id=1078 and nearby discussion on the Austin Group list may also be relevant to the discussion (namely, that if [[:digit:]] corresponds to the locale and ctype definition of isdigit(), it MUST be exactly 10 characters in ALL locales, and not add any non-ASCII-but-Unicode-digits, because doing that violates the principle of least surprise. (In reply to Florian Weimer from comment #14) > (In reply to Rich Felker from comment #13) > > I think bracket range expressions should just always be codepoint ranges. > > They can't reliably be used for anything else (since the behavior isn't > > specified), but anything else severely violates the principle of least > > surprise. > > Codepoint-based ranges would address my concerns. > > I was just trying to come up with something that would open a backdoor to > get access to collation-order-based ranges in case they are needed (because > some people seem to like them). But I can't see a useful application for > them, either. If you NEED collation-order-based range, we could spell it [[.0.]-[.9.]] (that is, the range between the two collating sequences. (In reply to Eric Blake from comment #30) > (In reply to Florian Weimer from comment #12) > > I find it very dubious that the current implementation of ranges is useful > > for anything at all, exception implementation convenience (as it's what we > > have today). > > > > Two possible improvements come to my mind: > > > > (a) If the both ranges are ASCII, match only ASCII characters. > > > > (b) Ranges include all characters with the same primary collation weight as > > the endpoints. > > > > It's possible to implement both, with (a) superseding (b). I'm not sure if > > today, range expressions can match collating elements consisting of multiple > > characters, in which case the following variant might be less surprising: > > > > (b') Ranges include all collating elements with the same primary weight as > > the endpoints. > > > > Both approaches are conforming to POSIX because ranges in other locales are > > undefined anyway. As far as I can see, available user feedback suggests > > that (a) is the expected behavior. > > Well, close to (a), at any rate. You're looking for Rational Range > Interpretation, which has been picked up by several GNU tools already (awk, > coreutils, sed, bash, ...) Yes, I finally found the old discussion, see comment 26. 8-) Do you know why bash doesn't default to it, and you have to enable it using “shopt -s globasciiranges”? (In reply to Florian Weimer from comment #32) > > Well, close to (a), at any rate. You're looking for Rational Range > > Interpretation, which has been picked up by several GNU tools already (awk, > > coreutils, sed, bash, ...) > > Yes, I finally found the old discussion, see comment 26. 8-) > > Do you know why bash doesn't default to it, and you have to enable it using > “shopt -s globasciiranges”? I'm guessing Chet was worried about potential backwards-compatibility issues, so he made an option rather than switching the default right away. I'm also wondering whether distros should pre-enable that shopt as part of their tweaks to the defaults used by their flavor of bash (along with any other defaults they tweak), if it would help matters. This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "GNU C Library master sources". The branch, master has been updated via 7cd7d36f1feb3ccacf476e909b115b45cdd46e77 (commit) from 3fb455b318c48f76280b4f8a0ff23b2cb1af9a3e (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=7cd7d36f1feb3ccacf476e909b115b45cdd46e77 commit 7cd7d36f1feb3ccacf476e909b115b45cdd46e77 Author: Carlos O'Donell <carlos@redhat.com> Date: Wed Jul 25 17:00:45 2018 -0400 Keep expected behaviour for [a-z] and [A-z] (Bug 23393). In commit 9479b6d5e08eacce06c6ab60abc9b2f4eb8b71e4 we updated all of the collation data to harmonize with the new version of ISO 14651 which is derived from Unicode 9.0.0. This collation update brought with it some changes to locales which were not desirable by some users, in particular it altered the meaning of the locale-dependent-range regular expression, namely [a-z] and [A-Z], and for en_US it caused uppercase letters to be matched by [a-z] for the first time. The matching of uppercase letters by [a-z] is something which is already known to users of other locales which have this property, but this change could cause significant problems to en_US and other similar locales that had never had this change before. Whether this behaviour is desirable or not is contentious and GNU Awk has this to say on the topic: https://www.gnu.org/software/gawk/manual/html_node/Ranges-and-Locales.html While the POSIX standard also has this further to say: "RE Bracket Expression": http://pubs.opengroup.org/onlinepubs/9699919799/xrat/V4_xbd_chap09.html "The current standard leaves unspecified the behavior of a range expression outside the POSIX locale. ... As noted above, efforts were made to resolve the differences, but no solution has been found that would be specific enough to allow for portable software while not invalidating existing implementations." In glibc we implement the requirement of ISO POSIX-2:1993 and use collation element order (CEO) to construct the range expression, the API internally is __collseq_table_lookup(). The fact that we use CEO and also have 4-level weights on each collation rule means that we can in practice reorder the collation rules in iso14651_t1_common (the new data) to provide consistent range expression resolution *and* the weights should maintain the expected total order. Therefore this patch does three things: * Reorder the collation rules for the LATIN script in iso14651_t1_common to deinterlace uppercase and lowercase letters in the collation element orders. * Adds new test data en_US.UTF-8.in for sort-test.sh which exercises strcoll* and strxfrm* and ensures the ISO 14651 collation remains. * Add back tests to tst-fnmatch.input and tst-regexloc.c which exercise that [a-z] does not match A or Z. The reordering of the ISO 14651 data is done in an entirely mechanical fashion using the following program attached to the bug: https://sourceware.org/bugzilla/show_bug.cgi?id=23393#c28 It is up for discussion if the iso14651_t1_common data should be refined further to have 3 very tight collation element ranges that include only a-z, A-Z, and 0-9, which would implement the solution sought after in: https://sourceware.org/bugzilla/show_bug.cgi?id=23393#c12 and implemented here: https://www.sourceware.org/ml/libc-alpha/2018-07/msg00854.html No regressions on x86_64. Verified that removal of the iso14651_t1_common change causes tst-fnmatch to regress with: 422: fnmatch ("[a-z]", "A", 0) = 0 (FAIL, expected FNM_NOMATCH) *** ... 425: fnmatch ("[A-Z]", "z", 0) = 0 (FAIL, expected FNM_NOMATCH) *** ----------------------------------------------------------------------- Summary of changes: ChangeLog | 11 + localedata/Makefile | 1 + localedata/locales/iso14651_t1_common | 1928 ++++++++++++++++---------------- posix/tst-fnmatch.input | 125 ++- posix/tst-regexloc.c | 8 +- 5 files changed, 1065 insertions(+), 1008 deletions(-) As a temprary measure I have committed the deinterleaving of upper and lower cases in iso14651_t1_common for glibc 2.28 to fix the surprises caused to en_US.UTF-8 users who do not want to have [a-z] match A-Y. This fixes the regression for 2.28, but doesn't fix this issue. This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "GNU C Library master sources". The branch, master has been updated via 49dddc3e99a69ecaa022ce703566b9abea2834c5 (commit) from 7cd7d36f1feb3ccacf476e909b115b45cdd46e77 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=49dddc3e99a69ecaa022ce703566b9abea2834c5 commit 49dddc3e99a69ecaa022ce703566b9abea2834c5 Author: Carlos O'Donell <carlos@redhat.com> Date: Wed Jul 25 21:58:10 2018 -0400 Add missing localedata/en_US.UTF-8.in (Bug 23393). Commit 7cd7d36f1feb3ccacf476e909b115b45cdd46e77 failed to include the new testing file en_US.UTF-8.in. ----------------------------------------------------------------------- Summary of changes: localedata/en_US.UTF-8.in | 2159 +++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 2159 insertions(+), 0 deletions(-) create mode 100644 localedata/en_US.UTF-8.in This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "GNU C Library master sources". The annotated tag, glibc-2.28 has been created at 0774a9618b539692317d0950477e16a8c5074caf (tag) tagging 3c03baca37fdcb52c3881e653ca392bba7a99c2b (commit) replaces glibc-2.27.9000 tagged by Carlos O'Donell on Wed Aug 1 01:20:23 2018 -0400 - Log ----------------------------------------------------------------- The GNU C Library ================= The GNU C Library version 2.28 is now available. The GNU C Library is used as *the* C library in the GNU system and in GNU/Linux systems, as well as many other systems that use Linux as the kernel. The GNU C Library is primarily designed to be a portable and high performance C library. It follows all relevant standards including ISO C11 and POSIX.1-2008. It is also internationalized and has one of the most complete internationalization interfaces known. The GNU C Library webpage is at http://www.gnu.org/software/libc/ Packages for the 2.28 release may be downloaded from: http://ftpmirror.gnu.org/libc/ http://ftp.gnu.org/gnu/libc/ The mirror list is at http://www.gnu.org/order/ftp.html NEWS for version 2.28 ===================== Major new features: * The localization data for ISO 14651 is updated to match the 2016 Edition 4 release of the standard, this matches data provided by Unicode 9.0.0. This update introduces significant improvements to the collation of Unicode characters. This release deviates slightly from the standard in that the collation element ordering for lowercase and uppercase LATIN script characters is adjusted to ensure that regular expressions with ranges like [a-z] and [A-Z] don't interleave e.g. A is not matched by [a-z]. With the update many locales have been updated to take advantage of the new collation information. The new collation information has increased the size of the compiled locale archive or binary locales. * The GNU C Library can now be compiled with support for Intel CET, AKA Intel Control-flow Enforcement Technology. When the library is built with --enable-cet, the resulting glibc is protected with indirect branch tracking (IBT) and shadow stack (SHSTK). CET-enabled glibc is compatible with all existing executables and shared libraries. This feature is currently supported on i386, x86_64 and x32 with GCC 8 and binutils 2.29 or later. Note that CET-enabled glibc requires CPUs capable of multi-byte NOPs, like x86-64 processors as well as Intel Pentium Pro or newer. NOTE: --enable-cet has been tested for i686, x86_64 and x32 on non-CET processors. --enable-cet has been tested for x86_64 and x32 on CET SDVs, but Intel CET support hasn't been validated for i686. * The GNU C Library now has correct support for ABSOLUTE symbols (SHN_ABS-relative symbols). Previously such ABSOLUTE symbols were relocated incorrectly or in some cases discarded. The GNU linker can make use of the newer semantics, but it must communicate it to the dynamic loader by setting the ELF file's identification (EI_ABIVERSION field) to indicate such support is required. * Unicode 11.0.0 Support: Character encoding, character type info, and transliteration tables are all updated to Unicode 11.0.0, using generator scripts contributed by Mike FABIAN (Red Hat). * <math.h> functions that round their results to a narrower type are added from TS 18661-1:2014 and TS 18661-3:2015: - fadd, faddl, daddl and corresponding fMaddfN, fMaddfNx, fMxaddfN and fMxaddfNx functions. - fsub, fsubl, dsubl and corresponding fMsubfN, fMsubfNx, fMxsubfN and fMxsubfNx functions. - fmul, fmull, dmull and corresponding fMmulfN, fMmulfNx, fMxmulfN and fMxmulfNx functions. - fdiv, fdivl, ddivl and corresponding fMdivfN, fMdivfNx, fMxdivfN and fMxdivfNx functions. * Two grammatical forms of month names are now supported for the following languages: Armenian, Asturian, Catalan, Czech, Kashubian, Occitan, Ossetian, Scottish Gaelic, Upper Sorbian, and Walloon. The following languages now support two grammatical forms in abbreviated month names: Catalan, Greek, and Kashubian. * Newly added locales: Lower Sorbian (dsb_DE) and Yakut (sah_RU) also include the support for two grammatical forms of month names. * Building and running on GNU/Hurd systems now works without out-of-tree patches. * The renameat2 function has been added, a variant of the renameat function which has a flags argument. If the flags are zero, the renameat2 function acts like renameat. If the flag is not zero and there is no kernel support for renameat2, the function will fail with an errno value of EINVAL. This is different from the existing gnulib function renameatu, which performs a plain rename operation in case of a RENAME_NOREPLACE flags and a non-existing destination (and therefore has a race condition that can clobber the destination inadvertently). * The statx function has been added, a variant of the fstatat64 function with an additional flags argument. If there is no direct kernel support for statx, glibc provides basic stat support based on the fstatat64 function. * IDN domain names in getaddrinfo and getnameinfo now use the system libidn2 library if installed. libidn2 version 2.0.5 or later is recommended. If libidn2 is not available, internationalized domain names are not encoded or decoded even if the AI_IDN or NI_IDN flags are passed to getaddrinfo or getnameinfo. (getaddrinfo calls with non-ASCII names and AI_IDN will fail with an encoding error.) Flags which used to change the IDN encoding and decoding behavior (AI_IDN_ALLOW_UNASSIGNED, AI_IDN_USE_STD3_ASCII_RULES, NI_IDN_ALLOW_UNASSIGNED, NI_IDN_USE_STD3_ASCII_RULES) have been deprecated. They no longer have any effect. * Parsing of dynamic string tokens in DT_RPATH, DT_RUNPATH, DT_NEEDED, DT_AUXILIARY, and DT_FILTER has been expanded to support the full range of ELF gABI expressions including such constructs as '$ORIGIN$ORIGIN' (if valid). For SUID/GUID applications the rules have been further restricted, and where in the past a dynamic string token sequence may have been interpreted as a literal string it will now cause a load failure. These load failures were always considered unspecified behaviour from the perspective of the dynamic loader, and for safety are now load errors e.g. /foo/${ORIGIN}.so in DT_NEEDED results in a load failure now. * Support for ISO C threads (ISO/IEC 9899:2011) has been added. The implementation includes all the standard functions provided by <threads.h>: - thrd_current, thrd_equal, thrd_sleep, thrd_yield, thrd_create, thrd_detach, thrd_exit, and thrd_join for thread management. - mtx_init, mtx_lock, mtx_timedlock, mtx_trylock, mtx_unlock, and mtx_destroy for mutual exclusion. - call_once for function call synchronization. - cnd_broadcast, cnd_destroy, cnd_init, cnd_signal, cnd_timedwait, and cnd_wait for conditional variables. - tss_create, tss_delete, tss_get, and tss_set for thread-local storage. Application developers must link against libpthread to use ISO C threads. Deprecated and removed features, and other changes affecting compatibility: * The nonstandard header files <libio.h> and <_G_config.h> are no longer installed. Software that was using either header should be updated to use standard <stdio.h> interfaces instead. * The stdio functions 'getc' and 'putc' are no longer defined as macros. This was never required by the C standard, and the macros just expanded to call alternative names for the same functions. If you hoped getc and putc would provide performance improvements over fgetc and fputc, instead investigate using (f)getc_unlocked and (f)putc_unlocked, and, if necessary, flockfile and funlockfile. * All stdio functions now treat end-of-file as a sticky condition. If you read from a file until EOF, and then the file is enlarged by another process, you must call clearerr or another function with the same effect (e.g. fseek, rewind) before you can read the additional data. This corrects a longstanding C99 conformance bug. It is most likely to affect programs that use stdio to read interactive input from a terminal. (Bug #1190.) * The macros 'major', 'minor', and 'makedev' are now only available from the header <sys/sysmacros.h>; not from <sys/types.h> or various other headers that happen to include <sys/types.h>. These macros are rarely used, not part of POSIX nor XSI, and their names frequently collide with user code; see https://sourceware.org/bugzilla/show_bug.cgi?id=19239 for further explanation. <sys/sysmacros.h> is a GNU extension. Portable programs that require these macros should first include <sys/types.h>, and then include <sys/sysmacros.h> if __GNU_LIBRARY__ is defined. * The tilegx*-*-linux-gnu configurations are no longer supported. * The obsolete function ustat is no longer available to newly linked binaries; the headers <ustat.h> and <sys/ustat.h> have been removed. This function has been deprecated in favor of fstatfs and statfs. * The obsolete function nfsservctl is no longer available to newly linked binaries. This function was specific to systems using the Linux kernel and could not usefully be used with the GNU C Library on systems with version 3.1 or later of the Linux kernel. * The obsolete function name llseek is no longer available to newly linked binaries. This function was specific to systems using the Linux kernel and was not declared in a header. Programs should use the lseek64 name for this function instead. * The AI_IDN_ALLOW_UNASSIGNED and NI_IDN_ALLOW_UNASSIGNED flags for the getaddrinfo and getnameinfo functions have been deprecated. The behavior previously selected by them is now always enabled. * The AI_IDN_USE_STD3_ASCII_RULES and NI_IDN_USE_STD3_ASCII_RULES flags for the getaddrinfo and getnameinfo functions have been deprecated. The STD3 restriction (rejecting '_' in host names, among other things) has been removed, for increased compatibility with non-IDN name resolution. * The fcntl function now have a Long File Support variant named fcntl64. It is added to fix some Linux Open File Description (OFD) locks usage on non LFS mode. As for others *64 functions, fcntl64 semantics are analogous with fcntl and LFS support is handled transparently. Also for Linux, the OFD locks act as a cancellation entrypoint. * The obsolete functions encrypt, encrypt_r, setkey, setkey_r, cbc_crypt, ecb_crypt, and des_setparity are no longer available to newly linked binaries, and the headers <rpc/des_crypt.h> and <rpc/rpc_des.h> are no longer installed. These functions encrypted and decrypted data with the DES block cipher, which is no longer considered secure. Software that still uses these functions should switch to a modern cryptography library, such as libgcrypt. * Reflecting the removal of the encrypt and setkey functions above, the macro _XOPEN_CRYPT is no longer defined. As a consequence, the crypt function is no longer declared unless _DEFAULT_SOURCE or _GNU_SOURCE is enabled. * The obsolete function fcrypt is no longer available to newly linked binaries. It was just another name for the standard function crypt, and it has not appeared in any header file in many years. * We have tentative plans to hand off maintenance of the passphrase-hashing library, libcrypt, to a separate development project that will, we hope, keep up better with new passphrase-hashing algorithms. We will continue to declare 'crypt' in <unistd.h>, and programs that use 'crypt' or 'crypt_r' should not need to change at all; however, distributions will need to install <crypt.h> and libcrypt from a separate project. In this release, if the configure option --disable-crypt is used, glibc will not install <crypt.h> or libcrypt, making room for the separate project's versions of these files. The plan is to make this the default behavior in a future release. Changes to build and runtime requirements: GNU make 4.0 or later is now required to build glibc. Security related changes: CVE-2016-6261, CVE-2016-6263, CVE-2017-14062: Various vulnerabilities have been fixed by removing the glibc-internal IDNA implementation and using the system-provided libidn2 library instead. Originally reported by Hanno Böck and Christian Weisgerber. CVE-2017-18269: An SSE2-based memmove implementation for the i386 architecture could corrupt memory. Reported by Max Horn. CVE-2018-11236: Very long pathname arguments to realpath function could result in an integer overflow and buffer overflow. Reported by Alexey Izbyshev. CVE-2018-11237: The mempcpy implementation for the Intel Xeon Phi architecture could write beyond the target buffer, resulting in a buffer overflow. Reported by Andreas Schwab. The following bugs are resolved with this release: [1190] stdio: fgetc()/fread() behaviour is not POSIX compliant [6889] manual: 'PWD' mentioned but not specified [13575] libc: SSIZE_MAX defined as LONG_MAX is inconsistent with ssize_t, when __WORDSIZE != 64 [13762] regex: re_search etc. should return -2 on memory exhaustion [13888] build: /tmp usage during testing [13932] math: dbl-64 pow unexpectedly slow for some inputs [14092] nptl: Support C11 threads [14095] localedata: Review / update collation data from Unicode / ISO 14651 [14508] libc: -Wformat warnings [14553] libc: Namespace pollution loff_t in sys/types.h [14890] libc: Make NT_PRFPREG canonical. [15105] libc: Extra PLT references with -Os [15512] libc: __bswap_constant_16 not compiled when -Werror -Wsign- conversion is given [16335] manual: Feature test macro documentation incomplete and out of date [16552] libc: Unify umount implementations in terms of umount2 [17082] libc: htons et al.: statement-expressions prevent use on global scope with -O1 and higher [17343] libc: Signed integer overflow in /stdlib/random_r.c [17438] localedata: pt_BR: wrong d_fmt delimiter [17662] libc: please implement binding for the new renameat2 syscall [17721] libc: __restrict defined as /* Ignore */ even in c11 [17979] libc: inconsistency between uchar.h and stdint.h [18018] dynamic-link: Additional $ORIGIN handling issues (CVE-2011-0536) [18023] libc: extend_alloca is broken (questionable pointer comparison, horrible machine code) [18124] libc: hppa: setcontext erroneously returns -1 as exit code for last constant. [18471] libc: llseek should be a compat symbol [18473] soft-fp: [powerpc-nofpu] __sqrtsf2, __sqrtdf2 should be compat symbols [18991] nss: nss_files skips large entry in database [19239] libc: Including stdlib.h ends up with macros major and minor being defined [19463] libc: linknamespace failures when compiled with -Os [19485] localedata: csb_PL: Update month translations + add yesstr/nostr [19527] locale: Normalized charset name not recognized by setlocale [19667] string: Missing Sanity Check for malloc calls in file 'testcopy.c' [19668] libc: Missing Sanity Check for malloc() in file 'tst-setcontext- fpscr.c' [19728] network: out of bounds stack read in libidn function idna_to_ascii_4i (CVE-2016-6261) [19729] network: out of bounds heap read on invalid utf-8 inputs in stringprep_utf8_nfkc_normalize (CVE-2016-6263) [19818] dynamic-link: Absolute (SHN_ABS) symbols incorrectly relocated by the base address [20079] libc: Add SHT_X86_64_UNWIND to elf.h [20251] libc: 32bit programs pass garbage in struct flock for OFD locks [20419] dynamic-link: files with large allocated notes crash in open_verify [20530] libc: bswap_16 should use __builtin_bswap16() when available [20890] dynamic-link: ldconfig: fsync the files before atomic rename [20980] manual: CFLAGS environment variable replaces vital options [21163] regex: Assertion failure in pop_fail_stack when executing a malformed regexp (CVE-2015-8985) [21234] manual: use of CFLAGS makes glibc detect no optimization [21269] dynamic-link: i386 sigaction sa_restorer handling is wrong [21313] build: Compile Error GCC 5.4.0 MIPS with -0S [21314] build: Compile Error GCC 5.2.0 MIPS with -0s [21508] locale: intl/tst-gettext failure with latest msgfmt [21547] localedata: Tibetan script collation broken (Dzongkha and Tibetan) [21812] network: getifaddrs() returns entries with ifa_name == NULL [21895] libc: ppc64 setjmp/longjmp not fully interoperable with static dlopen [21942] dynamic-link: _dl_dst_substitute incorrectly handles $ORIGIN: with AT_SECURE=1 [22241] localedata: New locale: Yakut (Sakha) locale for Russia (sah_RU) [22247] network: Integer overflow in the decode_digit function in puny_decode.c in libidn (CVE-2017-14062) [22342] nscd: NSCD not properly caching netgroup [22391] nptl: Signal function clear NPTL internal symbols inconsistently [22550] localedata: es_ES locale (and other es_* locales): collation should treat ñ as a primary different character, sync the collation for Spanish with CLDR [22638] dynamic-link: sparc: static binaries are broken if glibc is built by gcc configured with --enable-default-pie [22639] time: year 2039 bug for localtime etc. on 64-bit platforms [22644] string: memmove-sse2-unaligned on 32bit x86 produces garbage when crossing 2GB threshold (CVE-2017-18269) [22646] localedata: redundant data (LC_TIME) for es_CL, es_CU, es_EC and es_BO [22735] time: Misleading typo in time.h source comment regarding CLOCKS_PER_SECOND [22753] libc: preadv2/pwritev2 fallback code should handle offset=-1 [22761] libc: No trailing `%n' conversion specifier in FMT passed from `__assert_perror_fail ()' to `__assert_fail_base ()' [22766] libc: all glibc internal dlopen should use RTLD_NOW for robust dlopen failures [22786] libc: Stack buffer overflow in realpath() if input size is close to SSIZE_MAX (CVE-2018-11236) [22787] dynamic-link: _dl_check_caller returns false when libc is linked through an absolute DT_NEEDED path [22792] build: tcb-offsets.h dependency dropped [22797] libc: pkey_get() uses non-reserved name of argument [22807] libc: PTRACE_* constants missing for powerpc [22818] glob: posix/tst-glob_lstat_compat failure on alpha [22827] dynamic-link: RISC-V ELF64 parser mis-reads flag in ldconfig [22830] malloc: malloc_stats doesn't restore cancellation state on stderr [22848] localedata: ca_ES: update date definitions from CLDR [22862] build: _DEFAULT_SOURCE is defined even when _ISOC11_SOURCE is [22884] math: RISCV fmax/fmin handle signalling NANs incorrectly [22896] localedata: Update locale data for an_ES [22902] math: float128 test failures with GCC 8 [22918] libc: multiple common of `__nss_shadow_database' [22919] libc: sparc32: backtrace yields infinite backtrace with makecontext [22926] libc: FTBFS on powerpcspe [22932] localedata: lt_LT: Update of abbreviated month names from CLDR required [22937] localedata: Greek (el_GR, el_CY) locales actually need ab_alt_mon [22947] libc: FAIL: misc/tst-preadvwritev2 [22963] localedata: cs_CZ: Add alternative month names [22987] math: [powerpc/sparc] fdim inlines errno, exceptions handling [22996] localedata: change LC_PAPER to en_US in es_BO locale [22998] dynamic-link: execstack tests are disabled when SELinux is disabled [23005] network: Crash in __res_context_send after memory allocation failure [23007] math: strtod cannot handle -nan [23024] nss: getlogin_r is performing NSS lookups when loginid isn't set [23036] regex: regex equivalence class regression [23037] libc: initialize msg_flags to zero for sendmmsg() calls [23069] libc: sigaction broken on riscv64-linux-gnu [23094] localedata: hr_HR: wrong thousands_sep and mon_thousands_sep [23102] dynamic-link: Incorrect parsing of multiple consecutive $variable patterns in runpath entries (e.g. $ORIGIN$ORIGIN) [23137] nptl: s390: pthread_join sometimes block indefinitely (on 31bit and libc build with -Os) [23140] localedata: More languages need two forms of month names [23145] libc: _init/_fini aren't marked as hidden [23152] localedata: gd_GB: Fix typo in "May" (abbreviated) [23171] math: C++ iseqsig for long double converts arguments to double [23178] nscd: sudo will fail when it is run in concurrent with commands that changes /etc/passwd [23196] string: __mempcpy_avx512_no_vzeroupper mishandles large copies (CVE-2018-11237) [23206] dynamic-link: static-pie + dlopen breaks debugger interaction [23208] localedata: New locale - Lower Sorbian (dsb) [23233] regex: Memory leak in build_charclass_op function in file posix/regcomp.c [23236] stdio: Harden function pointers in _IO_str_fields [23250] nptl: Offset of __private_ss differs from GCC [23253] math: tgamma test suite failures on i686 with -march=x86-64 -mtune=generic -mfpmath=sse [23259] dynamic-link: Unsubstituted ${ORIGIN} remains in DT_NEEDED for AT_SECURE [23264] libc: posix_spawnp wrongly executes ENOEXEC in non compat mode [23266] nis: stringop-truncation warning with new gcc8.1 in nisplus- parser.c [23272] math: fma(INFINITY,INFIITY,0.0) should be INFINITY [23277] math: nan function should not have const attribute [23279] math: scanf and strtod wrong for some hex floating-point [23280] math: wscanf rounds wrong; wcstod is ok for negative numbers and directed rounding [23290] localedata: IBM273 is not equivalent to ISO-8859-1 [23303] build: undefined reference to symbol '__parse_hwcap_and_convert_at_platform@@GLIBC_2.23' [23307] dynamic-link: Absolute symbols whose value is zero ignored in lookup [23313] stdio: libio vtables validation and standard file object interposition [23329] libc: The __libc_freeres infrastructure is not properly run across DSO boundaries. [23349] libc: Various glibc headers no longer compatible with <linux/time.h> [23351] malloc: Remove unused code related to heap dumps and malloc checking [23363] stdio: stdio-common/tst-printf.c has non-free license [23396] regex: Regex equivalence regression in single-byte locales [23422] localedata: oc_FR: More updates of locale data [23442] build: New warning with GCC 8 [23448] libc: Out of bounds access in IBM-1390 converter [23456] libc: Wrong index_cpu_LZCNT [23458] build: tst-get-cpu-features-static isn't added to tests [23459] libc: COMMON_CPUID_INDEX_80000001 isn't populated for Intel processors [23467] dynamic-link: x86/CET: A property note parser bug Release Notes ============= https://sourceware.org/glibc/wiki/Release/2.28 Contributors ============ This release was made possible by the contributions of many people. The maintainers are grateful to everyone who has contributed changes or bug reports. These include: Adhemerval Zanella Agustina Arzille Alan Modra Alexandre Oliva Amit Pawar Andreas Schwab Andrew Senkevich Andrew Waterman Aurelien Jarno Carlos O'Donell Chung-Lin Tang DJ Delorie Daniel Alvarez David Michael Dmitry V. Levin Dragan Stanojevic - Nevidljivi Florian Weimer Flávio Cruz Francois Goichon Gabriel F. T. Gomes H.J. Lu Herman ten Brugge Hongbo Zhang Igor Gnatenko Jesse Hathaway John David Anglin Joseph Myers Leonardo Sandoval Maciej W. Rozycki Mark Wielaard Martin Sebor Michael Wolf Mike FABIAN Patrick McGehearty Patsy Franklin Paul Pluzhnikov Quentin PAGÈS Rafal Luzynski Rajalakshmi Srinivasaraghavan Raymond Nicholson Rical Jasan Richard Braun Robert Buj Rogerio Alves Samuel Thibault Sean McKean Siddhesh Poyarekar Stefan Liebler Steve Ellcey Sylvain Lesage Szabolcs Nagy Thomas Schwinge Tulio Magno Quites Machado Filho Valery Timiriliyev Vincent Chen Wilco Dijkstra Zack Weinberg Zong Li -----BEGIN PGP SIGNATURE----- iQIcBAABAgAGBQJbYUMhAAoJEBZ5K06iU0D4LV8QAJDI+9To34wUWGmYUmV48NFx 9Mug7Yd7Y8kpo0Rxi/yPpBBAjQadz4zJftkvZJUlZsYL83jypgRhxlXaOvyBATqT COHK3+RRaKqTcnBgSQmR34tGJh1k9CSfvfmRWxs1SycQQMhTbkQ7bLEGJEWDava6 PYCsQloDAaZdjumHNCoyTbg9fObqUlyqw3OyRJYWx07Bbl2nQc6Y/WLb4pgdWz0Y yy7kNM6P70+uFbb/+9iPnXJ4avWbpXO68Y1WeuMFtiL7sQ/qr6sNQ1HHdqut94LB XF7tiQ3/vWkMoJT+GkQr0rhrlTXBv+h77NFTPuewRPviYWgIWMThk3T7D2+TM8Sn Y9hkKTpCA2qrDRK6IMMzxKAfo9+DyO66cSXM3cwCzKOtpMXdlZqRg9TlAFMjmXGr r1KFpZzdHdw5qqktYQnIa1JBh0+31JhWXB/XxvoJx5nSDuBbJ4x55M8IeG3PCy3x ejgCJ6bJODOChlGhE6FN4VJM+WSjd8ZY8K4T2XGdP+3zVc+zyNqLDTpdydR6t1nB H5Peqbg12g8IJD7kY/i4Jm2uFpxP32CD3lUhp2gEbACRlZTmcxc6Bl13jgEdgKrW AD1dxH7i9xI/Rff2hp23U5d1NAiJmWTfAgUU2939rYU+02UWUPnk/TvzMzIaTYGo MIRvKIvblBn6bCUxYTQP =dTj9 -----END PGP SIGNATURE----- Adhemerval Zanella (48): Update SH libm-tests-ulps Rename nptl-signals.h to internal-signals.h Refactor atfork handlers Update sparc ulps i386: Fix i386 sigaction sa_restorer initialization (BZ#21269) nptl: Fix tst-cancel4 sendto tests Define _DIRENT_MATCHES_DIRENT64 regardless Refactor Linux ARCH_FORK implementation powerpc: Fix TLE build for SPE (BZ #22926) sparc: Fix arch_fork definition Add Changelog reference to BZ#23024 Assume O_DIRECTORY for opendir Filter out NPTL internal signals (BZ #22391) linux: Consolidate sigaction implementation Update ARM libm-test-ulps. Update SPARC libm-test-ulps. Update i386 libm-test-ulps. Consolidate Linux readdir{64}{_r} implementation arm: Remove ununsed ARM code in optimized implementation Consolidate Linux getdents{64} implementation Fix mips64n32 getdents alias Consolidate scandir{at}{64} implementation Update hppa libm-test-ulps Consolidate alphasort{64} and versionsort{64} implementation Consolidate getdirentries{64} implementation Consolidate Linux readahead implementation Deprecate ustat syscall interface Fix ChangeLog from cf2478d53ad commit Fix concurrent changes on nscd aware files (BZ #23178) posix: Fix posix_spawnp to not execute invalid binaries in non compat mode (BZ#23264) Fix Linux fcntl OFD locks for non-LFS architectures (BZ#20251) Revert hurd errno.h changes Fix hurd expected fcntl version posix: Sync gnulib regex implementation posix: Fix bug-regex33 after regex sync Comment tst-ofdlocks-compat expected failure in some Linux releases nptl: Add C11 threads thrd_* functions nptl: Add C11 threads mtx_* functions nptl: Add C11 threads call_once functions nptl: Add C11 threads cnd_* functions nptl: Add C11 threads tss_* functions nptl: Add abilist symbols for C11 threads nptl: Add test cases for ISO C11 threads Mention ISO C threads addition Fix C11 conformance issues Fix ISO C threads installed header and HURD assumption Fix Linux fcntl OFD locks on unsupported kernels Update SH libm-tests-ulps Agustina Arzille (2): hurd: Rewrite __libc_cleanup_* hurd: Reimplement libc locks using mach's gsync Alan Modra (1): R_PARISC_TLS_DTPOFF32 reloc handling Alexandre Oliva (1): Revert: Amit Pawar (1): Use AVX_Fast_Unaligned_Load from Zen onwards. Andreas Schwab (11): Fix uninitialized variable in assert_perror (bug 22761) Fix multiple definitions of __nss_*_database (bug 22918) RISC-V: add remaining relocations Fix crash in resolver on memory allocation failure (bug 23005) Fix missing @ before texinfo command Add aliases to recognize normalized charset names (bug 19527) Fix comment typo Remove unneeded setting of errno after malloc failure Don't write beyond destination in __mempcpy_avx512_no_vzeroupper (bug 23196) Fix out-of-bounds access in IBM-1390 converter (bug 23448) Fix out of bounds access in findidxwc (bug 23442) Andrew Senkevich (1): Fix i386 memmove issue (bug 22644). Andrew Waterman (1): RISC-V: fmax/fmin: Handle signalling NaNs correctly. Aurelien Jarno (4): intl/tst-gettext: fix failure with newest msgfmt Fix posix/tst-glob_lstat_compat on alpha [BZ #22818] sparc32: Add nop before __startcontext to stop unwinding [BZ #22919] Add tst-sigaction.c to test BZ #23069 Carlos O'Donell (17): Fix -Os log1p, log1pf build (bug 21314). Improve DST handling (Bug 23102, Bug 21942, Bug 18018, Bug 23259). Fix fallback path in __pthread_mutex_timedlock (). Fix comments in _dl_dst_count and _dl_dst_substitute. libc: Extend __libc_freeres framework (Bug 23329). locale: XFAIL newlocale usage in static binary (Bug 23164) Keep expected behaviour for [a-z] and [A-z] (Bug 23393). Add missing localedata/en_US.UTF-8.in (Bug 23393). Update libc.pot. Update NEWS with ISO 14651 update information. Update translations for cs, pl, and uk. Update translations for bg, de, hr, pt_BR, sv, and vi. Update translation for be. Update contrib.texi contributions. Update tooling versions verified to work with glibc. Synchronize translation project PO files. Update NEWS, version.h, and features.h for glibc 2.28. Chung-Lin Tang (1): Update sysdeps/nios2/libm-test-ulps DJ Delorie (5): [RISC-V] Fix parsing flags in ELF64 files. RISC-V: Do not initialize $gp in TLS macros. Update ChangeLog for BZ 22884 - riscv fmax/fmin [BZ #22342] Fix netgroup cache keys. Update kernel version in syscall-names.list to 4.16. Daniel Alvarez (1): getifaddrs: Don't return ifa entries with NULL names [BZ #21812] David Michael (1): Lookup the startup server through /servers/startup Dmitry V. Levin (3): linux/aarch64: sync sys/ptrace.h with Linux 4.15 [BZ #22433] linux/powerpc: sync sys/ptrace.h with Linux 4.15 [BZ #22433, #22807] Update translations from the Translation Project Dragan Stanojevic - Nevidljivi (1): hr_HR locale: fix thousands_sep and mon_thousands_sep Florian Weimer (72): preadv2/pwritev2: Handle offset == -1 [BZ #22753] Record CVE-2018-6551 in NEWS and ChangeLog [BZ #22774] getlogin_r: switch Linux variant to struct scratch_buffer elf: Remove ad-hoc restrictions on dlopen callers [BZ #22787] ldconfig: Sync temporary files to disk before renaming them [BZ #20890] nptl: Move pthread_atfork to libc_nonshared.a nptl: Drop libpthread_nonshared.a from libpthread.so nptl: Turn libpthread.so into a symbolic link to the real DSO malloc: Revert sense of prev_inuse in comments Linux i386: tst-bz21269 triggers SIGBUS on some kernels support_format_addrinfo: Include unknown error number in result inet: Actually build and run tst-deadline manual: Move mbstouwcs to an example C file manual: Various fixes to the mbstouwcs example, and mbrtowc update resolv: Fully initialize struct mmsghdr in send_dg [BZ #23037] sunrpc: Remove stray exports without --enable-obsolete-rpc [BZ #23166] time: Use 64-bit time values for time zone parsing math: Merge strtod_nan_*.h into math-type-macros-*.h support: Add TEST_COMPARE_BLOB, support_quote_blob math: Reverse include order in <math-type-macros-*.h> i386: Drop -mpreferred-stack-boundary=4 Implement allocate_once for atomic initialization with allocation Switch IDNA implementation to libidn2 [BZ #19728] [BZ #19729] [BZ #22247] Add references to CVE-2017-18269, CVE-2018-11236, CVE-2018-11237 stdlib: Additional tests need generated locale dependencies support: Add wrappers for pthread_barrierattr_t libio: Avoid _allocate_buffer, _free_buffer function pointers [BZ #23236] Remove sysdeps/generic/libcidn.abilist math: Update i686 ulps math: Update i686 ulps (--disable-multi-arch configuration) x86: Make strncmp usable from rtld scripts/update-abilist.sh: Accept empty list of files to patch localedata: Make IBM273 compatible with ISO-8859-1 [BZ #23290] Linux: Create Netlink socket with SOCK_CLOEXEC in __check_pf [BZ #15722] libio: Avoid ptrdiff_t overflow in IO_validate_vtable math: Set 387 and SSE2 rounding mode for tgamma on i386 [BZ #23253] nscd restart: Use malloc instead of extend_alloca [BZ #18023] nscd: Use struct scratch_buffer, not extend_alloca in most caches [BZ #18023] nscd: Switch to struct scratch_buffer in adhstaiX [BZ #18023] getgrent_next_nss (compat-initgroups): Remove alloca fallback [BZ #18023] _nss_nis_initgroups_dyn: Use struct scratch_buffer [BZ #18023] getent: Use dynarray in initgroups_keys [BZ #18023] nss_files: Use struct scratch_buffer instead of extend_alloca [BZ #18023] libio: Disable vtable validation in case of interposition [BZ #23313] support: Add TEST_NO_SETVBUF libio: Add tst-vtables, tst-vtables-interposed sunrpc: Remove always-defined _RPC_THREAD_SAFE_ macro Run thread shutdown functions in an explicit order wordexp: Rewrite parse_tilde to use struct scratch_buffer [BZ #18023] gethostid (Linux variant): Switch to struct scratch_buffer [BZ #18023] _dl_map_object_deps: Use struct scratch_buffer [BZ #18023] Remove macros extend_alloca, extend_alloca_account [BZ #18023] Use _STRUCT_TIMESPEC as guard in <bits/types/struct_timespec.h> [BZ #23349] malloc: Update heap dumping/undumping comments [BZ #23351] stdio-common/tst-printf.c: Remove part under a non-free license [BZ #23363] testrun.sh: Implement --tool=strace, --tool=valgrind Add renameat2 function [BZ #17662] Compile debug/stack_chk_fail_local.c with stack protector Build csu/elf-init.c and csu/static-reloc.c with stack protector conform/conformtest.pl: Escape literal braces in regular expressions libio: Implement internal function __libc_readline_unlocked nss_files: Fix re-reading of long lines [BZ #18991] Fix copyright years in recent commits regexec: Fix off-by-one bug in weight comparison [BZ #23036] Add the statx function Install <bits/statx.h> header nptl: Use __mprotect consistently for _STACK_GROWS_UP regcomp: Fix off-by-one bug in build_equiv_class [BZ #23396] sh: Do not define __ASSUME_STATX alpha: mlock2, copy_file_range syscalls were introduced in kernel 4.13 C11 threads: Fix timeout and locking issues htl: Use weak aliases for public symbols Flávio Cruz (1): hurd: Define and pass UTIME_NOW and UTIME_OMIT to new file_utimens RPC Francois Goichon (1): malloc: harden removal from unsorted list Gabriel F. T. Gomes (3): powerpc64*: fix the order of implied sysdeps directories Fix parameter type in C++ version of iseqsig (bug 23171) ldbl-128ibm-compat: Add printf_size H.J. Lu (76): sparc: Check PIC instead of SHARED in start.S [BZ #22638] x86-64: Use __glibc_likely/__glibc_likely in dl-machine.h Add a missing ChangeLog item in commit 371b220f620 Fix a typo in ChangeLog entry i386: Use __glibc_likely/__glibc_likely in dl-machine.h Add DT_SYMTAB_SHNDX from gABI Use ADDRIDX with DT_GNU_HASH Define GEN_AS_CONST_HEADERS when generating header files [BZ #22792] Fix a typo in ChangeLog (bit_cpu_BIT -> bit_cpu_IBT) Fix a typo in ChangeLog: auch_fork -> arch_fork Remove hidden __libc_longjmp Add $(tests-execstack-$(have-z-execstack)) after defined [BZ #22998] Update RWF_SUPPORTED for Linux kernel 4.16 [BZ #22947] x86: Use pad in pthread_unwind_buf to preserve shadow stack register x86-64/setcontext: Pop the pointer into %rdx after syscall cl x86-64/swapcontext: Restore the pointer into %rdx after syscall x86-64/memset: Mark the debugger symbol as hidden x86-64: Remove the unnecessary testl in strlen-avx2.S x86: Add sysdeps/x86/ldsodefs.h i386: Replace PREINIT_FUNCTION@PLT with *%eax in call x86-64: Use IFUNC strncat inside libc.so nptl: Remove __ASSUME_PRIVATE_FUTEX Initial Fast Short REP MOVSB (FSRM) support x86-64: Check Prefer_FSRM in ifunc-memmove.h Add a test case for [BZ #23196] x86-64: Skip zero length in __mem[pcpy|move|set]_erms static-PIE: Update DT_DEBUG for debugger [BZ #23206] Mark _init and _fini as hidden [BZ #23145] i386: Change offset of __private_ss to 0x30 [BZ #23250] benchtests: Add -f/--functions argument x86: Rename __glibc_reserved1 to feature_1 in tcbhead_t [BZ #22563] x86: Support shadow stack pointer in setjmp/longjmp x86_64: Undef SHADOW_STACK_POINTER_OFFSET last x86: Support IBT and SHSTK in Intel CET [BZ #21598] x86: Always include <dl-cet.h>/cet-tunables.h> for --enable-cet x86: Add _CET_ENDBR to functions in crti.S x86: Add _CET_ENDBR to functions in dl-tlsdesc.S x86-64: Add _CET_ENDBR to STRCMP_SSE42 i386: Add _CET_ENDBR to indirect jump targets in add_n.S/sub_n.S x86_64: Use _CET_NOTRACK in strcmp.S x86-64: Use _CET_NOTRACK in strcpy-sse2-unaligned.S x86-64: Use _CET_NOTRACK in strcmp-sse42.S x86-64: Use _CET_NOTRACK in memcpy-ssse3-back.S x86-64: Use _CET_NOTRACK in memcpy-ssse3.S i386: Use _CET_NOTRACK in i686/memcmp.S i386: Use _CET_NOTRACK in memset-sse2.S i386: Use _CET_NOTRACK in memcmp-sse4.S i386: Use _CET_NOTRACK in memcpy-ssse3-rep.S i386: Use _CET_NOTRACK in memcpy-ssse3.S i386: Use _CET_NOTRACK in strcpy-sse2.S i386: Use _CET_NOTRACK in strcat-sse2.S i386: Use _CET_NOTRACK in memset-sse2-rep.S x86-64: Use _CET_NOTRACK in memcmp-sse4.S Intel CET: Document --enable-cet x86/CET: Document glibc.tune.x86_ibt and glibc.tune.x86_shstk INSTALL: Add a note for Intel CET status x86-64: Add endbr64 to tst-quadmod[12].S x86: Update vfork to pop shadow stack Add <bits/indirect-return.h> x86/CET: Extend arch_prctl syscall for CET control x86: Rename __glibc_reserved2 to ssp_base in tcbhead_t x86/CET: Add tests with legacy non-CET shared objects Add a test for multiple makecontext calls Add another test for setcontext Add a test for multiple setcontext calls Add tests for setcontext on the context from makecontext x86-64/CET: Extend ucontext_t to save shadow stack x86/CET: Add a setcontext test for CET ia64: Work around incorrect type of IA64 uc_sigmask x86: Correct index_cpu_LZCNT [BZ # 23456] x86: Populate COMMON_CPUID_INDEX_80000001 for Intel CPUs [BZ #23459] Add the missing ChangeLog entry for commit be525a69a66 x86/CET: Don't parse beyond the note end x86: Add tst-get-cpu-features-static to $(tests) [BZ #23458] x86/CET: Fix property note parser [BZ #23467] Herman ten Brugge (1): Fix sign of NaN returned by strtod (bug 23007). Hongbo Zhang (1): aarch64: add HXT Phecda core memory operation ifuncs Igor Gnatenko (1): Linux: use reserved name __key in pkey_get [BZ #22797] Jesse Hathaway (1): getlogin_r: return early when linux sentinel value is set John David Anglin (2): Fix ulps for pow on hppa. The hppa-linux target still requires an executable stack for kernel Joseph Myers (110): Do not use packed structures in soft-fp. Fix m68k bits/fenv.h for no-FPU ColdFire. Add ColdFire math-tests.h. Move some fenv.h override macros to generic math_private.h. Move fenv.h override inline functions to generic math_private.h. Add feholdexcept inline in generic math_private.h. Remove some math_private.h libc_fe* overrides. Remove some math_private.h libc_feholdexcept_setround overrides. Move LDBL_CLASSIFY_COMPAT to its own header. Update syscall-names.list for 4.15. Add MAP_SHARED_VALIDATE from Linux 4.15. Add MAP_SYNC from Linux 4.15. Add elf.h NT_* macros from Linux 4.15 (bug 14890). Add IPV6_FREEBIND from Linux 4.15. Add TCP_FASTOPEN_KEY, TCP_FASTOPEN_NO_COOKIE from Linux 4.15. Only define loff_t for __USE_MISC (bug 14553). Use xmalloc in tst-setcontext-fpscr.c (bug 19668). Correct type of SSIZE_MAX for 32-bit (bug 13575). Move string/testcopy.c to test-driver.c and xmalloc (bug 19667). Fix non-__GNUC__ definitions of __inline and __restrict (bug 17721). Unify and simplify bits/byteswap.h, bits/byteswap-16.h headers (bug 14508, bug 15512, bug 17082, bug 20530). Fix -Os strcoll, wcscoll, build (bug 21313). Fix -Os gnu_dev_* linknamespace, localplt issues (bug 15105, bug 19463). Use MPFR 4.0.1 in build-many-glibcs.py. Define char16_t, char32_t consistently with uint_least16_t, uint_least32_t (bug 17979). Remove unused math/Makefile variable libm-test-incs. Add build infrastructure for narrowing libm functions. Add test infrastructure for narrowing libm functions. Handle narrowing function sNaN test disabling based on argument format. Fix narrowing function tests build for powerpc64le. Add narrowing add functions. Fix -Os feof_unlocked linknamespace, localplt issues (bug 15105, bug 19463). Use libc_hidden_* for fputs (bug 15105). Use libc_hidden_* for __cmsg_nxthdr (bug 15105). Use libc_hidden_* for argz_next, __argz_next (bug 15105). Fix hppa local PLT entries for sigprocmask (bug 18124). Document use of CC and CFLAGS in more detail (bug 20980, bug 21234). Fix -Os ferror_unlocked linknamespace, localplt issues (bug 15105, bug 19463). Fix -Os getc_unlocked linknamespace, localplt issues (bug 15105, bug 19463). Fix -Os putc_unlocked, fputc_unlocked linknamespace, localplt issues (bug 15105, bug 19463). Use libc_hidden_* for tolower, toupper (bug 15105). Use libc_hidden_* for atoi (bug 15105). Fix another -Os strcoll build issue. Fix two more -Os strcoll / wcscoll build failures. Use libc_hidden_* for strtoumax (bug 15105). Fix i386 fenv_private.h float128 for 32-bit --with-fpmath=sse (bug 22902). Fix powerpc ifunc-sel.h build for -Os. Fix s390 -Os iconv build. Remove old-GCC parts of x86 bits/mathinline.h. Remove more old-compilers parts of sysdeps/x86/fpu/bits/mathinline.h. Update i386 libm-test-ulps. Remove sysdeps/x86/fpu/bits/mathinline.h __finite inline. Add SHT_X86_64_UNWIND to elf.h (bug 20079). Add narrowing subtract functions. Fix signed integer overflow in random_r (bug 17343). Remove powerpc, sparc fdim inlines (bug 22987). Use x86_64 backtrace as generic version. Remove unused frame.h header, sigcontextinfo.h macros. Unify umount function implementations (bug 16552). Use Linux 4.16 in build-many-glibcs.py. Make build-many-glibcs.py build GCC for powerpcspe with --enable-obsolete. Update aarch64 bits/hwcap.h, dl-procinfo.c for Linux 4.16 HWCAP_ASIMDFHM. Define XTABS to TAB3 on alpha to match Linux 4.16. Add NT_PPC_PKEY from Linux 4.16 to elf.h. Add PTRACE_SECCOMP_GET_METADATA from Linux 4.16 to sys/ptrace.h. Fix Hurd glibc build with GCC 8. Use GCC 8 in build-many-glibcs.py by default. Remove tilegx port. Ignore absolute symbols in ABI tests. Move math_narrow_eval to separate math-narrow-eval.h. Move math_opt_barrier, math_force_eval to separate math-barriers.h. Move math_check_force_underflow macros to separate math-underflow.h. Do not include math-barriers.h in math_private.h. Add narrowing multiply functions. Update MIPS libm-test-ulps. Add narrowing divide functions. Fix year 2039 bug for localtime with 64-bit time_t (bug 22639). Obsolete nfsservctl. Split test-tgmath3 by function. Make llseek a compat symbol (bug 18471). Fix i686-linux-gnu build with GCC mainline. Remove sysdeps/aarch64/soft-fp directory. Remove sysdeps/alpha/soft-fp directory. Remove sysdeps/sh/soft-fp directory. Remove sysdeps/powerpc/soft-fp directory. Remove sysdeps/sparc/sparc32/soft-fp directory. Remove sysdeps/sparc/sparc64/soft-fp directory. Make powerpc-nofpu __sqrtsf2, __sqrtdf2 compat symbols (bug 18473). Use Linux 4.17 in build-many-glibcs.py. Update kernel version in syscall-names.list to 4.17. Add MAP_FIXED_NOREPLACE from Linux 4.17 to bits/mman.h. Add AArch64 hwcap values from Linux 4.17. Fix ldbl-96 fma (Inf, Inf, finite) (bug 23272). Do not use const attribute for nan functions (bug 23277). Fix strtod overflow detection (bug 23279). Ignore -Wrestrict for one strncat test. Add tests for sign of NaN returned by strtod (bug 23007). Fix powerpc64le build of nan-sign tests (bug 23303). Update MAP_TYPE value for hppa from Linux 4.17. Add MSG_STAT_ANY from Linux 4.17 to bits/msq.h. Add SEM_STAT_ANY from Linux 4.17 to bits/sem.h. Add SHM_STAT_ANY from Linux 4.17 to bits/shm.h. Fix scanf rounding of negative floating-point numbers (bug 23280). Fix bug-strspn1.c, bug-strpbrk1.c build with GCC mainline. Fix tst-cmp.c build with GCC mainline. Fix hardcoded /tmp paths in testing (bug 13888). Remove nptl/sockperf.c. Avoid insecure usage of tmpnam in tests. Use binutils 2.31 branch in build-many-glibcs.py. Update powerpc-nofpu ulps. Leonardo Sandoval (6): x86-64: remove duplicate line on PREFETCH_ONE_SET macro Add missing changelog from previous commit x86-64: Optimize strcmp/wcscmp and strncmp/wcsncmp with AVX2 benchtests: Add --no-diff and --no-header options benchtests: Catch exceptions in input arguments benchtests: improve argument parsing through argparse library Maciej W. Rozycki (6): nptl_db: Remove stale `match_pid' parameter from `iterate_thread_list' elf: Unify symbol address run-time calculation [BZ #19818] elf: Correct absolute (SHN_ABS) symbol run-time calculation [BZ #19818] nisplus: Correct pwent parsing issue and resulting build error [BZ #23266] elf: Accept absolute (SHN_ABS) symbols whose value is zero [BZ #23307] libc-abis: Define ABSOLUTE ABI [BZ #19818][BZ #23307] Mark Wielaard (1): elf.h: Add BPF relocation types. Martin Sebor (1): Document interaction with GCC built-ins in the Customizing Printf Michael Wolf (1): New locale: Lower Sorbian (dsb_DE) [BZ #23208] Mike FABIAN (23): Add missing “reorder-end” in LC_COLLATE of et_EE [BZ #22517] Use “copy "es_BO"” in LC_TIME of es_CU, es_CL, and es_EC Use / instead of - in d_fmt for pt_BR and pt_PT [BZ #17438] Remove --quiet argument when installing locales Update iso14651_t1_common file to ISO14651_2016_TABLE1_en.txt [BZ #14095] Necessary changes after updating the iso14651_t1_common file iso14651_t1_common: <U\([0-9A-F][0-9A-F][0-9A-F][0-9A-F][0-9A-F]\)> → <U000\1> Fixing syntax errors after updating the iso14651_t1_common file Add convenience symbols like <AFTER-A>, <BEFORE-A> to iso14651_t1_common iso14651_t1_common: make the fourth level the codepoint for characters which are ignorable on all 4 levels Add sections for various scripts to the iso14651_t1_common file Collation order of ȥ has changed in new iso14651_t1_common file, adapt test files Collation order of @-. and space has changed in new iso14651_t1_common file, adapt test files Fix posix/bug-regex5.c test case, adapt to iso14651_t1_common upate Fix test cases tst-fnmatch and tst-regexloc for the new iso14651_t1_common file. Improve gen-locales.mk and gen-locale.sh to make test files with @ options work Adapt collation in several locales to the new iso14651_t1_common file Remove the lines from cmn_TW.UTF-8.in which cannot work at the moment. bg_BG locale: Fix a typo in a comment an_ES locale: update some locale data [BZ #22896] Fix tst-strfmon_l test for hr_HR locale Bug 23308: Update to Unicode 11.0.0 Put the correct Unicode version number 11.0.0 into the generated files Patrick McGehearty (1): Improves __ieee754_exp(x) performance by 18-37% when |x| < 1.0397 Patsy Franklin (1): In sem_open.c, pad was not initialized when __HAVE_64B_ATOMICS was Paul Pluzhnikov (3): Fix BZ 20419. A PT_NOTE in a binary could be arbitratily large, so using Fix BZ 22786: integer addition overflow may cause stack buffer overflow Update ulps with "make regen-ulps" on AMD Ryzen 7 1800X. Quentin PAGÈS (1): oc_FR locale: Multiple updates (bug 23140, bug 23422). Rafal Luzynski (13): lt_LT locale: Update abbreviated month names (bug 22932). Greek (el_CY, el_GR) locales: Introduce ab_alt_mon (bug 22937). cs_CZ locale: Add alternative month names (bug 22963). NEWS: Mention the locale data changes (bug 22848, 22937, 22963). gd_GB: Fix typo in abbreviated "May" (bug 23152). gd_GB, hsb_DE, wa_BE: Add alternative month names (bug 23140). csb_PL: Update month translations + add yesstr/nostr (bug 19485). csb_PL: Add alternative month names (bug 23140). ast_ES: Add alternative month names (bug 23140). hy_AM: Add alternative month names (bug 23140). dsb_DE locale: Fix syntax error and add tests (bug 23208). os_RU: Add alternative month names (bug 23140). NEWS: Avoid the words "nominative" and "genitive". Rajalakshmi Srinivasaraghavan (3): powerpc: Add multiarch sqrtf128 for ppc64le ldbl-128ibm-compat: Introduce ieee128 symbols Add long double input for strfmon test Raymond Nicholson (1): manual/startup.texi (Aborting a Program): Remove inappropriate joke. Rical Jasan (9): manual: Fix Texinfo warnings about improper node names. manual: Fix a syntax error. manual: Improve documentation of get_current_dir_name. [BZ #6889] manual: Document missing feature test macros. manual: Update the _ISOC99_SOURCE description. manual: Update _DEFAULT_SOURCE. [BZ #22862] Fix a typo in a comment. Add [BZ #16335] annotation to ChangeLog entry. Add manual documentation for threads.h Richard Braun (1): Hurd: fix port leak in TLS Robert Buj (1): ca_ES locale: Update LC_TIME (bug 22848). Rogerio Alves (1): powerpc64: Always restore TOC on longjmp [BZ #21895] Samuel Thibault (131): hurd: Fix build nscd: don't unconditionally use PTHREAD_RWLOCK_WRITER_NONRECURSIVE_INITIALIZER_NP hurd: Define EXEC_PAGESIZE hurd: Fix build on missing __ptsname_internal function hurd: fix build hurd: Add sysdep-cancel.h Move NPTL-specific code to NPTL-specific header hurd: fix timer_routines.c build hurd: fix gai_misc build hurd: fix timer_routines.c build hurd: do not check Mach and Hurd headers hurd: Add missing includes hurd: Add missing includes hurd: Move mach/param.h to bits/mach/param.h hurd: avoid including hurd/signal.h when not needed hurd: fix header conformity hurd: Add missing include hurd: Avoid using ino64_t and loff_t in headers hurd: Fix inclusion of mach headers in all standards hurd: Make almost all hurd headers includable in all standards Separate out error_t definition hurd: Add futimens support hurd: Fix includability of <hurd/signal.h> in all standards hurd: Add futimesat and utimensat support Add missing start-of-file descriptive comment. hurd: add gscope support hurd: add TLS support hurd: Fix getting signal thread stack layout for fork hurd: Replace threadvars with TLS hurd: Fix link cthread/pthread symbol exposition. hurd: Fix coding style x86_64: Fix build with RTLD_PRIVATE_ERRNO defined to 1 hurd: Add missing include hurd: Fix copyright years hurd: Fix O_NOFOLLOW hurd: Fix O_DIRECTORY | O_NOFOLLOW hurd: Fix boot with statically-linked exec server hurd: Add mlockall support hurd: fix build hurd: Fix build with latest htl hurd: Code style fixes Fix errno values hurd: Fix accessing errno from rtld hurd: Initialize TLS and libpthread before signal thread start Add missing changelog from previous commit hurd: Fix calling __pthread_initialize_minimal in shared case hurd: Regenerate errno.h header hurd: advertise process memory locking option hurd: avoid letting signals go to thread created by timer_create hurd: Add hurd thread library hurd libpthread: add function missing in ABI list hurd: Advertise libpthread hurd: Remove bogus net/if_ppp.h hurd: Bump remaining LGPL2+ htl licences to LGPL 2.1+ hurd: Announce that glibc now builds unpatched hurd: Fix exposition of UTIME_NOW, UTIME_OMIT hurd: Avoid local PLTs in libpthread. hurd: Avoid some PLTs in libc and librt Revert __dirfd PLT avoidance for now hurd: whitelist rtld symbols expected to be overridable hurd: Add __errno_location to overridable ld.so symbols hurd: Update localplt.data hurd: whitelist ld.so PLTs supposed to be avoided by rtld_hidden hurd: Avoid some libc.so PLTs hurd: Avoid more libc.so PLTs hurd: Fix typo hurd: Avoid more libc.so local PLTs hurd: Avoid local PLT in libpthread s390x: Fix hidden aliases hurd: Fix buffer overrun in __if_nametoindex Revert "s390x: Fix hidden aliases" Revert parts of "hurd: Avoid more libc.so local PLTs" hurd: Make __if_nametoindex return ENODEV if ifname is too long hurd: Fix missing trailing NUL in __if_nametoindex hurd: Silence warning hurd: Add missing symbols hurd: fix build hurd: Fix typo hurd: Avoid PLTs for longjmp & siglongjmp hurd: Avoid PLT for dirfd Revert "hurd: Avoid PLTs for longjmp & siglongjmp" hurd: fix conformity test for sys/un.h hurd: Fix spurious installation of headers defining hidden prototypes Fix sched_param conform sys/un.h: Allow sun_ prefix, not only sun_len Revert "Fix sched_param" hurd: Fix mach installed headers test hurd: xfail some structure fields ABI incompatibility with standards hurd: Fix standard compliance of some statvfs fields hurd: Update struct statfs according to struct statvfs hurd: Fix symbols exposition hurd: Avoid exposing all <sched.h> symbols from sys/types.h hurd: fix sigevent's sigev_notify_attributes field type hurd: remove non-standard siginfo symbol hurd: Fix termios.h symbols hurd: Add missing RLIM_SAVED_MAX/CUR hurd: Fix hurd installed headers test Drop fpregset unused symbol exposition Revert "hurd: Fix mach installed headers test" hurd: XFAIL appearance of sched_param and sched_priority from <sys/types.h> hurd: XFAIL tests for signal features not implemented yet hurd xfails: Add missing bug references hurd: Fix shmid_ds's shm_segsz field type hurd: xfail missing abilist for libmachuser and libhurduser hurd: update localplt.data hurd: Avoid PLTs for _hurd_port_locked_get/set hurd: Avoid PLTs for __mach_thread_self and __mach_reply_port hurd: Avoid a PLT reference hurd: Fix htl link failure hurd: avoid PLT ref between sendfile and sendfile64 hurd: Detect 32bit overflow in value returned by lseek hurd: Avoid PLT ref for __pthread_get_cleanup_stack hurd: Avoid missing PLT ref from ld.so requirement hurd: Avoid PLT references to shortcuts hurd: Avoid PLT ref to __mach_msg hurd: Avoid PLT references to syscalls hurd: Whitelist PLT refs which are difficult to avoid hurd: Fix missing __pthread_get_cleanup_stack symbol hurd: Fix reference to _hurd_self_sigstate hurd: Fix "Missing required PLT reference" hurd: fix localplt.data format hurd: Enable thread-safe i386 atomic instructions Fix new file header hurd: Fix installed-headers tests check-execstack: Permit sysdeps to xfail some libs hurd: Fix some ld.so symbol override from libc hurd: Fix some ld.so symbol override from libc hurd: Fix some ld.so symbol override from libc hurd: Fix startup of static binaries linked against libpthread hurd: Add missing ChangeLog entry hurd: Fix exec usage of mach_setup_thread Sean McKean (1): time: Reference CLOCKS_PER_SEC in clock comment [BZ #22735] Siddhesh Poyarekar (18): benchtests: Reallocate buffers for every test run benchtests: Make bench-memcmp print json aarch64: Use the L() macro for labels in memcmp aarch64/strcmp: fix misaligned loop jump target benchtests: Convert strncmp benchmark output to json benchtests: Reallocate buffers for every strncmp implementation benchtests: Don't benchmark 0 length calls for strncmp Add ChangeLog entry for last 3 commits aarch64: Optimized memcmp for medium to large sizes aarch64: Fix branch target to loop16 aarch64: Improve strncmp for mutually misaligned inputs aarch64/strncmp: Unbreak builds with old binutils aarch64/strncmp: Use lsr instead of mov+lsr benchtests: Move iterator declaration into loop header aarch64,falkor: Ignore prefetcher hints for memmove tail aarch64,falkor: Ignore prefetcher tagging for smaller copies aarch64,falkor: Use vector registers for memmove aarch64,falkor: Use vector registers for memcpy Stefan Liebler (9): S390: Regenerate ULPs. Add runtime check if mutex will be elided in tst-mutex8 testcases. S390: Regenerate ULPs. S390: Regenerate ULPs. S390: Fix struct sigaction for 31bit in kernel_sigaction.h. Use volatile global counters in test-tgmath.c. Disable lock elision for mutex pretty printer tests. Fix blocking pthread_join. [BZ #23137] Fix string/tst-xbzero-opt if build with gcc head. Steve Ellcey (2): IFUNC for Cavium ThunderX2 aarch64: Use an ifunc/VDSO to implement gettimeofday in shared glibc. Sylvain Lesage (1): es_BO locale: Change LC_PAPER to en_US (bug 22996). Szabolcs Nagy (5): Remove slow paths from exp Fix documentation build with old makeinfo Use uint32_t sign in single precision math error handling functions aarch64: Remove HWCAP_CPUID from HWCAP_IMPORTANT aarch64: add HWCAP_ATOMICS to HWCAP_IMPORTANT Thomas Schwinge (3): hurd: SOCK_CLOEXEC and SOCK_NONBLOCK for socket hurd: SOCK_CLOEXEC and SOCK_NONBLOCK for socketpair hurd: Implement pipe2 Tulio Magno Quites Machado Filho (14): powerpc: Update pow() ULPs powerpc: Undefine Linux ptrace macros that conflict with __ptrace_request powerpc: Update sin, cos and sincos ULPs Increase robustness of internal dlopen() by using RTLD_NOW [BZ #22766] Replace M_SUF (fabs) with M_FABS Replace M_SUF (M_LN2) with M_MLIT (M_LN2) Replace hidden_def with libm_hidden_def in math powerpc: Fix the compiler type used with C++ when -mabi=ieeelongdouble powerpc: Move around math-related Implies powerpc64le: Fix TFtype in sqrtf128 when using -mabi=ieeelongdouble Move declare_mgen_finite_alias definition Add a generic significand implementation ldbl-128ibm-compat: Create libm-alias-float128.h m68k: Reorganize log1p and significand implementations Valery Timiriliyev (1): New locale: Yakut (Sakha) for Russia (sah_RU) [BZ #22241] Vincent Chen (1): Add Andes nds32 dynamic relocations to elf.h Wilco Dijkstra (20): Remove slow paths from log [AArch64] Use builtins for fpcr/fpsr [AArch64] Fix testsuite error due to fpsr/fscr change Remove slow paths from pow Remove mplog and mpexp [AArch64] Fix include. Use correct includes in benchtests Add support for sqrt asm redirects Rename all __ieee754_sqrt(f/l) calls to sqrt(f/l) Remove all target specific __ieee754_sqrt(f/l) inlines Revert m68k __ieee754_sqrt change Undefine attribute_hidden to fix benchtests sin/cos slow paths: avoid slow paths for small inputs sin/cos slow paths: remove large range reduction sin/cos slow paths: remove slow paths from small range reduction sin/cos slow paths: remove slow paths from huge range reduction sin/cos slow paths: remove unused slowpath functions sin/cos slow paths: refactor duplicated code into dosin sin/cos slow paths: refactor sincos implementation Improve strstr performance Zack Weinberg (23): Remove some unnecessary redefinitions of std symbols. Remove getc and putc macros from the public stdio.h. Don't install libio.h or _G_config.h. Post-cleanup 1: move libio.h back out of bits/. Post-cleanup 2: minimize _G_config.h. [BZ #22830] malloc_stats: restore cancellation for stderr correctly. [BZ #19239] Don't include sys/sysmacros.h from sys/types.h. Remove vestiges of external build support from libio headers. Mechanically remove _IO_ name aliases for types and constants. Remove legacy configuration knobs from libio. Remove _IO_file_flags define. Remove miscellaneous debris from libio. alpha/clone.S: Invoke .set noat/.set at around explicit uses of $at Don't include math.h/math_private.h in math_ldbl_opt.h. nldbl-compat.c: Include math.h before nldbl-compat.h. [BZ 1190] Make EOF sticky in stdio. Make sysdeps/generic/internal-signals.h less stubby. NEWS: Reindent and copyedit Avoid cancellable I/O primitives in ld.so. Disallow use of DES encryption functions in new programs. manual: Reorganize crypt.texi. manual: Revise crypt.texi. New configure option --disable-crypt. Zong Li (1): Change URL of gcc's tarball ----------------------------------------------------------------------- (In reply to Carlos O'Donell from comment #35) > As a temprary measure I have committed the deinterleaving of upper and lower > cases in iso14651_t1_common for glibc 2.28 to fix the surprises caused to > en_US.UTF-8 users who do not want to have [a-z] match A-Y. > > This fixes the regression for 2.28, but doesn't fix this issue. There is a user report [1] that [a-z] range matching is broken for sv_SE, and this is expected, the commit 159738548130d5ac4fe6178977e940ed5f8cfdc4 by Mike Fabian in upstream harmonized the collation with CLDR but changed CEO such that [a-z] no longer works as expected. This will be fixed by a full fix to this issue. [1] https://bugzilla.redhat.com/show_bug.cgi?id=1631472 *** Bug 23447 has been marked as a duplicate of this bug. *** (In reply to Carlos O'Donell from comment #35) > As a temprary measure I have committed the deinterleaving of upper and lower > cases in iso14651_t1_common for glibc 2.28 to fix the surprises caused to > en_US.UTF-8 users who do not want to have [a-z] match A-Y. > > This fixes the regression for 2.28, but doesn't fix this issue. There is a user report [1] that shows that the cyrillic ranges are also affected by the iso14651_t1_common update. The deinterleaving changes only fix the latin ranges. [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=926906 (In reply to Aurelien Jarno from comment #40) > (In reply to Carlos O'Donell from comment #35) > > As a temprary measure I have committed the deinterleaving of upper and lower > > cases in iso14651_t1_common for glibc 2.28 to fix the surprises caused to > > en_US.UTF-8 users who do not want to have [a-z] match A-Y. > > > > This fixes the regression for 2.28, but doesn't fix this issue. > > There is a user report [1] that shows that the cyrillic ranges are also > affected by the iso14651_t1_common update. The deinterleaving changes only > fix the latin ranges. > > [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=926906 This user is expecting the range to be made up of collation ordering, and that expectation is not valid. The range in any non-POSIX/C locale is undefined. Therefore the bug you reference is not a bug, but it's still difficult for users to use ranges without problems, and that makes them relatively useless and we'd like to fix that. The plan is to fix this with rational ranges that use UTF-8 code-point ordering for all ranges. The deinterleaving for LATIN is consciously to fix only the ASCII ranges and fix POSIC/C ranges. All other ranges are undefined. If we deinterlace non-LATIN ranges we'd have to duplicate all the data into the individual locales and list them in collation order (so collation order matches collation element ordering). Such a change would be quite drastic, and still not solve the problem of having collation changes change range expressions. It also wouldn't solve the broader problem that everyone still expects [a-z] to work all the time (code-point ordering). I would close your debian bug as an unsupported configuration for ranges, but point out that we are trying to make this better. .macro VOCAB name, cfa, thread .header "\name", \cfa \cfa: nop jal x25, do_voc .quad \thread .fill 63, 8, 0 .quad vlink .set vlink, \cfa .endm the above macro is failing for the same reason the .header macro was failing but while there was a work around for the .header macro the same work around does not work here for some reason that is utterly beyond my understanding. vocabs.s:123: Error: redefined symbol cannot be used on reloc line 123 invokes the above macro and that is failing on the .set vlink, \cfa. this is "it just works (tm)" in every other version of the gnu assembler I have ever used and this literally kills my project dead in the water. Why does the riscv assembler not allow this? it is a horrendous/fatal inconsistency! |