make 3.82 considered harmful

[Roland McGrath <roland at hack dot frob dot com>]

The recent build issues are more fallout of the extremely ill-advised
change to the 20-year-old semantics of GNU make's pattern rules that was
made in version 3.82 (despite my strenuous objections).  It's described
like this in make's NEWS:

* WARNING: Backward-incompatibility!
  The pattern-specific variables and pattern rules are now applied in the
  shortest stem first order instead of the definition order (variables
  and rules with the same stem length are still applied in the definition
  order). This produces the usually-desired behavior where more specific
  patterns are preferred. To detect this feature search for 'shortest-stem'
  in the .FEATURES special variable.

(In fact, 3.82 has seven NEWS items marked with the same scary line,
"WARNING: Backward-incompatibility!"  This alone is a clear indicator that
this was a thoroughly ill-conceived release.)

This change breaks the fundamental design of our build system, in
particular sysd-rules.  It's crucial that the rules there be applied in an
exact order that we choose.  With the 3.82 change in these long-established
semantics, it's now rather hard to figure out which rule will be chosen.

Most libc hackers are still using 3.81, but Adhemerval's recent change to
sysdeps/generic/symbol-hacks.h was tested (and works) only with 3.82.  This
has uncovered the fact that somehow we changed things at some point so that
at least the rtld-* rules in sysd-rules are only working right under 3.82.

We now have various source files called rtld-*.[cS] in places in the
sysdeps/ tree.  rtld-memset.os is built from sysdeps/x86_64/rtld-memset.c
but this is now happening (under 3.81) with the wrong rule, so it does not
include $(rtld-CPPFLAGS).  This has been happening for a long time, but we
never noticed before because it didn't happen to matter in practice.
Adhemerval's recent change makes it matter, but since he was using 3.82 he
didn't notice.

I used this temporary hack to figure out which rule was being chosen:

--- a/Makerules
+++ b/Makerules
@@ -238,6 +238,7 @@ $(common-objpfx)sysd-rules: $(common-objpfx)config.make $(..)Makerules \
 			    $(sysd-rules-force)
 	-@rm -f $@T
 	(echo 'sysd-rules-sysdirs := $(config-sysdirs)';		      \
+	 n=0; \
 	 for dir in $(config-sysdirs); do				      \
 	   case "$$dir" in						      \
 	   /*) ;;							      \
@@ -253,6 +254,8 @@ $(common-objpfx)sysd-rules: $(common-objpfx)config.make $(..)Makerules \
 	       v=$${t%%%}; [ x"$$v" = x ] || v="\$$($${v}CPPFLAGS)";	      \
 	       for s in $$asm .c; do					      \
 		 echo "\$$(objpfx)$$t$$o: $$dir/$$d$$s \$$(before-compile)";  \
+		 n=`expr $$n + 1`; \
+		 echo "	echo rule $$n"; \
 		 echo "	\$$(compile-command$$s) $$v";			      \
 	       done;							      \
 	     done;							      \

That told me rtld-memset.os was built by rule 1462:

$(objpfx)%.os: $(..)sysdeps/x86_64/%.c $(before-compile)
	echo rule 1462
	$(compile-command.c) 

The one we want is a slightly later one:

$(objpfx)rtld-%.os: $(..)sysdeps/x86_64/rtld-%.c $(before-compile)
	echo rule 1464
	$(compile-command.c) $(rtld-CPPFLAGS)

So the sysd-rules rule in Makerules needs to sort sysd-rules-patterns by
length or something like that.  I'll look into it further after lunch.


Thanks,
Roland