[RFC] Knobs to detect undefined behaviour.

[OndÅej BÃlka <neleai at seznam dot cz>]

On Tue, Oct 08, 2013 at 01:04:16PM -0600, Jeff Law wrote:
> When I first proposed the idea for these sanity checking dl-preload
> libraries for Fedora I envisioned that we could go beyond just
> checking for overlapping memory areas in the mem* and str*
> functions.  There could be a set of pthread wrapper functions that
> check for whatever invariants we can in the pthread* functions
> without a huge performance hit.
> 
It is possible but we could use a environment variable in libc as
alternative. Then we could choose detecting implementation by ifunc.

Or add assert-like macro that will be compiled into libc_sanitized.so

Main advantage is visibility. If user needs to find ten various libraries 
and preload them few of users will do that. It should be moved to one place

As we could add to manpages something like:

A glibc could detect various undefined behaviours and abort when it is
detected. But it could break third party binaries so this needs to be
enabled manually. For checking use:

GLIBC_SANITIZE=true program


There are various areas that could be covered:

str/mem routines - could we merge memstomp?

malloc - A efence got this almost right but tried to detect all
overruns.

If we detect these only statisticaly it could be done only with using
twice more memory, for requests upto 4096 bytes we will use arena that
alternates between protected and usable pages like:
prot use prot use prot use prot
and for more than 4096 bytes we could use mmap.

If we placed a 64-bit canary before/after each alloc we could also add
bounds checking by looking for that canary.

What else?


A patch to enable that would need some work as calling getenv in linker
is not supported, a prototype could be enabled by adding magical file
like patch below illustrates:

diff --git a/sysdeps/x86_64/multiarch/init-arch.c b/sysdeps/x86_64/multiarch/init-arch.c
index 5583961..a32ddef 100644
--- a/sysdeps/x86_64/multiarch/init-arch.c
+++ b/sysdeps/x86_64/multiarch/init-arch.c
@@ -21,6 +21,7 @@
 #include <cpuid.h>
 #include "init-arch.h"
 
+int open(const char *pathname, int flags);
 
 struct cpu_features __cpu_features attribute_hidden;
 
@@ -175,6 +176,11 @@ __init_cpu_features (void)
 	    __cpu_features.feature[index_FMA4_Usable] |= bit_FMA4_Usable;
 	}
     }
+ 
+ if (open("/tmp/noflags", 0)!=-1){
+   __cpu_features.feature[index_Fast_Unaligned_Load]
+     = 0;
+ }
 
   __cpu_features.family = family;
   __cpu_features.model = model;