[PATCH #14090] md5_process_block() produces incorrect result withlarge block sizes

[Andreas Jaeger <aj at suse dot com>]

md5_process_block 
>From the bug report:
"crypt/md5.c: md5_process_block() (lines 319..321) contains this code for total
length handling. With large block sizes (ctx->total[0] + len) can exceed 8 GB,
thus the total[1] needs to be incremented more than by one:

------------------------------------------------------------------------------
  /* First increment the byte count.  RFC 1321 specifies the possible
     length of the file up to 2^64 bits.  Here we only compute the
     number of bytes.  Do a double word increment.  */
  ctx->total[0] += len;
  if (ctx->total[0] < len)
    ++ctx->total[1];

------------------------------------------------------------------------------

This bug affects various copies of the same code located in gcc, sources and
gnulib repositories, and probably others, though it is rather hard to trigger
it: md5_process_block() is usually never fed with many gigabytes at once."

Paul has fixed this in gnulib and proposes the appended patch.
Tested on Linux/x86-64, the test supplied in the bug report succeeds
now. 

Btw. adding the test case is not feasible IMO, it links against libcrypto from openssl 
and runs very long - and allocates a buffer of 16 GB (which is not used completely but still...)

Ok to commit?

Andreas

2012-05-19 Paul Eggert <eggert@cs.ucla.edu>
	
	[BZ #14090]
	* crypt/md5.c (md5_process_block):Don't assume the buffer
	length is less than 2**32.

diff --git a/crypt/md5.c b/crypt/md5.c
index 292bee1..3d2e79b 100644
--- a/crypt/md5.c
+++ b/crypt/md5.c
@@ -1,7 +1,6 @@
 /* Functions to compute MD5 message digest of files or memory blocks.
    according to the definition of MD5 in RFC 1321 from April 1992.
-   Copyright (C) 1995,1996,1997,1999,2000,2001,2005,2011
-	Free Software Foundation, Inc.
+   Copyright (C) 1995-2012 Free Software Foundation, Inc.
    This file is part of the GNU C Library.
 
    The GNU C Library is free software; you can redistribute it and/or
@@ -312,13 +311,13 @@ md5_process_block (buffer, len, ctx)
   md5_uint32 B = ctx->B;
   md5_uint32 C = ctx->C;
   md5_uint32 D = ctx->D;
+  md5_uint32 lolen = len;
 
   /* First increment the byte count.  RFC 1321 specifies the possible
      length of the file up to 2^64 bits.  Here we only compute the
      number of bytes.  Do a double word increment.  */
-  ctx->total[0] += len;
-  if (ctx->total[0] < len)
-    ++ctx->total[1];
+  ctx->total[0] += lolen;
+  ctx->total[1] += (len >> 31 >> 1) + (ctx->total[0] < lolen);
 
   /* Process all bytes in the buffer with 64 bytes in each round of
      the loop.  */

-- 
 Andreas Jaeger aj@{suse.com,opensuse.org} Twitter/Identica: jaegerandi
  SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
   GF: Jeff Hawn,Jennifer Guild,Felix Imendörffer,HRB16746 (AG Nürnberg)
    GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126