Ticket History #626: shsUpdate, shsFinalize can trash core or produce bad results.

From mdw@quince.ifs.umich.edu Tue Sep 1 18:58:08 1998
Received: from MIT.EDU (SOUTH-STATION-ANNEX.MIT.EDU [18.72.1.2]) by rt-11.MIT.EDU (8.7.5/8.7.3) with SMTP id SAA23050 for <bugs@RT-11.MIT.EDU>; Tue, 1 Sep 1998 18:58:07 -0400
Received: from quince.ifs.umich.edu by MIT.EDU with SMTP
id AA03440; Tue, 1 Sep 98 18:58:02 EDT
Received: (from mdw@localhost) by quince.ifs.umich.edu (8.6.13/8.6.12) id SAA26461; Tue, 1 Sep 1998 18:57:54 -0400
Message-Id: <199809012257.SAA26461@quince.ifs.umich.edu>
Date: Tue, 1 Sep 1998 18:57:54 -0400
From: Marcus D Watts <mdw@quince.ifs.umich.edu>
Reply-To: mdw@quince.ifs.umich.edu
To: krb5-bugs@MIT.EDU
Subject: shsUpdate, shsFinalize can trash core or produce bad results.
X-Send-Pr-Version: 3.99

University of Michigan

System: SunOS pepper-pot 5.6 Generic_105181-03 sun4u sparc SUNW,Ultra-5_10
Architecture: sun4
If the first call to shsUpdate has left some data, and the 2nd call
to shsUpdate passes too little data to make up a complete block,
shsTransform may still be called in certain circumstances.

If the sum of counts to previous calls to shsUpdate is not a
multiple of 4, the first 1-3 bytes of data supplied is or'd into
the wrong part of a word.

If shsUpdate is given a very small count (1-6 bytes), it can under
certain conditions fetch and mis-use an extra word of data.

If the byte count of the total amount of data supplied is 60-63 mod
64, shsFinalize will zero an extra word of storage immediately past
the end of the SHS_INFO structure passed to it.

See
http://web.mit.edu/afs/umich.edu/user/m/d/mdw/src/shs/
for more information including test programs.
The file /afs/umich.edu/user/m/d/mdw/src/shs/t_shs3.c contains
a program will exercise the bugs.

--- shs_old.c Tue Sep 1 18:13:26 1998
+++ shs_n.c Tue Sep 1 18:13:25 1998
@@ -272,25 +272,21 @@
/* Handle any leading odd-sized chunks */
if (dataCount) {
lp = shsInfo->data + dataCount / 4;
- canfill = (count >= dataCount);
dataCount = SHS_DATASIZE - dataCount;
+ canfill = (count >= dataCount);

if (dataCount % 4) {
/* Fill out a full 32 bit word first if needed -- this
is not very efficient (computed shift amount),
but it shouldn't happen often. */
while (dataCount % 4 && count > 0) {
- *lp |= (LONG) *buffer++ << ((3 - dataCount++ % 4) * 8);
+ *lp |= (LONG) *buffer++ << ((--dataCount % 4) * 8);
count--;
}
lp++;
}
while (lp < shsInfo->data + 16) {
- *lp = (LONG) *buffer++ << 24;
- *lp |= (LONG) *buffer++ << 16;
- *lp |= (LONG) *buffer++ << 8;
- *lp++ |= (LONG) *buffer++;
- if ((count -= 4) < 4 && lp < shsInfo->data + 16) {
+ if (count < 4) {
*lp = 0;
switch (count % 4) {
case 3:
@@ -300,9 +296,14 @@
case 1:
*lp |= (LONG) buffer[0] << 24;
}
- break;
count = 0;
+ break;
}
+ *lp = (LONG) *buffer++ << 24;
+ *lp |= (LONG) *buffer++ << 16;
+ *lp |= (LONG) *buffer++ << 8;
+ *lp++ |= (LONG) *buffer++;
+ count -= 4;
}
if (canfill) {
SHSTransform(shsInfo->digest, shsInfo->data);
@@ -378,7 +379,8 @@

if (lp > shsInfo->data + 14) {
/* Pad out to 64 bytes if not enough room for length words */
- *lp = 0;
+ if (count < 60)
+ *lp = 0;
SHSTransform(shsInfo->digest, shsInfo->data);
lp = shsInfo->data;
}

State-Changed-From-To: open-feedback
State-Changed-By: raeburn
State-Changed-When: Sat Jul 14 00:58:50 2001
State-Changed-Why:

I've applied your patch. May I include your t_shs3.c in our source
tree too as a test case to run?

Responsible-Changed-From-To: krb5-unassigned->raeburn
Responsible-Changed-By: raeburn
Responsible-Changed-When: Sat Jul 14 01:04:57 2001
Responsible-Changed-Why:

I'll hold on to it.