Subject: kcrypto: checksums for DES algorithms

From: Ken Raeburn <[email-masked]>
Date: Fri, 08 Oct 2004 19:01:21 -0400

This should've been reported here, but at the time it looked like an
obvious and minor correction to a small technical inconsistency, and I
don't think it got mentioned....

In June, it was pointed out to me that the description of the checksum
calculation for DES-based encryption systems in crypto-07 is
inconsistent.

The text (section 6.2, page 19) seems to indicate that the checksum
(md5, md4, or crc) is computed before the padding bytes are added when
encrypting, and after decrypting, the padding bytes are ignored for
verifying the checksum.

The tables describing each encryption system (sections 6.2.1-6.2.3,
pages 22-25) clearly show that the padding bytes are included in the
checksum calculation.

The reference implementation of RFC 1510 from MIT, and the Heimdal
implementation from KTH, and any others compatible with us for DES
encryption, have always done the latter.  However, RFC 1510 described
only the former, although it's never indicated how you might figure
out how many bytes at the end are actually padding.  (Trying the eight
different possible padding lengths until one works is one possibility.
Assuming the content is a DER-encoded ASN.1 object and parsing it far
enough to extract the encoded outer length is another, though as a
general technique it is incompatible with kcrypto applications like
draft-ietf-krb-wg-gssapi-cfx-07.txt that don't encrypt ASN.1 objects.)

I believe this has been regarded as just one more error in RFC 1510's
reflection of reality by those of us working on the crypto
specification.

I sent email to Russ, with copies to the WG chairs, suggesting a fix
to the text to align it with the tables and all DES implementations I
knew of:

   One generates a random confounder of one block, placing it in
   'confounder'; zeroes out the 'checksum' field (of length
   appropriate to exactly hold the checksum to be computed); adds the
   necessary padding; calculates the appropriate checksum over the
   whole sequence, placing the result in 'checksum'; then encrypts
   using the specified encryption type and the appropriate key.

I offered to produce a new draft, but Russ said he'd take the change
to the IESG for approval for incorporation into the final document.

Ken

P.S.  If I recall correctly, this leaves us with the final RFC being
different from draft -07 in two ways, the consistency fix indicated
above, and the PRF change in section 5.3 that Sam recommended in 27
July, using the string "prf" I suggested on 30 August.  Both of these
have already been discussed with Russ.