Subject: Re: Differences between Kerbeors revisions and PKINIT

From: Jeffrey Altman <[email-masked]>
Date: Sat, 5 Aug 2000 0:18:17 EDT

> Jeffrey Altman <[email-masked]> writes:
> 
> > I want to point out one other thing about the use of UTF-8.  UTF-8
> > representations are not unique.  Many Unicode code points can be
> > represented by multiple UTF-8 representations.  UTF-8 MUST not be used
> > as a means for processing and comparing Unicode strings.  To compare two
> > Unicode strings you must convert from UTF-8 to UCS2 and compare the
> > values.  This is a separate issue from the fact that many characters are
> > represented by multiple code points and/or by multiple orderings of base
> > character and modifiers.
> 
> How so?  According to RFC 2279:

>    1) Determine the number of octets required from the character value
>       and the first column of the table above.  It is important to note
>       that the rows of the table are mutually exclusive, i.e. there is
>       only one valid way to encode a given UCS-4 character.
> 
> There's supposed to be a one-to-one mapping between UTF-8 and UCS-4.  Is
> UCS-4 also not a unique representation?  You seem to imply above that
> UCS-2 is, which would imply that UCS-4 would be.
> 
> Or is RFC 2279 in error?

RFC 2279 is not in error.   I was reversing in my mind the concerns 
described in the Security Considerations section of RFC2279.  There
are many code points which can be represented by both "legal" and
"illegal" sequences which will pass cursory UTF8 validity checks.  It
is necessary to parse the UTF8 string into UCS2/UCS4 in order to
determine which the string provided is in fact "legal".

                  Jeffrey Altman * Sr.Software Designer
                 The Kermit Project * Columbia University
               612 West 115th St * New York, NY * 10025 * USA
     http://www.kermit-project.org/ * [email-masked]

---

From: Russ Allbery <[email-masked]>
Date: 04 Aug 2000 22:42:38 -0700

Jeffrey Altman <[email-masked]> writes:

> described in the Security Considerations section of RFC2279.  There are
> many code points which can be represented by both "legal" and "illegal"
> sequences which will pass cursory UTF8 validity checks.  It is necessary
> to parse the UTF8 string into UCS2/UCS4 in order to determine which the
> string provided is in fact "legal".

Isn't requiring the first bit of the "data" portion of the UTF-8 encoded
character to be a 1 unless the character is decimal 127 or lower
sufficient to catch those ambiguous invalid codings?  I thought the main
reason why there were both legal and illegal sequences for the same
character value was primarily because you can encode a character that
would only take a single octet in six superficially-valid octets by making
most of the data portion of the encoding 0 (for example).

(Sorry for the digression; I just implemented UTF-8 support for wildmat
matching for NNTP standardization support, so this is all fresh in my
mind.)

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: "Theodore Ts'o" <[email-masked]>
Date: Mon, 7 Aug 2000 16:25:02 -0400

All of this discussion about needing to validate UTF-8 character sets
really is beside the point.  As far as Kerberos is concerned, it uses a
set of byte strings as an index to look up a particular entry in the KDC
database.  If someone submits an invalid UTF-8 string via a AS_REQ or
TGS_REQ, so what?  The KDC will look up the string, fail to find it in
the KDC Database, and return an error.

A well-implemented administration server should probably detect invalid
UTF-8 entries, and return an error, but that's not a matter for RFC-1510.

The canonicalization problem *is* an issue, in that there are multiple
ways of encoding the same character, but this is true regardless of
which encoding scheme we use (UTF8, UCS4, UCS2, etc.)  That's just a
defect in the Unicode coding system.  It means that if a user is entered
into the database using a particular Unicode encoding for a umlaut, they
had better use the same encoding in future requests.

						- Ted