Subject: Re: [precis] RFC 8264 / 8265 Order of rules

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 4 May 2018 12:40:46 -0600

On 3/14/18 10:04 AM, Paul Crovella wrote:
> Followup question on
> https://www.ietf.org/mail-archive/web/precis/current/msg01445.html
> 
>> implementations should follow the order of rules in Section 7 of RFC 8264.
> 
> Should string class validation then be moved from the preparation step
> of all profiles to the end of enforcement? I don't know whether
> there'd be a practical effect on profiles using the FreeformClass
> string class (OpaqueString and RFC 8266's Nickname), or if there's the
> potential to be, but it'd be nice to know where to do things.

The intent in my head when working on RFC 8265 & RFC 8266 was to define
the set of rules for a profile and then in each subsection (preparation,
enforcement, comparison) specify each rule to be applied for that
operation, without having them be additive (i.e., don't say in the
enforcement operation than you first do everything in preparation, then
some things in addition). Clearly, this did *not* get translated into
the final text because I failed in my responsibility as a spec author.

I will send proposed text for errata to this list sometime soon.

Peter

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 9 May 2018 15:57:30 -0600

> I will send proposed text for errata to this list sometime soon.

I have in mind something like the following for Section 3.3 of RFC 8265.

###

OLD

3.3.3.  Enforcement

   An entity that performs enforcement according to this profile MUST
   prepare an input string as described in Section 3.3.2 and MUST also
   apply the following rules specified in Section 3.3.1 in the order
   shown:

   1.  Case Mapping Rule

   2.  Normalization Rule

   3.  Directionality Rule

   After all of the foregoing rules have been enforced, the entity MUST
   ensure that the username is not zero bytes in length (this is done
   after enforcing the rules to prevent applications from mistakenly
   omitting a username entirely, because when internationalized strings
   are accepted, a non-empty sequence of characters can result in a
   zero-length username after canonicalization).

   The result of the foregoing operations is an output string that
   conforms to the UsernameCaseMapped profile.  Until an implementation
   produces such an output string, it MUST NOT treat the string as
   conforming (in particular, it MUST NOT assume that an input string is
   conforming before the enforcement operation has been completed).

3.3.4.  Comparison

   An entity that performs comparison of two strings according to this
   profile MUST prepare each string as specified in Section 3.3.2 and
   then MUST enforce the rules specified in Section 3.3.3.  The two
   strings are to be considered equivalent if and only if they are an
   exact octet-for-octet match (sometimes called "bit-string identity").

   Until an implementation determines whether two strings are to be
   considered equivalent, it MUST NOT treat them as equivalent (in
   particular, it MUST NOT assume that two input strings are equivalent
   before the comparison operation has been completed).

NEW

3.3.3.  Enforcement

   An entity that performs enforcement according to this profile MUST
   apply the following rules specified in Section 3.3.1 in the order
   shown:

   1.  Width Mapping Rule

   2.  Case Mapping Rule

   3.  Normalization Rule

   4.  Directionality Rule

   After all of the foregoing rules have been enforced, the entity MUST
   ensure that the username is not zero bytes in length (this is done
   after enforcing the rules to prevent applications from mistakenly
   omitting a username entirely, because when internationalized strings
   are accepted, a non-empty sequence of characters can result in a
   zero-length username after canonicalization).

   The result of the foregoing operations is an output string that
   conforms to the UsernameCaseMapped profile.  Until an implementation
   produces such an output string, it MUST NOT treat the string as
   conforming (in particular, it MUST NOT assume that an input string is
   conforming before the enforcement operation has been completed).

3.3.4.  Comparison

   An entity that performs comparison of two strings according to this
   profile MUST apply the following rules specified in Section 3.3.1
   in the order shown:

   1.  Width Mapping Rule

   2.  Case Mapping Rule

   3.  Normalization Rule

   4.  Directionality Rule

   The two strings are to be considered equivalent if and only if they
   are an exact octet-for-octet match (sometimes called "bit-string
   identity").

   Until an implementation determines whether two strings are to be
   considered equivalent, it MUST NOT treat them as equivalent (in
   particular, it MUST NOT assume that two input strings are equivalent
   before the comparison operation has been completed).

###

Would this clarify string handling enough to be useful?

/psa

---

From: William Fisher <[email-masked]>
Date: Fri, 11 May 2018 12:04:29 -0700

Hi Peter,

My thoughts on your proposed text is below.

I, for one, thought you did a good job with such a complex topic.

-Bill

>     zero-length username after canonicalization).

The output string still needs to be validated by the base "IdentifierClass"
before it can be said to conform to the UsernameCaseMapped profile.

INSERT:

     Finally, the entity MUST ensure that the username consists only of
Unicode code points that are explicitly allowed by the PRECIS
IdentifierClass defined in Section 4.2 of [RFC8264].

>     conforms to the UsernameCaseMapped profile.

In RFC 8265, the opening sentence of section "3.3.2 Preparation" implies
preparation is done before enforcement. This can lead to confusion:

OLD:

    An entity that prepares an input string for subsequent enforcement
    according to this profile MUST proceed as follows (applying the steps
    in the order shown).

PROPOSED:

    An entity that performs preparation according to this profile MUST apply
the following steps in the order shown.

(I am wary of the separate preparation step as something that an
application developer might use. The notion mentioned in RFC 8264 (section
3) that a client might use the "preparation" step before handing the
protocol string to an authoritative server which does the "enforcement"
step is problematic for the Username profiles. It has been shown that
preparation by itself can reject strings that enforcement would have
accepted.  Example: '\u212b' under the UsernameCasePreserved profile.)

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 16 May 2018 11:17:43 -0600

> IdentifierClass defined in Section 4.2 of [RFC8264].

Yes indeed.

> preparation is done before enforcement. 

Right - I'm proposing that we remove that text by explicitly specifiying
for each operation what the steps are (and not saying that they are
additive, i.e., stop saying that enforcement is the steps defined in
preparation plus some other steps).

> This can lead to confusion:
> 
> OLD:
> 

Ah, I see what you mean. Yes, that's better.

> accepted.  Example: '\u212b' under the UsernameCasePreserved profile.)

The original idea was that some resource-constrained entities might not
have the capacity to do things like Unicode normalization, in part
because they might not have the storage capacity to hold the Unicode
tables and in part because they might lack the computing resources to
perform normalization operations (back then we were thinking of things
like in-browser messaging clients, but these days we might think of
things like IoT devices). As a result, we were thinking that such
entities might simply restrict themselves to a safe subset of code
points (not necessarily just the ASCII range) and consider that "good
enough". Naturally, other steps might be needed depending on the string
class or profile (e.g., the width mapping rule for usernames).

Peter