Subject: [Lager] Stephen Farrell's Discuss on draft-ietf-lager-specification-11: (with DISCUSS and COMMENT)

From: "Stephen Farrell" <[email-masked]>
Date: Thu, 21 Apr 2016 03:24:01 -0700

Stephen Farrell has entered the following ballot position for
draft-ietf-lager-specification-11: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-lager-specification/

----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

This spec is tackling a hard problem, (machines doing what
humans do to classify identifiers), and as a result there are
some tricky bits here. That inherently hard problem has thrown
up a few things I'd like to discuss (though the 1st one is
easy:-)...

(1) section 5: this says code points MUST be 4 hex digits.
What is s/w supposed to do if it sees only 2 hex digits?
Should it ignore the range or char element or fail to process
the entire LGR document? I think the same issue applies to
other uses of 2119 language as well, (e.g. "MUST be treated as
an error at the end of p19), so I'd recommend you state some
kind of general rule if you can.

(2) 5.2: when and not-when etc seem to me to allow for
infinitely baroque representations of useless things like: 

	<char cp="200D" when="cpisnot200D" />
	<classs "cpis200D">
		200D
	</class>
	<rule name="cpisnot200D">
		<start/>
			<complement><class by-ref="cpis200D"/></complement>
		<end/>
	</rule>

What is parsing s/w supposed to do with structures like that?
For example, how would you handle the likes of this or more
convoluted but equivalent structures which could be delivered
by accident or deliberately?  I think the response to this
discuss point needs to either be a) all such constructs are
automatically detectable and here's why, or else b) here's how
s/w can handle that (without crashing or looping forever).
Note that I don't think that the "MUST be rejected" at the end
of 6.1 provides an answer to this point.  (But if you do,
please argue that.)

(3) 6.3.4: While recursion is said to be disallowed, the "for
which the complete definition has not been seen" is pretty odd
for an XML specification, as it means that you need a full
ordering for the elements in the document (or at least within
the <rules> element).  That means if some editor decodes from
disk and then encodes to disk, you need to be sure that the
order is preserved or else you break the "has been seen"
constraint. (And if you do that, then you're allowing rules to
mutually refer to one another, which brings us back to discuss
point 2.) 7.4 maybe has a similar issue. I think for this you
could simply state up front that these XML documents MUST NOT
be re-ordered during editing. (Or else add some kind of
attribute to help with ordering which seems ickky.)

(4) section 12: I don't think this is at all sufficient.
Missing aspects include: Imprecise LGRs could result in
registration of identifiers that are unexpected in many other
protocols, leading to new vulnerabilities; LGRs could be
deliberately manipulated so as to create such imprecision, and
if I could feed one such to a registry (e.g. via some nice
friendly looking git repo) then I could exploit the vuln later
for fun and profit - that seems to call for some
interoperable form of data integrity and origin
authentication (is the lager WG doing that?) and lastly (for
now), the XML language defined here is very flexible as noted
earlier - I would expect there to be many implementation bugs
in new code that attempts to parse this language. So I think
the security considerations needs to be re-done really.

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

- 4.3.6: my bet: validity-end is a mistake.

- 4.3.8: odd that the examples have no URLs

- 4.3 generally: I bet that these objects will evolve and will
be stored in some VCS. I'm surprised so that there's no
meta-data element to represent information about the version
of say the previous instance of the object. If there were,
then it would then be possible to establish a hash-chain which
I'd have thought would be useful in disputes.

- 5.3.1: "normally not only symmetric, but also transitive" -
isn't that hugely ambiguous?  What is the programmer supposed
to do? The answer is "nothing" I think so I'd just recommend
to delete that paragraph and say that only the explicitly
stated rules apply.

- 6.2.4: "repertoire" - please define that term or reference a
definition. I don't think it's a commonly understood technical
term, at least for implementers.  

- 8.2: What "suitable optimisations" do you mean? It seems
cruel to the implementer to say that but not even have a
reference.

- 8.5: I think you could have explained this more clearly - the
2nd para there is not really useful as it's too opaque.

- Appendix C: Another example of BNF being impressively
more terse than XML :-)

---

From: Alexey Melnikov <[email-masked]>
Date: Sat, 23 Apr 2016 12:49:51 +0100

Hi Stephen,
I think I will need help from the WG on more subtantial issues, but I
will quickly reply to a couple of your DISCUSS points (WG should feel
free to correct me if I am wrong):

> draft-ietf-lager-specification-11: Discuss

 [snip]

> the entire LGR document?

I don't mind clarifying this, but in general it means that the whole
document is syntactically invalid and thus must be ignored.

I don't think it is very common for IETF documents to specify error
handling when a MUST is violated, but of course it is not prohibited.

> I think the same issue applies to

 [snip]

> attribute to help with ordering which seems ickky.)

Correct me if I am wrong, but I haven't seen any XML generator that
reorders XML elements, unless they understand XML schema for a
particular XML document. So by default order of XML elements withing
parent elements is fixed, so I don't think this is an issue.

Best Regards,
Alexey

---

From: Stephen Farrell <[email-masked]>
Date: Sat, 23 Apr 2016 12:58:07 +0100

> document is syntactically invalid and thus must be ignored. 

That's a fine answer.

> handling when a MUST is violated, but of course it is not prohibited.

Well, in this case the error might only affect checking some labels
and not others, so I guess folks may be tempted to use the rest of
the LGR if the spec doesn't say to fail to load the LGR on such errors.

> parent elements is fixed, so I don't think this is an issue.

I'm happy to be corrected, but this was a big (and painful) deal
for xmldsig, so I'm not sure why it'd not be an equal PITA here;-)

Cheers,
S.

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Sat, 23 Apr 2016 10:54:10 -0700

> for xmldsig, so I'm not sure why it'd not be an equal PITA here;-)

There are many XML schema where the order of elements cannot be 
rearranged without changing the content of what the document describes. 
Anything where the XML elements correspond to nodes of a text document, 
for example.

I'm sure we can add a sentence, perhaps to the document structure 
section, that warns about the nature of this constraint, so that one 
doesn't have to discover it from the places where the specific details 
are mentioned.

Otherwise, the issue seems to have been a non-issue for the existing 
implementations.

A./

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Sat, 23 Apr 2016 11:03:27 -0700

>> >document is syntactically invalid and thus must be ignored.

One would expect the schema validator to catch such issues,
> the LGR if the spec doesn't say to fail to load the LGR on such errors.

There's no reason to be permissive in this way. The LGR is supposed to 
describe a definite disposition for any label.

A./

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Sat, 23 Apr 2016 11:06:05 -0700

> free to correct me if I am wrong):

I've had a preliminary look, and it looks very much like all points can 
be addressed with minor additions of clarifying text. I'm in the middle 
of drafting.

A./

---

From: Alexey Melnikov <[email-masked]>
Date: Mon, 25 Apr 2016 09:41:10 +0100

Hi,

> On 23 Apr 2016, at 19:06, Asmus Freytag (c) <[email-masked]> wrote:
> 
> I've had a preliminary look, and it looks very much like all points can be addressed with minor additions of clarifying text. I'm in the middle of drafting.

If you can post a new version this week, that would be very helpful to get the document done.

Thank you,
Alexey

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Thu, 28 Apr 2016 10:41:52 -0700

I've tried to address these comments and have come up with some wording, 
which I will now check into the draft on the GitHub. I'll need to 
coordinate with Kim about pushing up a draft.

Does anyone see additional need to change text in response to Stephen's 
comments or do discuss them further. In particular, I did not make any 
change to the securities issues section, so if someone has a suggestion 
of text that we could/should add there in response, that would be useful.

A./

> kind of general rule if you can.

One would expect the schema validator to catch such issues.
It's a simple matter to in to provide explicit language in Section 4 
that an LGR that does not conform to the schema in Appendix D is to be 
rejected.

"An LGR is expressed as a well-formed XML Document <xref target="XML"/> 
that conforms to the schema defined in <xref target="schema"/>."

Then we only need to discuss any instances of constraints that are not 
enforced/enforceable by schema validation, and we do that, where those 
are discussed.
> What is parsing s/w supposed to do with structures like that?

There is no issue with parsing.

It's an example of deliberately obfuscatory design of context rules, but 
a conformant implementation should have no problem evaluating such an 
LGR. The result is that 200D is never valid in a label, because its 
conditional context (a single character label not containing 200D) is 
never matched.

Specifying contexts that cannot be matched by any label is not an error. 
In fact, it is a useful design feature for LGR templates. Such templates 
can describe optional code points that can be enabled/disabled by simply 
changing a context rule from never matching to always matching or vice 
versa. Incidentally, there are less obfuscatory ways to write a 
never-matching context, for example:
<rule name="never">
     <start />
     <end />
</rule>

> s/w can handle that (without crashing or looping forever).

These kinds of convoluted constructs do not lead to crashes or looping. 
Rules are simply regular expressions, and no matter how convoluted a 
rule definition appears to be, it can always be converted (resolved) 
regex string which can then be used in a regex engine to do the matching.

> of 6.1 provides an answer to this point.  (But if you do,

That text addresses a different issue altogether, so I agree entirely.

However, section 6.1 flowed badly and didn't mention the "rules" 
element, so I suggest the following tweak:

  6.1. <#rfc.section.6.1>Basic Concepts

The "rules" element contains the specification of both context-based and 
whole Whole Label Evaluation (WLE) rules (Section 6.3 <#whole_label>), 
the character classes (Section 6.2 <#character_classes>) that they 
depend on and any actions (Section 7 <#actions>) that assign 
dispositions to labels based on rules or variant mappings.

A Whole Label Evaluation rule (WLE) is applied to the whole label. It is 
used to validate both original labels and any variant labels computed 
from them.

A conditional context rule is a specialized form of rule, that does not 
necessarily apply to the whole label, but may be specific to the context 
around a single code point or code point sequence. Certain code points 
in a label sometimes need to satisfy context-based rules, for example 
for the label to be considered valid, or to satisfy the context for a 
variant mapping (see the description of the "when" attribute inSection 
6.4 <#parameterized_context_rule>).

> attribute to help with ordering which seems ickky.)

In line with the discussion we had on this issue, the following text
are proposed be added to section 4.2

    Some elements that are direct or nested child elements of the
    <rules> element
    MUST be placed in a specific relative order to other elements for
    the LGR to be valid.
    An LGR that violates these constraints MUST be rejected. In other
    cases, a different
    ordering would result in a changed specification.

> the security considerations needs to be re-done really.

My reading of this comment is that it presupposes a particular use-case.

The use case we've identified so far is that these will substitute for 
IDN tables.

IDN tables are controlled by registries and define registry policy. As
such it would be up to the registry to have a policy that is well-defined,
and the registries would be in charge of developing or adopting an LGR
that matches their policies.

The LGR data format (and that's what it is) doesn't change anything in who
can set and enforce registry policy.

I also do not understand what an "imprecise" LGR would be. In contrast
to what they replace (IDN tables), LGRs are way more definite.

Issues of deployment of LGRs in IDN registration seem out of scope for
this document.
         this comment is unclear. Does it refer to the existence of the 
validity-end
         element? Or to it's mention in the text?

         For use in describing historical registration processes, a 
validity-end appears
         useful enough.
> - 4.3.8: odd that the examples have no URLs
         The text will include the formal reference of the Unicode 
Standard,
         including a URL.
> I'd have thought would be useful in disputes.

    I have no opinion on this. It's a fine idea and if anybody wants to
    suggest additional metadata, we can put them in. This is also not
    part of the DISCUSS, so we are under no obligation to rush this.

> to delete that paragraph and say that only the explicitly

I suggest we reword the end of the second to last paragraph:
"As with symmetry, these transitive relations are only part of the LGR if
spelled out explicitly."

We could further add a suggestion to the programmer:
"Implementations that require an LGR to be symmetric
and transitive should verify this mechanically."
> term, at least for implementers.

  Added to Section 5, 2nd para: "Collectively, these are known as the 
repertoire."
> cruel to the implementer to say that but not even have a
a reference to 8.5 has been added
> 2nd para there is not really useful as it's too opaque.

This is never going to make "easy reading", but here's an attempt at 
tweaking this a bit.

              8.5 *Checking Labels for Collision*
                   The obvious method for checking collision between 
labels is to generate the
                   fully permuted set of variants for one of them and 
see whether it contains the
                   other label as a member. As discussed above, this can 
be prohibitive and is not
                   necessary.

                  Because of symmetry and transitivity, all variant 
mappings form disjoint sets.
                  In each of these sets, the source and target of each 
mapping are also variants
                  of the sources and targets of all the other mappings. 
However, members of
                  two different sets are never variants of each other.

                  If two labels have code points at the same position 
that are members of two
                  different of these variant mapping sets, any variant 
labels of one, cannot be
                  variant labels of the other:  the sets of their 
variant labels are likewise disjoint.
                  Instead of  generating all permutations to compare all 
possible variants, it is
                  enough to find out whether code points at the same 
position belong to the
                  same variant set or not.

                 For that, it is sufficient to substitute an "index" 
mapping that identifies the
                 set. This index mapping could be, for  example, the 
variant mapping for which
                 the target code point (or sequence) comes first in some 
sorting order. This
                 index mapping would, in effect, identify the set of 
variant mappings for that
                 position.

                 To check collision then means generating a single 
variant label from the original
                 by substituting the respective "index" value for each 
code point. This results in an
                 "index label". Two labels collide whenever the index 
labels
                 for them are the same.

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Thu, 28 Apr 2016 12:58:18 -0700

Following on our discussion by e-mail, here is suggested wording.

A./

These kinds of convoluted constructs do not lead to crashes or looping. 
Rules are simply regular expressions, and no matter how convoluted a 
rule definition appears to be, it can always be converted (resolved) to 
a regex pattern which can then be used in a regex engine to do the matching.

  6.1.
  <mailbox:///C:/Users/asmusf/AppData/Roaming/Thunderbird/Profiles/965oo05j.default/Mail/Local%20Folders/Inbox?number=3739746514#rfc.section.6.1>Basic
  Concepts

The "rules" element contains the specification of both context-based and 
whole Whole Label Evaluation (WLE) rules (Section 6.3 
<mailbox:///C:/Users/asmusf/AppData/Roaming/Thunderbird/Profiles/965oo05j.default/Mail/Local%20Folders/Inbox?number=3739746514#whole_label>), 
the character classes (Section 6.2 
<mailbox:///C:/Users/asmusf/AppData/Roaming/Thunderbird/Profiles/965oo05j.default/Mail/Local%20Folders/Inbox?number=3739746514#character_classes>) 
that they depend on and any actions (Section 7 
<mailbox:///C:/Users/asmusf/AppData/Roaming/Thunderbird/Profiles/965oo05j.default/Mail/Local%20Folders/Inbox?number=3739746514#actions>) 
that assign dispositions to labels based on rules or variant mappings.

6.4 
<mailbox:///C:/Users/asmusf/AppData/Roaming/Thunderbird/Profiles/965oo05j.default/Mail/Local%20Folders/Inbox?number=3739746514#parameterized_context_rule>).

> - Appendix C: Another example of BNF being impressively

_______________________________________________
Lager mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/lager

---

From: Alexey Melnikov <[email-masked]>
Date: Fri, 29 Apr 2016 11:05:19 +0100

Hi Asmus,
 
On Thu, Apr 28, 2016, at 08:58 PM, Asmus Freytag (c) wrote:
>> (1) section 5: this says code points MUST be 4 hex digits. What is
>>     s/w supposed to do if it sees only 2 hex digits? Should it ignore
>>     the range or char element or fail to process the entire LGR
>>     document? I think the same issue applies to other uses of 2119
>>     language as well, (e.g. "MUST be treated as an error at the end
>>     of p19), so I'd recommend you state some kind of general rule if
>>     you can.
>
>  that an LGR that does not conform to the schema in Appendix D is to
>  be rejected.
 
Please say so explicitly.
 
>
>  "An LGR is expressed as a well-formed XML Document <xref
>  target="XML"/> that conforms to the schema defined in <xref
>  target="schema"/>."
>
>  Then we only need to discuss any instances of constraints that are
>  not enforced/enforceable by schema validation, and we do that, where
>  those are discussed.
 
+1.
 
I scanned the rest of your proposed changes and they look reasonable to
me. Please post an updated draft ASAP.

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Fri, 29 Apr 2016 08:20:59 -0700

> Please say so explicitly.

Latest draft text (in preparation) for Section 4, "LGR Format" is:

"An LGR is expressed as a well-formed XML Document [XML] 
<cid:[email-masked]> that conforms to the schema 
defined in Appendix Appendix D <cid:[email-masked]>."

with the new following the [XML].

I think that ought to be sufficient. While it does not use the word 
"rejected" it makes clear that a non-conforming document does not 
constitute an LGR. Let me know if you think this needs to be even more 
explicit.
> I scanned the rest of your proposed changes and they look reasonable 
> to me. Please post an updated draft ASAP.
Will work with Kim on that.

A./

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Fri, 29 Apr 2016 09:10:53 -0700

> to me. Please post an updated draft ASAP.

Kim,

will you have a chance to review these and push up an update? If not, I 
can probably help out.

Just let me know.

A./

PS: we do need to pick up Wil's tweak on the schema, which is not merged yet

---

From: Alexey Melnikov <[email-masked]>
Date: Fri, 29 Apr 2016 17:21:42 +0100

Hi,

> On 29 Apr 2016, at 16:20, Asmus Freytag (c) <[email-masked]> wrote:
> 
>>> It's a simple matter to in to provide explicit language in Section 4 that an LGR that does not conform to the schema in Appendix D is to be rejected.
>>  
> "An LGR is expressed as a well-formed XML Document [XML] that conforms to the schema defined in Appendix Appendix D."

This is good.

> I think that ought to be sufficient. While it does not use the word "rejected" it makes clear that a non-conforming document does not  constitute an LGR. Let me know if you think this needs to be even more explicit.

I think Stephen wanted more text about what happens if a document is non conforming. Or at least what are valid handling options.

So yes, I think spelling this out would help.

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Fri, 29 Apr 2016 09:36:35 -0700

> Please say so explicitly.

New text. (Compared to draft 11, the second half of the first sentence 
and the last sentence are new).
------------------------------------------------------------------------

  4. <#rfc.section.4>LGR Format

An LGR is expressed as a well-formed XML Document[XML] <#XML>that 
conforms to the schema defined inAppendix Appendix D <#schema>.

As XML is case-sensitive, an LGR must be authored with the correct 
casing. For example, the XML element names MUST be in lower case as 
described in this specification, and matching of attribute values, is 
only performed in a case-sensitive manner.

A document that is not well-formed, non-conforming or violates other 
constraints specified in this specification MUST be rejected.

------------------------------------------------------------------------

I think this makes it abundantly clear that the spec does not condone 
"guessing".

A./

---

From: Stephen Farrell <[email-masked]>
Date: Fri, 29 Apr 2016 17:53:45 +0100

(Sorry on the way out the door, and I may owe you more of a response
but I noticed this...)

> only performed in a case-sensitive manner.

That's an odd example though. Surely the more likely way to get
a non-validating is more like the one I called out where a code
point omits leading zeros or similar?

S.

---

From: "Asmus Freytag (c)" <[email-masked]>
Date: Fri, 29 Apr 2016 10:26:26 -0700

> point omits leading zeros or similar?

I think we do not want to repeat all the constraints from the schema in 
this introductory section.

Casing was mentioned by somebody in a different comment, so this text 
was added in an earlier round
and should have been in the version you reviewed. I think it is a useful 
reminder of the case-sensitive
nature of XML document parsing, but, like the issue you raised, really 
is already implicit in the fact that the
specification is about defining an XML-based, schema-validated 
representation.

The new text in response to your comment focuses on the schema. The number
format is described in the schema via a pattern, as in:

## a single code point
code-point = xsd:token {
     pattern = "[0-9A-F]{4,6}"
}

I believe that the existing language in the spec (where the effect of 
the pattern is described in
words) together with the pattern in the schema is sufficient to make 
clear that "01" is not
valid while "0001" is.

The new text in section 4 serves the purpose to act as a general 
reminder that documents that
fail the schema validation must be rejected. Like the casing issue, this 
reminder is intended to
be helpful, but one can argue that it isn't strictly needed.

A./

---

From: Alexey Melnikov <[email-masked]>
Date: Fri, 29 Apr 2016 19:04:18 +0100

Hi,

> On 29 Apr 2016, at 18:26, Asmus Freytag (c) <[email-masked]> wrote:
> 
> I believe that the existing language in the spec (where the effect of the pattern is described in
> words) together with the pattern in the schema is sufficient to make clear that "01" is not
> valid while "0001" is.

I agree that your proposed text addresses this issue, as 2 character violates the pattern regex.

---

From: Alexey Melnikov <[email-masked]>
Date: Fri, 29 Apr 2016 19:02:31 +0100

Hi Asmus,

> On 29 Apr 2016, at 17:36, Asmus Freytag (c) <[email-masked]> wrote:
> 
> New text. (Compared to draft 11, the second half of the first sentence and the last sentence are new).
> 4. LGR Format
> 
> An LGR is expressed as a well-formed XML Document [XML] that conforms to the schema defined in Appendix Appendix D.
> 
> As XML is case-sensitive, an LGR must be authored with the correct casing. For example, the XML element names MUST be           in lower case as described in this specification, and matching of attribute values, is only performed in a case-sensitive manner.
> 
> A document that is not well-formed, non-conforming or violates other constraints specified in this specification MUST be rejected.
> 
> 
> 
> I think this makes it abundantly clear that the spec does not condone "guessing".

This text looks good to me. And I believe it covers Stephen's case of 2 digit code points.