Subject: Re: [EAI] Re: I-D Action:draft-hansen-4468upd-mailesc-registry-03.txt
From: Jeff Macdonald <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: Tony Hansen <[email-masked]>, [email-masked]
Date: Thu, 31 Jan 2008 21:25:19 -0500
List-Id: <ietf-smtp.imc.org>

On Tue, Jan 15, 2008 at 11:45:16AM +0000, Alexey Melnikov wrote:
>
> Tony Hansen wrote:
>
>> This version does not define X.8.Z. I was not convinced there was enough
>> of a consensus to add it.
>>
> Tony, can you refresh my memory on where X.8.Z codes are defined?

I proposed x.8.z codes to help distinquish policy related codes from
'security policy' codes. By policy I suppose I really mean 'anti-spam'
policy. I failed to convince anyone at this point simply due to lack of
available time. I do however have a RFC like document that lists a
bunch codes and the phrases to go with those codes. I will post this to
the list shortly.

-- 
:: Jeff Macdonald | Director of Messaging Technologies
:: e-Dialog | [email-masked]
:: 131 Hartwell Ave. | Lexington, MA 02421
:: v: 781-372-1922 | f: 781-863-8118
:: www.e-dialog.com

Date: Wed, 30 Jan 2008 08:41:27 +0100
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: fix forwarding: sect. 3.9 of draft-klensin-rfc2821bis
Sender: [email-masked]

Frank Ellermann wrote:
> Alessandro Vesely wrote:
>>| We classify such a pseudo-mailbox as an "alias and similar 
>>| forwarding" or a "list", depending upon the expansion rules.
> 
> No "we" style in 2821 so far.  I don't get this classification,
> what's the idea here ?  In essence you can have "redistribution"
> (list), local aliases, or forwarding (non-local aliases).

I tried to minimize the changes to the original text. Perhaps I've been 
too lazy. I also don't like that classification, and the resulting text 
conceives two cases with no other reason than current practices showing 
that mailing list work quite better than aliases. I also agree that we 
might get ready of the term "pseudo-mailbox".

Some criteria that I'd deem relevant for classifying a forwarding 
arrangement are as follows:

* the arrangement has been configured manually / automatically
* if automatic, any opt-in evidence (e.g. double check, IP)
* an alias is meant for non-disclosure / is an address update
* expansion is 1-1 / 1-many

That might suggest how to exert administrative control appropriately. 
However, encouraging servers to keep track of any opt-in evidence or other 
forwarding arrangement reasons would probably be in vain.

I'll answer your other comments during the next days, hoping that there 
will be some more involvement meanwhile. The current rfc2821bis-06 defines 
"alias" as a pseudo-mailbox subject to simplistic (broken) expansion, so I 
think this classification is the most important point.

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: fix forwarding: sect. 3.9 of draft-klensin-rfc2821bis
Date:  Mon, 28 Jan 2008 02:32:38 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 79
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Alessandro Vesely wrote:
=20
> As I received no objections, here's my 2 cents.

Watching this list for less than four years I fear it does
not work that way, but testing it is certainly allowed. ;-)

>| Because of its administrative relevance, this address MAY
>| be used to devise local acceptance policies based on the
[...]
>| While recognizing the existence of such policies, this
>| paragraph only describes general issues intended for
>| efficient and reliable transport and delivery of mail messages.

I'd remove that part, it is rather vague.  Of course the
MAIL FROM like anything else in the envelope can be used
for the decision to reject or accept a mail.  And as the
reverse path MUST be used to report non-delivery to the
originator it's arguably not only an option (MAY) to make
sure that it's no nonsense when accepting it.

>| A server SHOULD allow to change the envelope return=20
>| address if required by local forwarding policies.

Isn't that anyway the case for redistribution in the style
of a mailing list ?  Nothing's wrong if a mailing list has
only one member.

>| If the server changes the envelope return address, it
>| MUST also remove the Return-Path informative header, if
>| any exists.

Is that new ?  So far I thought that removing erroneous
Return-Path header fields inserted before final delivery
are the job of the last hop adding the real Return-Path.

And IIRC that's unfortunately no MUST yet, the 2821-to-DS
folks are very anxious about new MUSTard.  And as a third
thought, more MTAs trying to "fix" headers could mean more
potential trouble.

>| Since RFC1123 [2] banned source routing in the envelope
>| return path, error messages did not have to go through the
>| reverse path any more.

+1, maybe s/banned/deprecated/ =20

>| Therefore, in order to exert any administrative control on failed
>| deliveries, it is necessary that each forwarding server accurately
>| plans how to change the envelope return address.

Yes, something in this direction.  It is also in order to take the
responsibility as *required* by 821, inadvertently removed by 1123.

>| We classify such a pseudo-mailbox as an "alias and similar=20
>| forwarding" or a "list", depending upon the expansion rules.

No "we" style in 2821 so far.  I don't get this classification,
what's the idea here ?  In essence you can have "redistribution"
(list), local aliases, or forwarding (non-local aliases).

The third case is generally not more desirable after RFC 1123
removed the indication of responsibility in the reverse path -
it can be still okay by mutual agreement, but that's special.

>| The envelope return address SHOULD be changed according to
>| local forwarding policies, or left intact in case of lack
>| thereof.  The message is then delivered or forwarded to each
>| expanded address.

Changing it would be odd for local aliases (your "delivered"
case).  And leaving it as is without an agreement with the
admin for the "forwarded to" address is no question of local
policy, it breaks the simple and accurate RFC 821 concept of
responsibility for forwarding.  That is IMO the RFC 1123 bug.

Another "+1" for the rest of your proposal.

 Frank

Date: Sun, 27 Jan 2008 17:45:34 +0100
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: fix forwarding: sect. 3.9 of draft-klensin-rfc2821bis
Sender: [email-masked]

I wrote:
> IMHO, a consistent change should affect all of section 3.9
> (Mailing Lists and Aliases), beginning with its title, and
> previous sections on relaying (3.4 and 3.6) would implicitly
> or explicitly refer to this one. Should I post a tentative
> text?

As I received no objections, here's my 2 cents.

3.9. Replacing the Envelope Return Address on Forwarding

    An SMTP-capable host SHOULD support both the alias and the list
    models of address expansion for single or multiple deliveries.  When
    a message is delivered or forwarded to each expanded address, the
    return address in the envelope ("MAIL FROM:") is particularly
    important for administrative purposes, as it is the destination of
    any error messages.  Because of its administrative relevance, this
    address MAY be used to devise local acceptance policies based on the
    entity exerting administrative control on the message
    transportation, e.g. the domain part of the envelope address in
    SPF.  While recognizing the existence of such policies, this
    paragraph only describes general issues intended for efficient and
    reliable transport and delivery of mail messages.

    A server SHOULD allow to change the envelope return address if
    required by local forwarding policies.  If the server changes the
    envelope return address, it MUST also remove the Return-Path
    informative header, if any exists.  However, the message header
    section (RFC2822 [9]) MUST be left unchanged; in particular, the
    "From" field of the header section is unaffected.

       Note: Since RFC1123 [2] banned source routing in the envelope
       return path, error messages did not have to go through the
       reverse path any more.  Therefore, in order to exert any
       administrative control on failed deliveries, it is necessary that
       each forwarding server accurately plans how to change the
       envelope return address.

    An important mail facility is a mechanism for multi-destination
    delivery of a single message, by transforming (or "expanding" or
    "exploding") a pseudo-mailbox address into a list of destination
    mailbox addresses.  When a message is sent to such a pseudo-mailbox
    (sometimes called an "exploder"), copies are forwarded or
    redistributed to each mailbox in the expanded list.  Servers SHOULD
    simply utilize the addresses on the list; application of heuristics
    or other matching rules to eliminate some addresses, such as that of
    the originator, is strongly discouraged.  We classify such a pseudo-
    mailbox as an "alias and similar forwarding" or a "list", depending
    upon the expansion rules.

3.9.1. Alias and similar forwarding

    To expand an alias, the recipient mailer replaces the pseudo-mailbox
    address in the envelope with each of the expanded addresses in
    turn.  The envelope return address SHOULD be changed according to
    local forwarding policies, or left intact in case of lack thereof.
    The message is then delivered or forwarded to each expanded address.

    The server administrator should specify a privacy policy and
    carefully determine the kind of forwarding that each alias expansion
    implies, in order to configure the mailer accordingly.  If an alias
    is meant to conceal its expanded address(es), the envelope return
    address has to be changed so that any error message will not reveal
    the final mailbox name to the message originator.  When forwarding
    for address correction or updating (see section 3.4) a delivery
    failure due to the fact that the updated address has been deleted in
    turn should also reach a person or other entity who is in control of
    removing the relevant alias configuration directive, besides the
    current message originator who, by that time, might have already
    received a 251 response (but see also section 7.7).

3.9.2. List

    The envelope return address MUST be changed to be the address of a
    person or other entity who administers the list.

    A mailing list may be said to operate by "redistribution" rather
    than by "forwarding".  To expand a list, the recipient mailer
    replaces the pseudo-mailbox address in the envelope with each of the
    expanded addresses in turn.  The envelope return address is changed
    so that all error messages generated by the final deliveries will be
    returned to a list administrator, not to the message originator, who
    generally has no control over the contents of the list and will
    typically find error messages annoying.

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: Spaces in quoted-string
Date:  Sat, 26 Jan 2008 12:50:20 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 26
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Frank Ellermann wrote nonsense, see also
<http://permalink.gmane.org/gmane.ietf.rfc822/12049>

What Alexey found is an ordinary 2821 bug, no 2822 issue.

2822 + 2822upd have:

| quoted-string =3D [CFWS]
|                 DQUOTE *([FWS] qcontent) [FWS] DQUOTE
|                 [CFWS]

2821 + 2821bis have:

| Quoted-string =3D DQUOTE *qcontent DQUOTE

The "bare SP" case is AWOL, and 2821bis can simply add it:

  Quoted-string =3D DQUOTE *(SP / qcontent) DQUOTE

John's proposal <qcontentSMTP> does this, eliminating all
undesired 2822upd-obs cases (roughly 2822-obs + NO-WS-CTL)

 Frank
--=20
@ Alexey, if you need the missing SP *now* (in RFC 2821)
  maybe submit it as 2821 erratum.  Or wait for 2821bis.

Date: Fri, 25 Jan 2008 09:12:31 +0100
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: Last Call: draft-klensin-rfc2821bis [...]
Sender: [email-masked]

Mark Martinec wrote:
> On Thursday 24 January 2008 09:55:41 Alessandro Vesely wrote:
>> Assuming that SMTP doesn't need to delve into boundaries and complex
>> network topologies, the best it can do is to refer to current best
>> practices. E.g, like so:
>>     To expand an alias, the recipient mailer replaces the recipient
>>     address in the envelope with each of the expanded addresses in
>>     turn. Trace information SHOULD be removed and the envelope sender
>>     address SHOULD be changed according to local forwarding policies,
>>     or left intact in case of lack thereof.
> 
> If by 'Trace information' you mean a Return-Path, that is fine,
> but the text needs to say so. The 'Trace information' also covers
> the Received header fields, which must not be deleted!

That's fine for me as well. Would the text resulting after
s/Trace information/Return-Path/ be acceptable?

The resulting text permits SPF-compliant mail forwarding.
However, the section title is just "Alias." Forwarding is
mentioned in section 3.4. (Forwarding for Address Correction or
Updating.) That section also refers to local policies, calling
them "configuration mechanisms", but doesn't say anything about
the envelope.

The next section, 3.9.2 (List) notes that the key difference
is the change to the backward-pointing address. That would not
be exact any more.

IMHO, a consistent change should affect all of section 3.9
(Mailing Lists and Aliases), beginning with its title, and
previous sections on relaying (3.4 and 3.6) would implicitly
or explicitly refer to this one. Should I post a tentative
text?

On a stylistic note, section 3.6.3 (Message Submission Servers
as Relays) introduces a mail submission protocol which, after
the additions in subsequent versions of the document, sounds
somewhat redundant:

    A standardized mail submission protocol has been developed
    that is gradually superseding practices based on SMTP
    (see RFC4409 [24]).

In addition, that text may result in an endless loop when
RFC4409, section 3.1. (Submission Identification) says that
"The protocol used is ESMTP, with additional restrictions or
allowances as specified here." I'd replace that text with

    Additional restrictions or allowances w.r.t. the practices
    described here are specified in the mail submission protocol
    (see RFC4409 [24]).

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: Last Call: draft-klensin-rfc2821bis [...]
Date:  Thu, 24 Jan 2008 12:45:10 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 22
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Mark Martinec wrote:

> The 'Trace information' also covers the Received=20
> header fields, which must not be deleted!

+1  2821 and 2821bis are very strict about this,
as quoted by you in 4.4.  Also confirmed in 3.7.2
for gateways, which could be summarized to "even
if you think that foreign trace info is broken
or pointless leave it alone, it might still help".

> it might be prudent to be say:
>   MUST NOT change --> MUST NOT change or delete

Yes, a problem for the Authentication-Results I-D,
which proposes to delete older header fields at
the "border" (IIRC).  Yet another case of "border",
where hoping that mail-arch explains what it is
likely won't fly - any "border" defined by "MX"
is orthogonal to Dave's POV, a matrix of ADMDs.
=20
 Frank

From: Mark Martinec <[email-masked]>
Organization: Jozef Stefan Institute
To: [email-masked]
Subject: Re: Last Call: draft-klensin-rfc2821bis [...]
Date: Thu, 24 Jan 2008 10:45:40 +0100
Sender: [email-masked]

On Thursday 24 January 2008 09:55:41 Alessandro Vesely wrote:
> Assuming that SMTP doesn't need to delve into boundaries and complex
> network topologies, the best it can do is to refer to current best
> practices. E.g, like so:
>     To expand an alias, the recipient mailer replaces the recipient
>     address in the envelope with each of the expanded addresses in
>     turn. Trace information SHOULD be removed and the envelope sender
>     address SHOULD be changed according to local forwarding policies,
>     or left intact in case of lack thereof.

If by 'Trace information' you mean a Return-Path, that is fine,
but the text needs to say so. The 'Trace information' also covers
the Received header fields, which must not be deleted!

Secton 4.4. of the draft currently say:

   An Internet mail program MUST NOT change a Received: line that was
   previously added to the message header section.  SMTP servers MUST
   prepend Received lines to messages; they MUST NOT change the order of
   existing lines or insert Received lines in any other location.

it might be prudent to be say:
  MUST NOT change --> MUST NOT change or delete

Mark

Date: Thu, 24 Jan 2008 09:55:41 +0100
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: Last Call: draft-klensin-rfc2821bis [...]
Sender: [email-masked]

> At 07:50 16-01-2008, Frank Ellermann wrote:
>> The sender can't fix any problems between "forwarder" and
>> "receiver" (or next "forwarder"), the complete concept of
>> "forwarding to 3rd parties" is utter dubious (after 1123).
> 
> You may be right but it still won't be right unless the other 
> participants agree with your point of view.

+1 agreeing with Frank's POV.

Section 4.4 (Trace Information) mentions that

    any further (forwarding, gateway, or relay) systems MAY remove the
    return path and rebuild the MAIL command as needed to ensure that
    exactly one such line appears in a delivered message.

However, the previous section 3.9.1. (Alias) recommends that

    To expand an alias, the recipient mailer simply replaces the pseudo-
    mailbox address in the envelope with each of the expanded addresses
    in turn; the rest of the envelope and the message body are left
    unchanged.

Assuming that SMTP doesn't need to delve into boundaries and complex
network topologies, the best it can do is to refer to current best
practices. E.g, like so:

    To expand an alias, the recipient mailer replaces the recipient
    address in the envelope with each of the expanded addresses in
    turn. Trace information SHOULD be removed and the envelope sender
    address SHOULD be changed according to local forwarding policies,
    or left intact in case of lack thereof.

Date: Wed, 23 Jan 2008 22:44:20 -0800
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: Last Call: draft-klensin-rfc2821bis [...]
Sender: [email-masked]

Hi Frank,
At 07:50 16-01-2008, Frank Ellermann wrote:
>Okay, BUT... :-)  2821bis still creates the dilemma of
>"drop false positive" vs. "bounce to innocent bystander"
>if an unlucky receiver prematurely accepted a mail where
>that's possible.

2821bis does not deal with verification of return paths (Section 
3.6.2).  It does mention SPF. :-)

The dilemma in 2821bis is "local site policy" which means that you 
are in a better position to determine what is right for your environment.

>Various RFCs and drafts talk about "border MTA", and the
>mail-arch draft doesn't say what it is.  2821bis would be
>in an ideal position to explain it, it has all necessary
>ingredients (MX and address fallback).  If it can't solve
>the problems it creates (or rather inherits from 1123) it
>could still explain what's going on for naive postmasters.

I think that the mail-arch draft is better placed for that as it aims 
at giving an overall view of the architecture.

>The sender can't fix any problems between "forwarder" and
>"receiver" (or next "forwarder"), the complete concept of
>"forwarding to 3rd parties" is utter dubious (after 1123).
>
>Quoting an article on the SPF discuss list with a slightly
>different POV (= whose "fault" is it if forwarding doesn't
>work as designed in RFC 821):
>
><quote>
>The admin of the forwarder is not forced to support SRS,
>the admin of the receiver is not forced to white list
>the forwarder.  They didn't *break* anything that wasn't
>already broken by RFC 1123 5.3.6(a).  I submitted an
>erratum for this RFC, but this wasn't about 5.3.6(a).
>
>2821bis does not acknowledge that RFC 1123 5.3.6(a) was
>always broken.  I think we differ from 2821bis, but if
>forwarders strictly follow 2821bis they only *break*
>RFC 821, and everybody knows that RFC 821 isn't up to
>date.  RFC 4408 claims that RFC 821 reverse routes are
>"archaic", an appeal against this wording was rejected.

You may be right but it still won't be right unless the other 
participants agree with your point of view.

Regards,
-sm 

Date: Tue, 22 Jan 2008 09:49:01 +0000
From: Alexey Melnikov <[email-masked]>
To: Tony Hansen <[email-masked]>
CC: [email-masked], John Klensin <[email-masked]>
Subject: Re: [EAI] Re: I-D Action:draft-hansen-4468upd-mailesc-registry-03.txt
Sender: [email-masked]

Tony Hansen wrote:

>[email-masked] wrote:  
>
>>A New Internet-Draft is available from the on-line Internet-Drafts
>>directories.
>>
>>	Title           : A Registry for SMTP Enhanced Mail System
>>			  Status Codes
>>	Author(s)       : T. Hansen, J. Klensin
>>	Filename        : draft-hansen-4468upd-mailesc-registry-03.txt
>>	Date            : 2008-01-10    
>>
>Ok, so A Registry for SMTP Enhanced Mail System Status Codes,
>draft-hansen-4468upd-mailesc-registry, has been updated.
>
>This one represents a proper merger with
>draft-klensin-smtp-code-registry, and (I think) addresses almost all of
>the comments that have come out since -02.
>
>As requested, it has added a typical Associated Basic Status Code entry
>to the registration. It also has a table showing the typical Associated
>Basic Status Code for many of the existing codes, seeded with those
>found in RFCs 4468 and 4954. There are many holes in this table, though,
>that need to be filled in. Please feel free to contribute to this table.
>
For Isode M-Switch:

   | X.0.0        | any              | X.1.0        | ???              |
   | X.1.1        | ???              | X.1.2        | ???              |

X.1.1 --> 550, 451

   | X.1.3        | ???              | X.1.4        | ???              |
   | X.1.5        | 250              | X.1.6        | ???              |
   | X.1.7        | ???              | X.1.8        | ???              |

X.1.8 --> 501, 451

   | X.2.0        | ???              | X.2.1        | ???              |
   | X.2.2        | 552              | X.2.3        | ???              |
   | X.2.4        | ???              | X.3.0        | ???              |

X.3.0 --> 550, 554, 451

   | X.3.1        | ???              | X.3.2        | ???              |

X.3.1 --> 452

   | X.3.3        | ???              | X.3.4        | 554              |

X.3.4 --> 552 (I think 552 should definitely be allowed in this case)

   | X.3.5        | ???              | X.4.0        | ???              |
   | X.4.1        | 451              | X.4.2        | ???              |
   | X.4.3        | ???              | X.4.4        | ???              |

X.4.3 --> 451, 550

   | X.4.5        | ???              | X.4.6        | ???              |
   | X.4.7        | ???              | X.5.0        | 250, 554, 503    |

X.5.0 --> 250, 503, 451, 452

   | X.5.1        | ???              | X.5.2        | ???              |

X.5.1 --> 501, 503, 530, 550, 554, 555, 430
X.5.2 --> 500, 501, 502, 555

   | X.5.3        | ???              | X.5.4        | ???              |

X.5.4 --> 501, 451, 555

   | X.5.5        | ???              | X.5.6        | 500              |
   | X.6.0        | ???              | X.6.1        | ???              |
   | X.6.2        | ???              | X.6.3        | 554              |
   | X.6.4        | 250              | X.6.5        | ???              |
   | X.6.6        | 554              | X.7.0        | 235, 454, 530,   |

X.6.6 --> 554

   |              |                  |              | 554              |

X.7.0 --> 504, 535, 454, 220

   | X.7.1        | 550              | X.7.2        | ???              |

X.7.1 --> 550, 551, 451

X.7.2 --> 550

   | X.7.3        | ???              | X.7.4        | ???              |
   | X.7.5        | ???              | X.7.6        | ???              |
   | X.7.7        | ???              | X.7.8        | 554, 535         |
   | X.7.9        | 534              | X.7.11       | 538              |
   | X.7.12       | 432              |              |                  |

Date: Mon, 21 Jan 2008 22:53:01 +0100
From: Harald Tveit Alvestrand <[email-masked]>
To: Tony Hansen <[email-masked]>, [email-masked]
Cc: John Klensin <[email-masked]>
Subject: Re: I-D Action:draft-hansen-4468upd-mailesc-registry-03.txt
Sender: [email-masked]

Keeping this on ietf-smtp as requested... this draft is a dependency for an 
EAI deliverable, so I'm interested in pushing it out ASAP.

Some comments that might want to be addressed before Last Call:

- In section 2.1, the entry description is free-form. I'd prefer a strict 
format that fits the later registration templates.

- In section 2.1, the text claims that the class and subject sub-codes will 
include the same info as the enumerated status code table. That seems a bit 
strange.

- In section 2.1, I personally dislike "Owner", preferring "Change 
controller". But that is a matter of personal taste.

- In section 2.1, "Text expected to be associated with the code" seems too 
strong. We want to reinforce the idea that the text can change at the 
implementor's whim; what about "Example of a message that might be sent 
along with the code"?

- In section 2.2, "Specification required" should be allowed to stand on 
its own, without the verbiage about "evidence of significant deployment"; 
"specification required" in the 2434 sense includes "this is specified in 
Moon Software's MailBean applet manual, version 2.43, page 247", as long as 
such a reference qualifies as "readily available".

- In section 2.4, the notation "2.XXX.XXX" seems confusing, since all the 
real examples have one digit in the second position. "2.X.XXX" would be 
better.

- In section 2.4, I would prefer to replace the "???" for "associated basic 
status code" with "Not given". Before submitting for publication, the note 
"NOTE: this table is incomplete" should go, no matter how incomplete the 
table is.
(I also prefer a 2-column table; I get less confused that way. YMMV.)

- In section 2.4, I don't quite see the justification for X.7.10 and X.7.13 
- if they have been in use in the industry, a sentence saying so would be a 
Good Thing; if they're just thought up as "this seems like a good idea", 
that should be stated too.

Hope these comments help improve the document!

                    Harald

From: [email-masked]
Cc: John C Klensin <[email-masked]>, [email-masked]
Date: Fri, 18 Jan 2008 10:17:37 -0800 (PST)
Subject: Re: Last Call: draft-klensin-rfc2821bis         (Simple Mail Transfer Protocol) to Draft Standard
To: Alexey Melnikov <[email-masked]>
Sender: [email-masked]

> John,
> I have one more question:

> FRC 1869 defines 2 new SMTP reply codes in section 6.1:

>    If the server SMTP does not recognize or cannot implement one or more
>    of the parameters associated with a particular MAIL FROM or RCPT TO
>    command, it will return code 555.

>    If for some reason the server is temporarily unable to accomodate one
>    or more of the parameters associated with a MAIL FROM or RCPT TO
>    command, and if the definition of the specific parameter does not
>    mandate the use of another code, it should return code 455.

> They are not in RFC 2821 which obsoleted RFC 1869. Bug or feature?

Missed this, sorry. I'd have to call it a bug. IMO these need to be added
to 2821bis.

				Ned

Date: Fri, 18 Jan 2008 10:18:42 +0000
From: Alexey Melnikov <[email-masked]>
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: Last Call: draft-klensin-rfc2821bis  (Simple Mail Transfer Protocol) to Draft Standard
Sender: [email-masked]

John,
I have one more question:

FRC 1869 defines 2 new SMTP reply codes in section 6.1:

   If the server SMTP does not recognize or cannot implement one or more
   of the parameters associated with a particular MAIL FROM or RCPT TO
   command, it will return code 555.

   If for some reason the server is temporarily unable to accomodate one
   or more of the parameters associated with a MAIL FROM or RCPT TO
   command, and if the definition of the specific parameter does not
   mandate the use of another code, it should return code 455.

They are not in RFC 2821 which obsoleted RFC 1869. Bug or feature?

Regards,
Alexey

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: Last Call: draft-klensin-rfc2821bis [...]
Date:  Wed, 16 Jan 2008 16:50:50 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 69
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

"SM" <[email-masked]> wrote:

Hi, sorry for the late answer, sometimes flame wars with
Doug on various lists and other issues eat up my time ;-)

> I wouldn't read that section together with the other
> sections as a license to send bounces to innocent=20
> bystanders.

Okay, BUT... :-)  2821bis still creates the dilemma of
"drop false positive" vs. "bounce to innocent bystander"
if an unlucky receiver prematurely accepted a mail where
that's possible.

> It's better not to get into a definition of border MTA
> in 2821bis as each mail environment is different.  We're
> are discussing the protocol here and not antispam practice.

Various RFCs and drafts talk about "border MTA", and the
mail-arch draft doesn't say what it is.  2821bis would be
in an ideal position to explain it, it has all necessary
ingredients (MX and address fallback).  If it can't solve
the problems it creates (or rather inherits from 1123) it
could still explain what's going on for naive postmasters.

> If we change the envelope sender, the sender won't know
> if the mail was not delivered.

The sender can't fix any problems between "forwarder" and
"receiver" (or next "forwarder"), the complete concept of
"forwarding to 3rd parties" is utter dubious (after 1123).

Quoting an article on the SPF discuss list with a slightly
different POV (=3D whose "fault" is it if forwarding doesn't
work as designed in RFC 821):

<quote>
The admin of the forwarder is not forced to support SRS,
the admin of the receiver is not forced to white list=20
the forwarder.  They didn't *break* anything that wasn't
already broken by RFC 1123 5.3.6(a).  I submitted an
erratum for this RFC, but this wasn't about 5.3.6(a).

2821bis does not acknowledge that RFC 1123 5.3.6(a) was
always broken.  I think we differ from 2821bis, but if
forwarders strictly follow 2821bis they only *break*
RFC 821, and everybody knows that RFC 821 isn't up to
date.  RFC 4408 claims that RFC 821 reverse routes are
"archaic", an appeal against this wording was rejected.

Unless the IESG decides that 2821bis is too important=20
to do it outside of a proper IETF WG we'll get what it
says at the moment:

| To expand an alias, the recipient mailer simply
| replaces the pseudo-mailbox address in the envelope
| with each of the expanded addresses in turn; the rest
| of the envelope and the message body are left unchanged.
| The message is then delivered or forwarded to each
| expanded address.

"We" know that this cannot work for the 821-architecture
after 1123 killed the reverse routes, but obviously "we"
didn't manage to convince the author of 2821bis-06 that
this is madness for addresses at 3rd parties (in another
ADMD or MRN, pick what you like best).

 Frank
</quote>  The "we" is of course unrelated to this list.

Date: Tue, 15 Jan 2008 09:44:58 -0500
From: Tony Hansen <[email-masked]>
To: Alexey Melnikov <[email-masked]>
CC: [email-masked]
Subject: Re: [EAI] Re: I-D         Action:draft-hansen-4468upd-mailesc-registry-03.txt
Sender: [email-masked]

There was a discussion of possibly defining X.8.Z as a place holder for 
future "policy" codes, as opposed to the current mixture of both 
"security" and "policy" codes found in X.7.Z. I chose not to do this.

	Tony Hansen
	[email-masked]

Alexey Melnikov wrote:
> Tony Hansen wrote:
> 
>> This version does not define X.8.Z. I was not convinced there was enough
>> of a consensus to add it.
>>
> Tony, can you refresh my memory on where X.8.Z codes are defined?
> 

Date: Tue, 15 Jan 2008 11:45:16 +0000
From: Alexey Melnikov <[email-masked]>
To: Tony Hansen <[email-masked]>
CC: [email-masked]
Subject: Re: [EAI] Re: I-D Action:draft-hansen-4468upd-mailesc-registry-03.txt
Sender: [email-masked]

Tony Hansen wrote:

>This version does not define X.8.Z. I was not convinced there was enough
>of a consensus to add it.
>
Tony, can you refresh my memory on where X.8.Z codes are defined?

Cc: [email-masked]
From: Douglas Otis <[email-masked]>
To: "Frank Ellermann" <[email-masked]>
Subject: Re: SPF and IPv6
Date: Sun, 13 Jan 2008 19:39:52 -0800
Sender: [email-masked]

On Jan 9, 2008, at 7:48 PM, Frank Ellermann wrote:

>> Even if a "no answer" or "no domain" limitation were to become  
>> specified as a recommended limitation for SPF parsing routines
> [...]
>> The number of transactions that can be induced by a cached SPF  
>> record remains critically relevant to a DDoS concern
>
> The proposed 2..5 limit in the "rebuttal" (one way to implement an  
> existing SHOULD in RFC 4408) reduces your worst case of 100 bogus  
> overlong attack-queries to 2..5.  Implementation details, after  
> reaching 3..6 just trying this *again* for the next mail of the  
> attacker is not the smartest thing to do.

Botnets remain a threat.  A direct attack consumes the bot's resources  
and allows BCP 38 detection.  Even DNS open recursive reflective  
attacks allow bots to remain hidden until BCP 38 is instantiated.

The SPF DoS draft's macro exploit example with an MX mechanism caused  
some confusion.  The draft didn't clarify the motivations for using  
SPF as an attack mechanism.  SPF, as defined and typically  
implemented, allows botnets to become orders of magnitude more  
dangerous and not controllable by BCP 38.  SPF allows bots to remain  
hidden, uncontrolled, while also staging a free attack.

SPF parsing routines currently enable DNS resolver reflected attacks  
at levels significantly above that of the spam traffic acting as a  
trigger.  Assuming the SPF macro exploit concern causes  "no answer",  
or "no domain" responses to limit the extent of SPF macro induced  
transactions, then ensuring the critical recipient adoption  
necessitates exclusive publishing of a different SPF record version.

Such a change could help limit the magnitude of SPF exploit attacks  
triggered by spam.  However, adding an answer failure limit would not  
protect domains publishing MX or A wildcards.  In addition, rather  
than a maximum of 100 targeted transactions, quitting at N answer  
failures limits the number of targeted transactions to N times 4.   
Even so, the SPF macro exploit still allows the targeted transaction  
attacks to be free and beyond BCP 38 control.  A botnet is dangerous  
at 1:1 gains, and extremely dangerous at 100:1.

> Your "fast flux local parts" idea selects malicious MX records, but  
> after reaching 3..6 once it's clear that the policy is up to no  
> good, or something on the side of a legit sender has to be fixed  
> a.s.a.p.

Spam exploiting the SPF local-part macro can induce a series of new  
type 99, TXT, A, AAAA, or MX RR transactions from cached SPF records.   
SPF routines return Pass, Fail, SoftFail, Neutral, TempError,  
PermError, and None.  Due to SPF expansion of macros, even a PRA email- 
address within the same domain as MailFrom still requires repeating an  
entire SPF process whenever PASS is not first obtained.

Email path registration safe for domains using wildcards must first  
verify SMTP client names and then determine authorization (path  
registration) with records like TPA labels.  SMTP client authorization  
by-name scales well beyond that of SPF.   SMTP client authorization by- 
name requires only one transaction, avoids manual entry of address  
CIDRs, and can directly merge with DKIM domain signing policies.

Making SPF safer requires:

1) Changing the record version and specifying the scope of its coverage

2) Excluding processing of prior SPF versions

3) Limiting answer failures and returning TempError, instead of as  
specified by the ALL mechanism

4) Limiting the MX mechanism to 1 where host resolution min/max is  
specified

5) Prohibiting macro expansion for all but the EXP mechanism record

6) Delaying subsequent processing after time-outs until outstanding  
DNS transactions complete

7) Removing the Exists mechanism and dropping SoftFail

8) Specifying None is to be handled as Neutral

A different and safer SPF strategy could be adopted, but it is  
unlikely.  Why not consider SMTP client authorization by-name instead?

-Doug

  

Date: Thu, 10 Jan 2008 12:43:20 -0500
From: Tony Hansen <[email-masked]>
To: [email-masked]
CC: John Klensin <[email-masked]>, EAI WG <[email-masked]>, Harald Tveit Alvestrand <[email-masked]>
Subject: Re: I-D Action:draft-hansen-4468upd-mailesc-registry-03.txt
Sender: [email-masked]

[email-masked] wrote:
> A New Internet-Draft is available from the on-line Internet-Drafts
> directories.
>
> 	Title           : A Registry for SMTP Enhanced Mail System
>			  Status Codes
> 	Author(s)       : T. Hansen, J. Klensin
> 	Filename        : draft-hansen-4468upd-mailesc-registry-03.txt
> 	Date            : 2008-01-10

Ok, so A Registry for SMTP Enhanced Mail System Status Codes,
draft-hansen-4468upd-mailesc-registry, has been updated.

This one represents a proper merger with
draft-klensin-smtp-code-registry, and (I think) addresses almost all of
the comments that have come out since -02.

As requested, it has added a typical Associated Basic Status Code entry
to the registration. It also has a table showing the typical Associated
Basic Status Code for many of the existing codes, seeded with those
found in RFCs 4468 and 4954. There are many holes in this table, though,
that need to be filled in. Please feel free to contribute to this table.

This version does not define X.8.Z. I was not convinced there was enough
of a consensus to add it.

Discussion should occur on [email-masked].

	Tony Hansen
	[email-masked]

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: SPF and IPv6
Date:  Thu, 10 Jan 2008 04:48:55 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 69
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Douglas Otis wrote:

> SPF combines everything into one response

Yes, in other words it's a tool to put all IPs (IPv4 and IPv6)
into one of three classes PASS, FAIL, or "DUNNO" (the latter
is an over-simplification for NEUTRAL, SOFTFAIL, and "error",
anything that doesn't help for one or both of the tasks at
hand, match given IP against "permitted" or "forbidden" IPs)

> SPF's lack of resource record certitude means not finding=20
> TXT or type 99, or A or AAAA must be accommodated.

I can't parse this.  If a receiver doesn't find a policy, or
doesn't try it, or it's impossible (e.g. domain literal) to
find a policy, then the receiver continues as usual following
some receiver policy.  That MAY be a "best guess" evaluation
of "v=3Dspf1 a/24 mx/24 ?all", add/replace IPv6 CIDR depending
on the receiver, and toggle ?all into ~all or -all as desired.
Hint: the a-mechanism means AAAA if you try to match an IPv6.

Of course that is a "receiver policy", and RFC 4408 discusses
only "sender policy", not creative "best guess" uses of the
checkhost-algorithm for related purposes.  Receivers could do
what "a/24 mx/24" means also without SPF, if they think it is
a good "best guess".

> Even if a "no answer" or "no domain" limitation were to become =20
> specified as a recommended limitation for SPF parsing routines
[...]
> The number of transactions that can be induced by a cached SPF
> record remains critically relevant to a DDoS concern

The proposed 2..5 limit in the "rebuttal" (one way to implement
an existing SHOULD in RFC 4408) reduces your worst case of 100
bogus overlong attack-queries to 2..5.  Implementation details,
after reaching 3..6 just trying this *again* for the next mail
of the attacker is not the smartest thing to do. =20

Your "fast flux local parts" idea selects malicious MX records,
but after reaching 3..6 once it's clear that the policy is up
to no good, or something on the side of a legit sender has to
be fixed a.s.a.p.

> There is not any indication whether an MX mechanism host names =20
> resolves IPv4 or IPv6 addresses

That's unnecessary, you get a set of names (+ priorities), and
if you try to match an IPv6 query AAAA, and for an IPv4 query A.

> or how SPF CIDR masks should be applied when a mixture of
> addresses is returned.

That is clear, look for <ip4-cidr-length>, <ip6-cidr-length>,
and <dual-cidr-length> in the collected ABNF.  In a nutshell
x/y//z means IPv4 CIDR y or IPv6 CIDR z, short forms of this
are x//z (no IPv4 CIDR), x/y (no IPv6 CIDR), and x (no CIDR).

Exception:  For a "by number" mechanism x (either ip4 or ip6)
<dual-cidr-length> makes no sense, therefore ip4:x/y works as
always, and ip6:x/z has only a single slash <ip6-cidr-length>.

Admittedly it took me some time (MARID 2004) to figure it out,
more or less the only tangible result of MARID for v=3Dspf1 was
a serious clean-up of the ABNF.  Credits to Jutta Degener...=20

Well, and you contributed to the current processing limits ;-)

 Frank

Cc: [email-masked]
From: Douglas Otis <[email-masked]>
To: "Frank Ellermann" <[email-masked]>
Subject: Re: SPF and IPv6 (was: Last Call: draft-klensin-rfc2821bis: Permitted and  prohibited characters in addresses)
Date: Wed, 9 Jan 2008 12:21:13 -0800
Sender: [email-masked]

On Jan 8, 2008, at 9:40 PM, Frank Ellermann wrote:

>
> Douglas Otis wrote:
>
>> The http://dns.measurement-factory.com/surveys/200710.html show  
>> 12.6% of domains with name servers publish SPF records.
>
> "Percentage of domains" and "percentage of mails" (incl. spam) are  
> different.  Not all domains send mails (really or forged, I'm not  
> going to explain again why forging domains that can't receive mails  
> is likely not what the spammer wants).  The SPF deployment for  
> "relevant domains" (defined by Alexa) is good enough, see <http://utility.nokia.net/~lars/meter/spf.html 
> >

Agreed.  The distribution of SPF use is heavily biased toward larger  
domains.  However, this distribution should increase concerns  
pertaining to targeted DDoS permitted by SPF macro exploits, owing to  
the typical size of networks utilizing SPF. : (

>> When 4% of those then offer a possibility for FAIL, this represents  
>> less than half a percent of domains overall.
>
> It is likely better measured in "mails" instead of "domains".  If  
> folks dare not publish FAILing IPs their PASSing IPs are still  
> relevant.  After a PASS receivers can delay their spam analysis and  
> bounce later without causing backscatter.

This strategy still results in DSNs being dropped for a majority of  
the domains!  The size of a domain does not alter the importance of a  
DSN.  Even for those domains registering their IP address path, a  
significant portion of email is forwarded, where being off the  
registered path reduces the integrity of DSNs when SPF PASS represents  
an added requirement.

Alternatively, TPA-SSP represents a path independent, single  
transaction (by-name) means for ensuring DSNs are not dropped.  TPA- 
SSP avoids inducing a DDoS risk, and remains compliant with RFC 1123.

>> SPF does not use different record types to verify IPv6 or IPv4 SMTP  
>> client authorizations.
>
> SPF supports both IPv4 and IPv6 natively.  Its job is to match the  
> IP (IPv4 or IPv6) of an alleged sender (HELO or envelope sender)  
> against an enumeration of "permitted" or "forbidden" IPs to get a  
> PASS or FAIL result.

SPF combines everything into one response, without a specific initial  
record type, and without a means to segregate records sets for IPv4  
and IPv6 clients.  SPF's lack of resource record certitude means not  
finding TXT or type 99, or A or AAAA must be accommodated.

Even if a "no answer" or "no domain" limitation were to become  
specified as a recommended limitation for SPF parsing routines, the  
number of transactions occurring per cached SPF must still accommodate  
this protocol's inherent lack of record type specificity.  The number  
of transactions that can be induced by a cached SPF record remains  
critically relevant to a DDoS concern that results from an SPF macro  
related exploit.  Again, this exploit does not require an attacker to  
expend their resources.  Instead the attacker is able to leverage the  
resources of larger networks utilizing SPF. : (

> It's no rocket science to look at AAAA records when trying to match  
> an IPv6, or at A records for an IPv4, as far as a policy contains  
> any "by name" mechanisms.  For the two SPF "by address" mechanisms  
> matching IPs is straight forward.

While a routine parsing SPF records should know whether finding an  
IPv6 or IPv4 address is desired, SPF remains a exploitable mixed set.   
There is not any indication whether an MX mechanism host names  
resolves IPv4 or IPv6 addresses, or how SPF CIDR masks should be  
applied when a mixture of addresses is returned.  SPF did not  
considered how its macro feature could be exploited, or the impact of  
combining both IPv4 and IPv6 records into one sequence of SPF records.

-Doug

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  SPF and IPv6 (was: Last Call: draft-klensin-rfc2821bis: Permitted and  prohibited characters in addresses)
Date:  Wed, 9 Jan 2008 06:40:33 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 37
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

Douglas Otis wrote:

> The http://dns.measurement-factory.com/surveys/200710.html
> show 12.6% of domains with name servers publish SPF records.

"Percentage of domains" and "percentage of mails" (incl. spam)
are different.  Not all domains send mails (really or forged,
I'm not going to explain again why forging domains that can't
receive mails is likely not what the spammer wants).  The SPF
deployment for "relevant domains" (defined by Alexa) is good
enough, see <http://utility.nokia.net/~lars/meter/spf.html>

> When 4% of those then offer a possibility for FAIL, this=20
> represents less than half a percent of domains overall.

It is likely better measured in "mails" instead of "domains".
If folks dare not publish FAILing IPs their PASSing IPs are
still relevant.  After a PASS receivers can delay their spam=20
analysis and bounce later without causing backscatter. =20

> SPF does not use different record types to verify IPv6 or=20
> IPv4 SMTP client authorizations.

SPF supports both IPv4 and IPv6 natively.  Its job is to
match the IP (IPv4 or IPv6) of an alleged sender (HELO or
envelope sender) against an enumeration of "permitted" or
"forbidden" IPs to get a PASS or FAIL result.

It's no rocket science to look at AAAA records when trying
to match an IPv6, or at A records for an IPv4, as far as a
policy contains any "by name" mechanisms.  For the two SPF
"by address" mechanisms matching IPs is straight forward. =20

Skipping the rest of your rant, I've already answered that
on various mailing lists (DKIM, IETF general, ASRG, etc.).

 Frank

Cc: [email-masked]
From: Douglas Otis <[email-masked]>
To: "Frank Ellermann" <[email-masked]>
Subject: Re: Last Call: draft-klensin-rfc2821bis: Permitted and  prohibited characters in addresses
Date: Tue, 8 Jan 2008 14:17:14 -0800
Sender: [email-masked]

On Jan 8, 2008, at 1:38 AM, Frank Ellermann wrote:

>> Quoting Section 3.6.2.:
>>  "This specification does not deal with the verification of return
>>   paths for use in delivery notifications.  Recent work, such as that
>>   on SPF [42] and DKIM [43] [44], has been done to provide ways to
>>   ascertain that an address is valid or belongs to the person who
>>   actually sent the message."
>
> DKIM doesn't look at return paths, it's a kind of "signed  
> timestamp" (Received) from an SMTP POV.  DKIM might help to identify  
> mails that should be accepted.  By itself DKIM doesn't help to  
> identify mails that should be rejected.  SPF FAIL can do, but the  
> deployment of FAIL is not yet so overwhelming to decree victory, see  
> <http://spf-all.com/>

A reference to a dangerous experimental draft is disconcerting. : (

The http://dns.measurement-factory.com/surveys/200710.html show 12.6%  
of domains with name servers publish SPF records.  When 4% of those  
then offer a possibility for FAIL, this represents less than half a  
percent of domains overall.  SPF is not a solution for mitigating  
MailFrom abuse, nor should SPF even appear to be promoted.

SPF does not consider "no domain" or "no answer" as a criteria for  
limiting DNS transactions.  Both TXT and type 99 SPF records are  
optional, and SPF does not use different record types to verify IPv6  
or IPv4 SMTP client authorizations.  Either SPF RRs or address RRs may  
legitimately return "no answer" when an alternative is being used.   
When "no answer" is obtained for an SPF record, often an MX mechanism  
is then automatically substituted.  Transactional limits for "no  
answer" are therefore doubled by the SPF's RR type uncertainty.  A  
purported policy advantage afforded by wildcard SPF records also  
directly exposes these domains to an SPF macro exploit that might have  
been otherwise mitigated by a "no answer" limit.

SPF's macros, especially those expanding an email-address local-part,  
enables a sequence of new and different DNS transactions to be  
generated by the recipients of spam and based upon cached records from  
an attacker.  An attack that generates a sequence of TXT, type 99, MX,  
or A, AAAA records may be used to stage a sizeable attack against any  
other domain, and yet require no additional resources of the  
attacker.  Amplification of the attack, as compared to the spam being  
sent, can be greater than reflected exploits against open recursive  
servers.  Of course, spam would be sent anyway, making the SPF macro  
exploit attack virtually free. : (

This SPF macro exploit could use an a infinite number of local-parts  
to induce recipients into generating an infinite number of new DNS  
transactions from cached SPF records directed toward any innocent  
domain.  Authorizations for PRA or various DKIM related email- 
addresses further increases the risks related to SPF macro exploits.   
Instances for attack span beyond MTAs, where many plug-ins for web- 
mail and MUAs evaluate SPF authorizations and do not impose even  
recommended limits.  When SPF is used for white-listing, "+all" better  
ensures delivery and should discourage use of dangerous SPF routines.

> I've no real problem with the statement you quoted, I'd only wish  
> that 2821bis offers a better description of the underlying problem:   
> The border MTA is the last place where "ordinary" mails can be  
> rejected without causing harm.  "Accept on probation" is a seriously  
> bad idea for "ordinary" mails.  "Extraordinary" cases include SPF  
> PASS, where "accept on probation" still works.

A TPA-SSP scheme in conjunction with DKIM could also qualify the  
validity of a DSN email-address, rather than a dependence upon the IP  
address path registration.  TPA-SSP would also be compatible RFC 1123  
5.3.6(a).  Either the SPF path registration authorization scheme  
should be renovated and transformed into a "by-name" process, or it  
should be dropped as being dangerously unsuitable. : (

-Doug

Date: Tue, 08 Jan 2008 13:58:29 -0800
To: Frank Ellermann <[email-masked]>, [email-masked]
From: SM <[email-masked]>
Subject: Re: Last Call: draft-klensin-rfc2821bis: Permitted and  prohibited characters in addresses
Sender: [email-masked]

Hi Frank,
At 01:38 08-01-2008, Frank Ellermann wrote:
>DKIM doesn't look at return paths, it's a kind of "signed timestamp"
>(Received) from an SMTP POV.  DKIM might help to identify mails that
>should be accepted.  By itself DKIM doesn't help to identify mails
>that should be rejected.  SPF FAIL can do, but the deployment of FAIL
>is not yet so overwhelming to decree victory, see <http://spf-all.com/>

Agreed.

>I've no real problem with the statement you quoted, I'd only wish that
>2821bis offers a better description of the underlying problem:  The
>border MTA is the last place where "ordinary" mails can be rejected
>without causing harm.  "Accept on probation" is a seriously bad idea
>for "ordinary" mails.  "Extraordinary" cases include SPF PASS, where
>"accept on probation" still works.

That's more of a BCP.  2821bis leaves operational policy such as 
whether to implement SPF or not to the operator.

>IOW folks accepting mails with unverified return paths are in trouble:
>If they follow 2821bis 3.6.3 they send bounces to innocent bystanders,
>that can get them blacklisted and/or their bounces are ignored (that's
>bad for real problems).  If they ignore 2821bis 3.6.3 and drop "spam"
>later they managed to create a black hole for false positives.  While
>RFC 2821 and 2821bis are clear about the latter issue (MUST in 3.6.3),
>2821bis could be clearer about the former problem, receivers have only
>one chance to get it right, at their border.

The problem is not unverified return paths but people using 2821bis 
3.6.3 to solve their "spam" problem.  Note that the section talks 
about Message Submission Servers acting as relays.  It's possible to 
curtail abuse there.  I wouldn't read that section together with the 
other sections as a license to send bounces to innocent bystanders.

>Yes, that's fine, when the identification of "hostile" is very near to
>perfect - otherwise we'd be back in the "drop false positive" black
>hole.  Extending the definition of "hostile" to various kinds of spam
>would reduce the identification rate, and create a conflict with the
>MUST in 3.6.3.  We don't need to talk about the merits of 3.6.3 "as
>indicated by the reverse-path" for mails identified as spam, but is
>it really clear that accepting spam was an irrevocable *error* at the
>border MTA ?

We already know that we will be running the mail through a filter at 
some point.  We also know that blackholes are bad.  If we leave it to 
our filter to send uncontrolled bounces, we'll get ignored or 
blacklisted.  It's clear what our course of action should be.

>It's not optional, between the lines it is REQUIRED, and there's only
>one place for "ordinary" mail to get it right, at the border.  That's
>clear for folks on this list, but is it also clear for other readers
>of 2821bis ?

If it's not clear, the folks on this list will send text to clarify 
it so that it may be clear for other readers of 2821bis.

It's better not to get into a definition of border MTA in 2821bis as 
each mail environment is different.  We're are discussing the 
protocol here and not antispam practice.

>There's no problem with local aliases.  And an alias at a third
>party is again forwarding, where keeping the return-path "as is"
>was indirectly introduced by RFC 1123 5.3.6(a).  The old envelope
>sender is not more reponsible for this transaction.  The user who
>arranged it and the MTA deciding to try a "251" are responsible.

The Return-path is there to inform the sender of delivery 
problems.  If we change the envelope sender, the sender won't know if 
the mail was not delivered.

When we get a "251", we have to update our address book with the new 
address.  Most forwarders "silently" pass the mail through instead of 
giving a "251" as the user who has arranged it doesn't want the 
address changed.

>Of course 2821bis can't *solve* the spam problem, or reintroduce
>reverse routes.  But it should be honest about critical points
>in the post-821 design:  Border MTAs accepting dubious mails have
>already screwed up badly, and forwarders thinking that "keep the
>MAIL FROM 'as is'" was the original design missed that this was a
>bug introduced later.

What are you advocating for the MAIL FROM in the case of forwarders?

>What I had in mind was mail from an IPv6 only network forwarded
>by a dual-stack forwarder to an IPv4 only network.  And at this
>point addresses in the forwarded mail, especially any "as is"
>MAIL FROM address, violate MUSTard in 2821bis:  The IPv4-only
>network cannot report errors to the IPv6-only network.  The user
>arranging this can't reply from his IPv4-only network, SNAFU.

The domain would have a MX entry pointing to the dual-stack host so 
that the IPv4-only network can send the reply to the IPv6-only network.

>For simpler (non-forwarding) scenarios we could find that either
>sender, or receiver, or both did something they shouldn't do as
>"IPvXOnly" network.  For the forwarding scenario we are back at
>square one, keep MAIL FROM "as is" was a design flaw.

Regards,
-sm 

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: Last Call: draft-klensin-rfc2821bis: Permitted and  prohibited characters in addresses
Date:  Tue, 8 Jan 2008 10:38:34 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 106
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

<[email-masked]> wrote:
=20
> Quoting Section 3.6.2.:
>    "This specification does not deal with the verification of return
>     paths for use in delivery notifications.  Recent work, such as =
that
>     on SPF [42] and DKIM [43] [44], has been done to provide ways to
>     ascertain that an address is valid or belongs to the person who
>     actually sent the message."

DKIM doesn't look at return paths, it's a kind of "signed timestamp"
(Received) from an SMTP POV.  DKIM might help to identify mails that
should be accepted.  By itself DKIM doesn't help to identify mails
that should be rejected.  SPF FAIL can do, but the deployment of FAIL
is not yet so overwhelming to decree victory, see <http://spf-all.com/>

I've no real problem with the statement you quoted, I'd only wish that
2821bis offers a better description of the underlying problem:  The
border MTA is the last place where "ordinary" mails can be rejected
without causing harm.  "Accept on probation" is a seriously bad idea
for "ordinary" mails.  "Extraordinary" cases include SPF PASS, where
"accept on probation" still works.

IOW folks accepting mails with unverified return paths are in trouble:
If they follow 2821bis 3.6.3 they send bounces to innocent bystanders,
that can get them blacklisted and/or their bounces are ignored (that's
bad for real problems).  If they ignore 2821bis 3.6.3 and drop "spam"
later they managed to create a black hole for false positives.  While
RFC 2821 and 2821bis are clear about the latter issue (MUST in 3.6.3),
2821bis could be clearer about the former problem, receivers have only
one chance to get it right, at their border.

> "Conversely, if a message is rejected because it is found to contain
>  hostile content (a decision that is outside the scope of an SMTP
>  server as defined in this document), rejection ("bounce") messages
>  SHOULD NOT be sent unless the receiving site is confident that those
>  messages will be usefully delivered."

Yes, that's fine, when the identification of "hostile" is very near to
perfect - otherwise we'd be back in the "drop false positive" black
hole.  Extending the definition of "hostile" to various kinds of spam=20
would reduce the identification rate, and create a conflict with the
MUST in 3.6.3.  We don't need to talk about the merits of 3.6.3 "as
indicated by the reverse-path" for mails identified as spam, but is
it really clear that accepting spam was an irrevocable *error* at the
border MTA ?

> In practice, you can avoid the "drop the mail if it didn't work" by=20
> making that decision at the SMTP stage as far as possible.

It's not optional, between the lines it is REQUIRED, and there's only
one place for "ordinary" mail to get it right, at the border.  That's
clear for folks on this list, but is it also clear for other readers
of 2821bis ?  =20
=20
> In the original SMTP, 251 or 551 are used when destination=20
> information is incorrect and the SMTP server knows the correct=20
> destination.

Yes, that's the only case of "forwarding" discussed in RFC 821. =20

> This is different from 1123 5.3.6(a) which  talks about alias
> expansion.  We have to keep the envelope sender as is unless we=20
> want to bring back source routing.

There's no problem with local aliases.  And an alias at a third
party is again forwarding, where keeping the return-path "as is"
was indirectly introduced by RFC 1123 5.3.6(a).  The old envelope
sender is not more reponsible for this transaction.  The user who
arranged it and the MTA deciding to try a "251" are responsible.

With RFC 821 reverse routes that was clear, but RFC 1123 5.3.6(a)
removed this responsibility from SMTP.  That was an "undesirable
side effect", IMO it created what's today known as spam problem.

> If we are going to address the spam problem, we have to restrict
> a lot of the mail functionality which 2821 offers.  We'll never
> see 2821bis published if we start that discussion.

Of course 2821bis can't *solve* the spam problem, or reintroduce
reverse routes.  But it should be honest about critical points
in the post-821 design:  Border MTAs accepting dubious mails have
already screwed up badly, and forwarders thinking that "keep the
MAIL FROM 'as is'" was the original design missed that this was a
bug introduced later.

>> I'm also not sure about IPv6 MX and AAAA issues when an IPv6-
>> only sender meets an IPv4-only receiver with the help of an
>> IPvAnything forwarder, a variant of the 1123 5.3.6(a) issue.
=20
> The IPv6-only sender can relay the mail through a host that=20
> supports dual-stack operation.

What I had in mind was mail from an IPv6 only network forwarded
by a dual-stack forwarder to an IPv4 only network.  And at this
point addresses in the forwarded mail, especially any "as is"
MAIL FROM address, violate MUSTard in 2821bis:  The IPv4-only
network cannot report errors to the IPv6-only network.  The user
arranging this can't reply from his IPv4-only network, SNAFU.

For simpler (non-forwarding) scenarios we could find that either
sender, or receiver, or both did something they shouldn't do as
"IPvXOnly" network.  For the forwarding scenario we are back at
square one, keep MAIL FROM "as is" was a design flaw.

 Frank

Date: Mon, 07 Jan 2008 22:11:56 -0800
To: Frank Ellermann <[email-masked]>, [email-masked]
From: SM <[email-masked]>
Subject: Re: Last Call: draft-klensin-rfc2821bis: Permitted and prohibited characters in addresses
Sender: [email-masked]

At 19:36 07-01-2008, Frank Ellermann wrote:
>If RFC 2822 was a product of DRUMS it is IMO a showcase, and if
>RFC 2821 was a product of DRUMS it is a counter-example.  That
>is now ancient history, the reality is:
>
>- RFC 2821 doesn't tell us what to do with IPv6, 2821bis admits
>   that this isn't trivial.
>- RFC 2821 silently assumed that nobody forges envelope sender
>   addresses because that made no sense in the time of 821, and
>   it also made no sense with the design of 821.  But 1123 broke
>   the 821 design, and 2821 did not admit that it is broken.

Quoting Section 3.6.2.:

   "This specification does not deal with the verification of return
    paths for use in delivery notifications.  Recent work, such as that
    on SPF [42] and DKIM [43] [44], has been done to provide ways to
    ascertain that an address is valid or belongs to the person who
    actually sent the message."

>- RFC 2821 silently assumed that "accept, and let's see how it
>   works out" is the best strategy.  But in practice this either
>   means "bounce to a forged envelope sender address" or "drop
>   the mail if it didn't work out".

Section 6.2 talks about this:

   "Conversely, if a message is rejected because it is found to contain
    hostile content (a decision that is outside the scope of an SMTP
    server as defined in this document), rejection ("bounce") messages
    SHOULD NOT be sent unless the receiving site is confident that those
    messages will be usefully delivered."

In practice, you can avoid the "drop the mail if it didn't work" by 
making that decision at the SMTP stage as far as possible.

>- While RFC 2821 inherited the 1123 5.3.6(a) bug 2821bis should
>   admit that "keeping the envelope sender as is" in forwarding
>   to third parties violates an essential point in the original
>   SMTP, the complete transfer of responsibility at all relays,
>   with obvious ways (251+551) to accept or reject this transfer
>   of responsibility.

In the original SMTP, 251 or 551 are used when destination 
information is incorrect and the SMTP server knows the correct 
destination.  This is different from 1123 5.3.6(a) which  talks about 
alias expansion.  We have to keep the envelope sender as is unless we 
want to bring back source routing.

>- The discrepancies about what mailing lists really are (noted
>   in the Last Call) should be also addressed.  Bugs and issues
>   should be fixed or documented.  Just "not talk about it" (if
>   it could hurt the feelings of ML operators) does not cut it.

Mailing lists may use either an alias or a list model.  The 
distinction between the two is that the alias operates by forwarding 
(envelope sender is not rewritten) whereas the list operates by 
redistribution, changing the envelope sender to point to the list 
administrator.

>For that particular issue, assuming that 2822upd can't fix
>it, yes.  Generally I've no good idea, of course 2821bis is
>better than 2821, therefore it should be published.  But it
>doesn't address important problems with the 1123 design for
>a world where almost 90% of the mail traffic is spam.

If we are going to address the spam problem, we have to restrict a 
lot of the mail functionality which 2821 offers.  We'll never see 
2821bis published if we start that discussion.

>I'm also not sure about IPv6 MX and AAAA issues when an IPv6-
>only sender meets an IPv4-only receiver with the help of an
>IPvAnything forwarder, a variant of the 1123 5.3.6(a) issue.

The IPv6-only sender can relay the mail through a host that supports 
dual-stack operation.

Regards,
-sm 

To: [email-masked]
From: "Frank Ellermann" <[email-masked]>
Subject:  Re: Last Call: draft-klensin-rfc2821bis: Permitted and prohibited characters in addresses
Date:  Tue, 8 Jan 2008 04:36:21 +0100
Organization:  <http://purl.net/xyzzy>
Lines: 164
Reply-To: "Frank Ellermann" <[email-masked]>
Sender: [email-masked]

John C Klensin wrote:

> it is clear that the rules are different from the syntax alone.

For details like length restrictions that might be the case, but
for simple questions I expect that the syntax means what it says.

> In other words,=20
>    RCPT TO:<"Acct\ UserID"@domain>     and
>    RCPT TO:<"Acct UserID"@domain>
> are valid (although the first one is discouraged now)

A perfect example where something is NOT clear.  *Unnecessary*
quoted pairs in a quoted string are discouraged, but Alexey has
shown us that <"Acct UserID"@domain> is not allowed, and so the=20
quoted pair in <"Acct\ UserID"@domain> is apparently necessary.

I vaguely recall a discussion about spaces in local parts years
ago in a Geman mail abuse newsgroup, unfortunately I forgot the
result, but if it was "RFC 2822 is only a PS" that would be bad. =20

IMO this is no 2821(bis) issue but a problem for 2822(upd).  And
for EAI, they try to introduce *new* unnecessary quoted pairs,
as if the existing mess (e.g. 3696 errata) isn't bad enough.

>    RCPT TO:<Acct UserID@domain>
> is not and has never been.

ACK =20

> Despite the way the 2821 syntax is constructed, with an
> invocation of "qcontent" which implicitly references a
> definition in 2822 and its use of the contentious NO-WS-CTL,
> there is very clear (to me at least) prose in 2821 that
> prohibits those controls entirely.

NAK, not clear for me.  My interpretation of re-using 2822(upd)=20
rules in 2821(bis), where that's possible, is that the concepts
of "local part", "domain", and "address" must be identical in
2821(bis) and 2822(upd).  Of course you can't import CFWS into
2821(bis), sometimes you're forced to roll your own ABNF.  The
syntax should still be "identical modulo CFWS".

> we don't need to have a further discussion about whether it
> is time to finally get rid of them because we did it during
> DRUMS.

If RFC 2822 was a product of DRUMS it is IMO a showcase, and if
RFC 2821 was a product of DRUMS it is a counter-example.  That
is now ancient history, the reality is: =20

- RFC 2821 doesn't tell us what to do with IPv6, 2821bis admits
  that this isn't trivial.
- RFC 2821 silently assumed that nobody forges envelope sender
  addresses because that made no sense in the time of 821, and
  it also made no sense with the design of 821.  But 1123 broke
  the 821 design, and 2821 did not admit that it is broken.
- RFC 2821 silently assumed that "accept, and let's see how it
  works out" is the best strategy.  But in practice this either
  means "bounce to a forged envelope sender address" or "drop=20
  the mail if it didn't work out". =20
- There's too much cruft in 2821, e.g. the source route business
  is only relevant for folks interested where 1123 broke 821.
  Anything about it could be removed from 2821bis keeping only
  <A-d-l> and <At-domain> as "MUST accept, MUST ignore, MUST NOT
  generate, no need to know what it was decades ago".
- While RFC 2821 inherited the 1123 5.3.6(a) bug 2821bis should
  admit that "keeping the envelope sender as is" in forwarding
  to third parties violates an essential point in the original
  SMTP, the complete transfer of responsibility at all relays,
  with obvious ways (251+551) to accept or reject this transfer
  of responsibility.
- The discrepancies about what mailing lists really are (noted
  in the Last Call) should be also addressed.  Bugs and issues
  should be fixed or documented.  Just "not talk about it" (if
  it could hurt the feelings of ML operators) does not cut it.

> I will leave to others whether or not that raises an=20
> interoperability testing/documentation problem with 2821bis

Or rather for 2822(upd).  SMTP tests could verify that nobody
rejects <"Acct UserID"@domain> as SMTP syntax error, then it's
an issue for 2822(upd).  <recycle> Your original 2005 proposal
to work on 2821bis and 28822upd simultaneously in a proper WG,
after fixing non-controversial errata and some ABNF nits, was
the way to go </recycle>

2821bis and 2822upd are far too important to get them wrong.
It's a bad sign if *new* issues turn up in an IETF Last Call.

  [CTL]
># These characters MUST NOT be used in MAIL or RCPT commands
># or other commands that require mailbox names.
=20
> This seems very clear to me.

It is clear, so let 2822upd fix it where it has other ideas:

If CTL made sense in addresses for other protocols (not SMTP)
2822upd could move NO-WS-CTL to "obs" with a historical note.

If CTL never made sense 2822upd could drop it completely, all
"obs" constructs are still "MUST accept" (outside of Netnews).

For 2821bis I fail to see why it permits DEL in <ehlo-param>,
and why it permits NUL and NO-WS-CTL in <ehlo-greet>.  If the
only justification is "copied from 2821" it is IMO not enough
to violate net-utf8.  Four occurences of 127 in 2821bis, one
explaining why CTL is bad, two excluding CTL but permitting
DEL, and one ugly <ehlo-greet> permitting NO-WS-CTL plus NUL.

> Quoted-string  =3D DQUOTE *qcontentSMTP DQUOTE

A separate solution should be "plan B" if 2822upd is forced=20
(or wishes) to keep something that cannot fly for 2821bis.
And "moved to obs" could still mean "kept" for your purposes.

Unless 2822upd introduces a syntactical finesse allowing to
import clean "non-obs" terms in other RFCs.  DKIM and others
needed a "clean" FWS, unfortunately 2822 only offers FWS in
combination with obs-FWS.  That is only a notorious example,
FWS does not affect 2821bis.

> this exactly restores both the substance and form of the
> 821 rule, adjusted to reflect the prohibition on control
> characters introduced in 2821.

Okay, maybe pick nicer names than <quoted-pairSMTP> etc.,
how about <S-quoted-pair> ? =20

> While (3.a) makes part of it redundant, I suggest leaving
> the paragraph quoted above in the text and intact.  It
> contains the explicit MUST NOT prohibition and may be more
> clear in context than the ABNF comments.

Yes, "make a shorter RFC" was never an objective for 2821bis,
and starting with it here would be strange.

>| The reader is cautioned that the grammar expressed in
>| the metalanguage is not comprehensive.  There are many
>| instances in which provisions in the text constrain or
>| otherwise modify the syntax or semantics implied by the
>| grammar.

<shrug />  Of course the prose and especially 2119 keywords
have a purpose, and stating the obvious can be confusing...
=20
As noted in the Last Call I also fail to understand why you
copied 2119 definitions verbatim instead of referencing the
source.

> Does that help move us forward?

For that particular issue, assuming that 2822upd can't fix
it, yes.  Generally I've no good idea, of course 2821bis is
better than 2821, therefore it should be published.  But it
doesn't address important problems with the 1123 design for
a world where almost 90% of the mail traffic is spam.

I'm also not sure about IPv6 MX and AAAA issues when an IPv6-
only sender meets an IPv4-only receiver with the help of an
IPvAnything forwarder, a variant of the 1123 5.3.6(a) issue.

 Frank

Date: Fri, 04 Jan 2008 14:55:02 -0500
From: John C Klensin <[email-masked]>
To: Philip Guenther <[email-masked]>
cc: [email-masked], [email-masked]
Subject: Re: Last Call: draft-klensin-rfc2821bis: Permitted and prohibited characters in addresses
Sender: [email-masked]

--On Friday, 04 January, 2008 12:03 -0700 Philip Guenther
<[email-masked]> wrote:

> 
> On Fri, 4 Jan 2008, John C Klensin wrote:
> ...
>> 	qtextSMTP            = %d32-91 / %d93-126
>> 	                 ; i.e., within a quoted string, any
>> 	                 ; ASCII graphic or space is permitted
>> 	                 ; without blackslash-quoting except the
>> 	                 ; backslash itself.
> 
> Better make that:
>          qtextSMTP            = %d32-33 / %d35-91 / %d93-126
>                           ; i.e., within a quoted string, any
>                           ; ASCII graphic or space is permitted
>                           ; without blackslash-quoting except
>                           ; double-quote and the backslash
> itself.

Arggh.  yes, thanks.  

>> Does that help move us forward?
> 
> It looks like the right step to me.

Good.  Unless I hear some negative comments RSN, I'll start
modifying the text (changes that can, of course, be pulled out
later if needed).

    john

Date: Fri, 4 Jan 2008 12:03:29 -0700
From: Philip Guenther <[email-masked]>
To: John C Klensin <[email-masked]>
cc: [email-masked], [email-masked]
Subject: Re: Last Call: draft-klensin-rfc2821bis: Permitted and prohibited characters in addresses
Sender: [email-masked]

On Fri, 4 Jan 2008, John C Klensin wrote:
...
> 	qtextSMTP            = %d32-91 / %d93-126
> 	                 ; i.e., within a quoted string, any
> 	                 ; ASCII graphic or space is permitted
> 	                 ; without blackslash-quoting except the
> 	                 ; backslash itself.

Better make that:
         qtextSMTP            = %d32-33 / %d35-91 / %d93-126
                          ; i.e., within a quoted string, any
                          ; ASCII graphic or space is permitted
                          ; without blackslash-quoting except
                          ; double-quote and the backslash itself.

> Does that help move us forward?

It looks like the right step to me.

Philip Guenther

Date: Fri, 04 Jan 2008 12:55:09 -0500
From: John C Klensin <[email-masked]>
To: [email-masked]
cc: [email-masked]
Subject: Re: Last Call: draft-klensin-rfc2821bis: Permitted and prohibited characters in addresses
Sender: [email-masked]

Hi.

I'm not copying the IETF list on this because the discussion has
already gotten too noisy as a consequence of people not reading
the text carefully enough (sadly enough, myself included).

First, please remember that the syntax of an address in 821 and
its successors have always been more restrictive than the
address syntax of 822 and it lineage.  The most obvious example
is that 822 permits personal name phrases and 821 does not.
While some of the other differences are probably just historical
accidents, others are due to the fact that, at one time, a large
number of non-Internet mail systems used an 822, or 822-like,
header structure and it seemed better to include than to ban
them.   Once messages passed from those systems into the public
Internet, the more restrictive Internet rules for the mailbox
name itself applied.  The text about gateways in RFC 1123
(especially Section 5.3.7) and text in 2821 should make that
abundantly clear.

In principle, we could announce that non-Internet mail
(presumably other than things covered by USEFOR) is ancient
history and go back and eliminate the differences.  But, even if
there were not philosophical reasons to not try to do that (and
I think there are), trying to get all the ducks lined up, verify
that none of the edge cases are problems and that all of the
"conform only to one" systems are completely gone (or otherwise
account for them) is not a trivial matter and is certainly not
appropriate for a Proposed -> Draft transition.   The
differences are inconvenient for those who believe there aren't
any; they are not a sign that anything is broken.

Second, while we have been moving increasingly in the direction
of having as much specification in syntax productions as
possible and in having those productions constitute a complete
grammar of the relevant protocol, there has never been a
requirement that specifications be written that way.   RFC 821
was certainly not written that way and 2821 isn't either.  For
them, there is no top-level note from which everything else
descends.  Syntax productions are used to _illustrate_ what a
command looks like, not to completely define its grammar.  The
grammar is defined by a combination of

	* syntax productions
	* narrative rules imbedded as comments in the syntax
	   tables
	* prose text in the document that restricts or 
	  qualifies the syntax productions.

In the particular case of permitted characters in mailbox
local-parts, the transformation between the BNF of 821 and the
revised ABNF of 2821 and the desire to minimize the number of
basic grammar definitions that were duplicated between 2821 and
2822 seems to have introduced some additional confusion.  But,
because of the prose (if anyone actually reads it), it is clear
that the rules are different from the syntax alone.

In particular:

(1) Alexey, as Frank points out, there is no prohibition on the
use of space in local-parts in 2821, but those spaces must be
quoted. In other words, 
   RCPT TO:<"Acct\ UserID"@domain>     and
   RCPT TO:<"Acct UserID"@domain>
are valid (although the first one is discouraged now) but
   RCPT TO:<Acct UserID@domain>
is not and has never been.  See (3) below before you reread the
current text.

(2) Despite the way the 2821 syntax is constructed, with an
invocation of "qcontent" which implicitly references a
definition in 2822 and its use of the contentious NO-WS-CTL,
there is very clear (to me at least) prose in 2821 that
prohibits those controls entirely.  That is, we don't need to
have a further discussion about whether it is time to finally
get rid of them because we did it during DRUMS.   I will leave
to others whether or not that raises an interoperability
testing/documentation problem with 2821bis, but note that 2821
(and 2821bis) are written to simply move excessive
permissibility outside the scope of the standard and not to
prohibit them.  The relevant text, from Section 4.1.2 of 2821bis
(both the text and the section number are identical in 2821)
reads:

# Systems MUST NOT define mailboxes in such a way as to
# require the use in SMTP of non-ASCII characters (octets with
# the high order bit set to one) or ASCII "control characters"
# (decimal value 0-31 and 127).  These characters MUST NOT be
# used in MAIL or RCPT commands or other commands that require
# mailbox names.

This seems very clear to me.  While it introduces an additional
issue for the EAI work (already noted on their mailing list and,
by definition, not a 2821bis problem), it is not an issue here.

(3) While I think this all works, the observation that it has
confused, in no particular order and among others, Alexey about
spaces; Frank about NO-WS-CTL in 2821; Ned, Pete, and myself
about permitting control characters in local-parts of mailbox
names suggests to me that it is just too confusing for normal
readers. Consequently, I suggest the following two changes to
2821bis while emphasizing that these change the way in which the
specification is defined but not the spec itself.

(3.a) Change the definition of Quoted-string in 2821bis more or
less as follows (the author of the 2821 ABNF is asked to improve
this if possible).

	Quoted-string  = DQUOTE *qcontentSMTP DQUOTE
	
	QcontentSMTP   = qtextSMTP / quoted-pairSMTP
	
	quoted-pairSMTP      = %d92 %d32-126
	                 ; i.e., backslash followed by any
	                 ; ASCII graphic (including itself) or
	                 ; SPace
	
	qtextSMTP            = %d32-91 / %d93-126
	                 ; i.e., within a quoted string, any
	                 ; ASCII graphic or space is permitted
	                 ; without blackslash-quoting except the 
	                 ; backslash itself.

Unless I've missed something, this exactly restores both the
substance and form of the 821 rule, adjusted to reflect the
prohibition on control characters introduced in 2821.  Since the
paragraph quoted above prohibits an SMTP-sender from ever
sending a mailbox string containing control characters, we don't
even need a way to escape them any more.

I'll leave it to Pete to straighten this situation out in
2822bis, nothing that 2821 does not contain the clear
prohibition on the use of control characters in addresses that
2821 does.

(3.b) While (3.a) makes part of it redundant, I suggest leaving
the paragraph quoted above in the text and intact.  It contains
the explicit MUST NOT prohibition and may be more clear in
context than the ABNF comments.

(3.c) In order to reduce the odds of another round of this sort
of problem, I recommend adding the following sentences to the
last paragraph of Section 2 (the paragraph that starts "The
metalinguistic notation used..."):

		The reader is cautioned that the grammar expressed in
		the metalanguage is not comprehensive.  There are many
		instances in which provisions in the text constrain or
		otherwise modify the syntax or semantics implied by the
		grammar.

Does that help move us forward?

     john

Date: Wed, 02 Jan 2008 11:03:38 -0800
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
Subject: Re-query and plea:  RFC2821 Implementation Survey
Sender: [email-masked]

Folks,

      Felìz Año Nuevo.

      I am sending the following again, in the hope that anyone who considers
their SMTP software to be a major industry component will view it as a matter
of industry responsibility to respond.  In other words, We've gotten far too
few responses from folks with SMTP products and services.

      RFC2821 cannot advance on the IETF standards track without these
implementation reports.

      So if you think your company's implementation of SMTP is important to
the global Internet's email service, please take the time to complete the
following survey...

Greetings.

The IETF will shortly be considering elevating the revised SMTP specification
RFC 2821, to the second-level standards status of DRAFT.

A requirement for this is to produce information about implementations ofthe
protocol specification.  It is therefore essential that we assemble a credible
collection of information about actual development and interoperability
testing -- and deployment is a form of testing -- for RFC 2821.

Because SMTP is a rich and widely-deployed protocol, the goal of this query is
to ask for enough information to be useful, but not to place undue burdenon
responders by asking them to do a massive checklist.

It is assumed that most implementations of RFC 2821 have implemented all of
its required (MUST) features and probably a fair number of its optional
(SHOULD or MAY) features.

This survey does NOT cover any specifications outside of the RFC2821 document
itself.  No enhancements, options, or the like.

Please take some time to complete the following questionnaire and send mea
private note with your responses.  I will assemble them into a public report:

Please respond by 10 December 2007.

RFC 2821 Implementation Questionnaire
=====================================

    0. Contact and Description

       Organization Name:

       Implementation (Software or Service) Name:

    1. Have you implemented RFC2821?

    2. For how long it has been deployed?

    3. What features have NOT been implemented from RFC2821?

    4. What features of RFC2821 are problematic for your implementation?

    5. Please add any other comments you wish to share:

Thank you!

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net