Subject: [EAI] IMAP UTF8 draft

From: Timo Sirainen <[email-masked]>
Date: Wed, 14 Apr 2010 00:29:47 +0300

The current method of always up/down-converting data breaks digital signatures. It should be possible to fetch signed messages with 7bit encoded and/or UTF8 headers without breaking them (worst case: mailing list digest message containing both). Clients that are going to support UTF8 headers will probably internally support up-conversion anyway. Maybe the IMAP UTF8 support should be less ambitious? Like:

When mailbox is selected with UTF8 flag, it only up-converts the following commands:

 - FETCH BODY
 - FETCH BODYSTRUCTURE
 - FETCH ENVELOPE

Other than that, it just passes the data as it was originally received, with or without UTF8 headers. That way the IMAP client can verify digital signatures correctly.

I could maybe also see these new commands useful:

 - FETCH BODY[HEADER.UTF8]
 - FETCH BODY[HEADER.FIELDS.UTF8 (..)]
 - FETCH BODY[HEADER.FIELDS.NOT.UTF8 (..)]
 - FETCH BODY[n.MIME.UTF8]
 - FETCH BODY[n.HEADER.UTF8]

The reason for the added .UTF8 is mainly to avoid confusion with message part sizes not matching as expected. For example RFC822.SIZE might not be size of BODY[HEADER.UTF8] + BODY[TEXT.UTF8]. These anyway add more server complexity and I'm not sure how useful they'll really be for clients. And if all goes well, in distant future everyone is using UTF8 and these commands will be useless then.

Anyway, even with those extra commands I think this simplifies server implementations enough so that [NOT-UTF-8] failure and \NoUTF8 LIST flag and UTF8=ALL capability could all be dropped completely.

Another thing:  If UTF8=ONLY is going to be realistic only in distant future, couldn't it be left into distant future? By then we might be talking about IMAP5 capability already. It's confusing to have so many UTF8=* capabilities.

---

From: Timo Sirainen <[email-masked]>
Date: Wed, 14 Apr 2010 17:05:46 +0300

On Wed, 2010-04-14 at 00:29 +0300, Timo Sirainen wrote:
> The current method of always up/down-converting data breaks digital
> signatures. It should be possible to fetch signed messages with 7bit
> encoded and/or UTF8 headers without breaking them (worst case: mailing
> list digest message containing both). 

Oh, it was mentioned in the draft after all. I did do a search for
"signature" before sending this mail, but it didn't find anything. :)
Anyway, the point is still pretty much the same. If you're not
up-converting signed MIME parts, why bother doing it for the rest of
them? Clients will have to handle it anyway.

Another point: IMAP requires that msg-att-static fields never change for
messages. UTF-8 up-conversion isn't defined in a way that guarantees
that upgrading a server version won't change the up-conversion code (bug
fixes, more features). So either servers will a) violate this "never
changes" requirement, b) have to cache old up-converted messages forever
(wasting disk space) or c) keep multiple versioned up-conversion
functions and remember which version was used to up-convert which old
message. All of these solutions seem pretty bad to me.

---

From: John C Klensin <[email-masked]>
Date: Wed, 14 Apr 2010 10:05:42 -0400

--On Wednesday, 14 April, 2010 00:29 +0300 Timo Sirainen
<[email-masked]> wrote:

> The current method of always up/down-converting data breaks
> digital signatures. It should be possible to fetch signed
> messages with 7bit encoded and/or UTF8 headers without
> breaking them (worst case: mailing list digest message
>...

Timo,

Without taking a position about the strengths or weaknesses of
your idea, I think it would be very helpful if you would write
down the details and mechanisms, and explore the implications,
in an I-D.  We often find that discussions are much better when
there is a specific proposal on the table.

    john