Subject: AS2 canonicalization for MIC calculation ?

From: "Caro Sebastien" <[email-masked]>
Date: Fri, 16 Jan 2004 11:11:14 +0100

It is said in AS2 section 7.3.1 (Signed receipt considerations) that in some 
cases canonicalization must be performed before the MIC is calculated.
No reference is given on where to find the canonicalization process:
- AS1 refers to RFC 1848
- MIME defines canonical form as the result of steps where one of them rely 
  on media type specific representation (RFC 2049 section 4)
- S/MIME refers to MIME

RFC 1848 (section 2.1.1) isn't very clear, but it says that the originator 
must replace local line terminators by CRLF before computing the digital 
signature and then send data with the local line terminators. If such a thing
is done to transfer data between 2 different systems, it is obvious that the 
receiver will not know the originator line terminator, and he will therefore 
not be able to re-insert CRLF at the right places.

RFC 1767 and RFC 2376 do not introduce any canonical representation.
By the way, RFC 2376 has been obsoleted by 3023, shouldn't AS2 RFC use 
references to this document instead?

S/MIME says that 'The data to be secured is always a canonical MIME entity'.
So, signed or ciphered data is sent already canonicalized and it doesn't seem 
necessary for the receiver to canonicalize once more the received data.

Still in AS2 section 7.3.1, 
> - For any signed messages, the MIC to be returned is
> calculated on the RFC1767/RFC2376 MIME header and content.
> Canonicalization on the MIME headers MUST be performed
> before the MIC is calculated, since the sender requesting
> the signed receipt was also REQUIRED to canonicalize.

> - For encrypted, unsigned messages, the MIC to be returned
> is calculated on the decrypted RFC 1767/RFC2376 MIME header
> and content. The content after decryption MUST be
> canonicalized before the MIC is calculated.

The MIME header is supposed to be in a canonical form, containing CRLFs. 
The sent data is contained in a MIME entity which is itself inside the request 
body. HTTP request bodies are not to be altered.

I am a bit confused by the amount of information I found and would appreciate 
clarification.

Sebastien

---

From: "Dale Moberg" <[email-masked]>
Date: Fri, 16 Jan 2004 08:11:51 -0700

Sebastien observes>
It is said in AS2 section 7.3.1 (Signed receipt considerations) that in
some 
cases canonicalization must be performed before the MIC is calculated. 

Canonicalization of _headers_ (mainly the MIME entity headers for MIC
calcs, of course)

For uniformity, canonicalization of MIME headers is always required.
Implementations could optimize whether they need to run a check in some
cases, though. Canonicalization of headers means, then, that they must
have CRLF at the end of lines and as a separator between header and
body. This iz just the MIME ABNF rules (RFC 2822, section 2.1, 2.2), 821
appendix, 2045, etc. 

message     =  fields *( CRLF *text )       ; Everything after
                                                 ;  first null line
                                                 ;  is message body

field       =  field-name ":" [ field-body ] CRLF

[...]

entity-headers := [ content CRLF ]
                    [ encoding CRLF ]
                    [ id CRLF ]
                    [ description CRLF ]
                    *( MIME-extension-field CRLF )

Sebastien observes>
RFC 1848 (section 2.1.1) isn't very clear, but it says that the
originator 
must replace local line terminators by CRLF before computing the digital

signature and then send data with the local line terminators. 

Well, the MIME conventions of HTTP and SMTP messages differ
in several ways previously discussed in this group, and elsewhere, and
in various contexts. RFC 1848 (1995) is oriented toward "SMTP-style"
MIME. It would be misleading to reference it for HTTP, and this is not
done, nor is it necessary or desirable. It would introduce more
confusion over what is really a very simple point.

Sebastien observes>

Yes.

Sebastien observes>
S/MIME says that 'The data to be secured is always a canonical MIME
entity'. So, signed or ciphered data is sent already canonicalized and
it doesn't seem 
necessary for the receiver to canonicalize once more the received data.

An implementation may choose to optimize this case. Nevertheless, for
uniformity, the specese is that for MIC calculations MIME headers must
be canonicalized (CRLF), and 
what are MIME headers will be syntactically clear in HTTP MIME. 

Sebastien observes>
> - For any signed messages, the MIC to be returned is calculated on the
> RFC1767/RFC2376 MIME header and content. Canonicalization on the MIME 
> headers MUST be performed before the MIC is calculated, since the 
> sender requesting the signed receipt was also REQUIRED to 
> canonicalize.

Yes, very few people have found it unclear what canonicalization of MIME
headers means. Are you saying that we should be more explicit here and
mention CRLF as end of line and as header/body separator? We can do so
in a parenthetical remark. I personally see no real need to cite a
spec. This is more a warning to implementers that CRLF may get munged,
so always calculate the MIC over the ABNF-correct MIME headers. I guess
the syntax for headers is the reference you hanker to have? Would RFC
2822 suffice?

> is calculated on the decrypted RFC 1767/RFC2376 MIME header and
> content. The content after decryption MUST be canonicalized before the

> MIC is calculated.

Caro Sebastien observes>

The sent data is contained in a MIME entity which is itself inside the
request 
body. HTTP request bodies are not to be altered.

Yes, and alteration [other than change of line endings!] should be
detectable when signatures exist.

---

From: "Caro Sebastien" <[email-masked]>
Date: Fri, 16 Jan 2004 20:06:28 +0100

Thanks, things are a bit clearer now, 

I took things literally where it is said that canonicalization MUST be
performed to the received data.

It is clear that signed data cannot have been altered. So, the received
MIME header (RFC1767/RFC2376) is the exactly one sent by the originator.
- If the originator has respected MIME standard, the received header 
contains CRLFs, is blank line terminated... It is already canonicalized.
Canonicalization will then not change anything for the MIC calculation.
- If the sender hasn't respected MIME standard and sent lines not ending
with CRLF, the result of the canonicalization will depend on the 
receiver interpretation of things. The receiver can't know anything 
about the originator's local properties. Maybe a canonicalization 
consisting of replacing bare CR and bare LF with CRLF will handle most 
of the cases, but it won't be a 100% sure process for MIC calculation.

For encrypted, unsigned messages, the MIC is calculated on the 
canonicalization of the whole content (decrypted RFC 1767/RFC2376 MIME 
header and content).
I will refer to the above paragraph for the header canonicalization. 
Body part canonicalization consists in converting data to a canonical 
form. It seems that nothing can be done by the receiver to convert 
application/EDIxxx data, or application/xml data, to a canonical form.

I might be wrong, but my reasoning is mostly based on the fact that data 
cannot be modified, and, that if it has, there is no way of recovering 
the original data (which I see here as the right canonical form).

I am really doubting that the result of the canonicalization on received
data will correctly lead to a valid MIC. It seems that, in the best 
cases, data will not be modified by it (canonicalization will then not 
intervene in the MIC calculation).

Maybe the mentioned lines in section 7.3.1 have been written to require
that, before it is sent, data must canonicalized and a MIC must be
calculated. The MIC will later be compared with the one returned by the
receiver. The facts are that it is explicitly written that the 
canonicalization MUST be applied by the receiver to the received data.

I totally agree on the importance of noticing implementers that data
MUST be canonicalized.

Sebastien

-----Message d'origine-----
De : Dale Moberg [mailto:[email-masked]]
Envoyé : vendredi 16 janvier 2004 16:12
À : Caro Sebastien; [email-masked]
Objet : RE: AS2 canonicalization for MIC calculation ?

Sebastien observes>

[...]

Sebastien observes>

Sebastien observes>

Yes.

Sebastien observes>

Sebastien observes>

Caro Sebastien observes>