Subject: [6lowpan] RFC 4944 fragmentation and future extensions

From: Richard Kelsey <[email-masked]>
Date: Tue, 16 Feb 2010 14:52:35 -0500

There is a forward-compatibility problem lurking in the
fragmentation specified by RFC 4944.  In the RFC, fragment
headers include sizes and offsets that are relative to the
uncompressed datagram, and not the compressed version that
is actually being communicated.  If a node receives a
fragmented datagram which includes a compressed next header
that the node does not know how to uncompress, the node may
not be able to determine if it has received all of the
fragments.  It knows neither the length of the compressed
datagram or the total number of fragments.

Further, the RFC states:

   Similarly, when a node first receives a fragment with a
   given datagram_tag, it starts a reassembly timer.  When
   this time expires, if the entire packet has not been
   reassembled, the existing fragments MUST be discarded and
   the reassembly state MUST be flushed.

This requirement assumes that the receiver is able to
determine when it has received a complete datagram.

My main concern is the possibility of compressing TCP
headers at some time in the future.  Pre-existing devices
would not necessarily be able to forward packets with
compressed TCP headers.

This could be fixed in several ways, such as using the
compressed sizes and offsets instead of the uncompressed
ones, or including the number of fragments in the first
fragment header.
                         -Richard Kelsey

---

From: Owen Kirby <[email-masked]>
Date: Tue, 16 Feb 2010 13:04:53 -0800

Richard,

We've been grappling with this problem internally over the last week or 
so, and I'd like to add our take on some of the problems in combining 
RFC4944 fragmentation with the draft header compression.

I think the RFC4944 fragmentation was optimized specifically for HC1 
compression under the assumption that the compressed headers never 
extend beyond the first fragment of the packet. If we assume that all 
subsequent fragments are uncompressed then you can quite easily tell 
when you have received all of the fragments, and the last fragment will 
have (dgram_offset + fragment size == dgram_size). The only oddity is 
that you have to decompress the first fragment before you can reassemble 
the datagram.

However, I suspect that this assumption is no longer true in the case of 
the draft header compression. Especially now that IPv6 extension headers 
can be compressed (what do we do if ROLL adds ~160 bytes of routing 
headers to a packet?). It becomes substantially more complicated to 
determine how to defragment packets since we need to know the 
uncompressed length of each header in order to know when reassembly is 
done and to detect missing fragments, but we can't decompress the 
headers until we have all of the compressed headers. We get stuck in a 
catch-22: can't defragment until after decompression, but can't 
decompress until after defragmentation.

|First Fragment (F1)      |Second Fragment (F2)   |Third Fragment (F3)
+---------------------+   +-------------------+   +--------+--------------+
| Compressed Headers  |   |Compressed Headers |   | C Hdrs | Payload      |
+---------------------+   +-------------------+   +--------+--------------+
:                      \  :                    \  :
:                       \ :                     \ :
|Uncompressed Datagram   \:                      \:
+-------------------------:-----------------------:--------+--------------+
|              Uncompressed Headers                        | Payload      |
+-------------------------:-----------------------:--------+--------------+
|                         |                       |                       |
|<---        L1       --->|<---       L2      --->|<---       L3      --->|
|                                                                         |
|<--------------------------    datagram_size     ----------------------->|

The only way to reassemble these three packets is to perform 
decompression and reassembly at the same time, since the lengths and 
offsets of fragments F1, F2, and F3 do not add up to the datagram size, 
and leave gaps in the packet that hopefully get filled after 
decompression. It also becomes necessary to store the state of your 
decompressor across multiple packets, which can get very ugly if header 
fields get split across fragments.

Either one of your recommended solutions would work, but if you want to 
include the total number of fragments I would recommend that each packet 
should also be identified by a sequence number. A simple, but less 
efficient solution, could also be to simply state that any headers which 
extend past the first fragment after compression should not be compressed.

Also, as the author of the 6LoWPAN dissectors for Wireshark, I'm pulling 
my hair out trying to find a way to implement this without throwing away 
all of Wireshark's reassembly support and starting from scratch, or 
implementing some sort of wild speculative guessing.

Thanks,
Owen Kirby

Richard Kelsey wrote:
> _______________________________________________
> 6lowpan mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/6lowpan
>

---

From: Richard Kelsey <[email-masked]>
Date: Wed, 17 Feb 2010 13:06:07 -0500

> Date: Tue, 16 Feb 2010 13:04:53 -0800
> From: Owen Kirby <[email-masked]>
> 
> However, I suspect that this assumption is no longer true
> in the case of the draft header compression. Especially
> now that IPv6 extension headers can be compressed (what do
> we do if ROLL adds ~160 bytes of routing headers to a
> packet?).

Owen,

A long source route could quite possibly push the transport
header into the second fragment.

> [...]  We get stuck in a catch-22: can't
> defragment until after decompression, but can't decompress
> until after defragmentation.
> 
> [...] It also becomes necessary to store the state of your 

This could be tricky because the size of the compressed IP
header and the MAC addressing and security overhead can
change hop-by-hop.  What was in the first fragment on the
first hop may not be in the first fragment on a subsequent
hop, which causes a problem if the forwarding node does
not know how to uncompress the entire packet.

I think that the best solution would be to switch to using
the compressed size and offsets in the fragment headers.
This would allow (un)compression and (de)fragmentation to
be done independently.

> implementing some sort of wild speculative guessing.

Would using the compressed size and offsets in the fragment
headers solve this?
                                      -Richard Kelsey

----------------
This message and the information it contains are the proprietary
and confidential property of Ember Corporation and may be privileged.
If you are not the intended recipient, please do not read, copy,
disclose or distribute its contents to any party, and notify the
sender immediately.

---

From: Owen Kirby <[email-masked]>
Date: Wed, 17 Feb 2010 11:36:44 -0800

Richard,

> not know how to uncompress the entire packet.
Perhaps I could use a little clarification on how reassembly and 
forwarding would work in a 6LoWPAN mesh. Please correct me if I've 
gotten anything wrong here.

In a route-over network, the IPv6 packet would need to be reassembled 
and decompressed at every hop to decrement TTL and check the routing 
headers. In this case, the forwarding node has control over all of the 
fields and can change a compressed field into an uncompressed field (and 
vice-versa) for the next hop in the route. As I suggested, this is not 
the most efficient solution, but it works and requires the least amount 
of change from what's in the RFC.

In the mesh-under case, the fragments get routed across the network 
using the mesh routing header in RFC4944, and reassembly only happens at 
the packet's final destination, the contents of the fragments can't be 
modified in-flight anyways, so I don't see how this would cause a 
problem, and changes in the MTU are something that the source would need 
to allow for anyways.

FWIW, all of the 6LoWPAN implementations I know of work on the 
assumption that compressed headers are always contained within the first 
fragment only.
> This would allow (un)compression and (de)fragmentation to
I absolutely agree.

> Would using the compressed size and offsets in the fragment
Yes, this change would be very easy to implement.

Thanks,
Owen Kirby

---

From: Richard Kelsey <[email-masked]>
Date: Wed, 17 Feb 2010 15:34:16 -0500

Owen,

> Date: Wed, 17 Feb 2010 11:36:44 -0800
> headers.

In theory, only the IPv6 header and any routing headers need
to be uncompressed at each hop.  The transport header should
not need to be uncompressed until the destination is
reached.

> In this case, the forwarding node has control over all of the 

Isn't that how RFC 4944 works today?  What needs to be
changed?

It works as long as the relaying node is able to completely
uncompress the packet.  Should anyone want to add a new
transport header compression later on, such as for TCP,
existing devices would not be able to relay packets that
used the new compression.  This is the forward-compatibility
problem that was the subject of my original message.

The use of the uncompressed length and offsets in the
fragment headers adds a lot of implementation complexity,
but I do think that it works, as long as relaying nodes
understand all of the compressions being used.

In other words, while changing the fragmentation to make it
simpler to implement would be nice, forward compatibility is
the reason that I think we must do so.

> problem, and changes in the MTU are something that the source would need 

Mesh-under networks do not necessarily use the mesh routing
header in RFC4944.  I suspect that similar problems could
come up in a mesh-under network, depending on the details of
how forwarding is accomplished.

> assumption that compressed headers are always contained within the first 

As you pointed out earlier, that probably isn't a safe
assumption when using the HC draft.  In particular, source
routes could easily push some or all of a compressed header
into the second fragment.
                               -Richard Kelsey

---

From: Daniel Gavelle <[email-masked]>
Date: Thu, 18 Feb 2010 12:34:15 +0000

The main problems with the existing RFC4944 / HC-06 draft when the
compressed data can extend over more than one header are:-

(1) Detecting the subsequent fragments are compressed.

(2) That it is not possible to detect the final fragment by 
(dgram_offset + fragment size == dgram_size).

These problems can solved by creating new dispatch codes.

We currently have 11 000 for first fragment FRAG1 and 11 100 for
subsequent fragments FRAGN.  A new code can be created for a subsequent 
fragment that contains compressed header information, e.g. 11 101 FRAGC.

The problem of detecting the final fragment does not occur for 
implementations that use a bit map manager. A new final fragment code of 
11 110 FRAGF can be added to assist implementations that detect the 
final fragment by (dgram_offset + fragment size == dgram_size).

Changing the datagram_size and datagram_offset to refer to the
compressed rather than the uncompressed sizes causes the following 
problems:-

(a) It would force major changes on existing implementations.  It would 
also completely break any compatibility with existing fragmentation code 
which has already been developed by contributors on this list.

(b) It will make determination of whether a packet is complete more 
difficult for implementations which use a bitmap based on uncompressed 
offsets.

(c) Storing the entire packet before decode / forwarding should be an 
implementation choice rather than something that is enforced because of 
the way the standard is written. Copying from one buffer to another is 
wasteful and not a luxury that an embedded engineer can count on, PC 
implementations aren't going to feel the pain in the same way.  It is 
important that 6LP can be implemented in real-time with a minimum of 
buffer copies - all that is needed is a buffer capable of storing 2 (or 
more) fragments to get the system working when header creep is detected. 
  When forwarding packets, a router node may need to deal with several 
packets at once and may not have space to fully buffer each IP datagram.

(d) In the most common case, there is only fragmentation in the first 
header and then many fragments of uncompressed UDP payload.  The 
existing method of specifying the uncompressed offset allows this UDP 
payload to be stored in the correct location as the incoming data arrives.

(e) Forwarding unknown headers through router nodes in a route over
implementation can be solved whether the length and offset are stored 
compressed or uncompressed. There are different advantages in this case 
with either method so we shouldn't let this cloud the issue.

(f) Not knowing the full uncompressed datagram size until the entire 
packet has been decoded also has implications for ICMP error handling, 
e.g. MTU exceeded.

Daniel.

__________________________________________________
Daniel Gavelle, Software Engineer
Tel: +44 114 281 2655
Fax: +44 114 281 2951
Jennic Ltd, Furnival Street, Sheffield, S1 4QT, UK
Comp Reg No: 3191371  Registered In England
http://www.jennic.com
__________________________________________________

--

---

From: Jonathan Hui <[email-masked]>
Date: Thu, 18 Feb 2010 08:24:16 -0800

I think that changing offset and length to indicate the *compressed*  
values is the best way forward.  It is architecturally clean because  
it preserves the layering - the only information needed to generate  
fragments is the data and length.  RFC 4944's use of uncompressed  
values requires the fragmentation layer to be aware of the specific  
header compression in use and creates dependencies, which is the  
primary cause of the issues described by Richard and Owen.

The primary downsides that I see are the following:
- Using compressed values does mean that a memmove() operation is  
likely when expanding the payload, but a memmove() on ~1K bytes does  
not represent a bottleneck relative to the 15.4 radios in use today.
- The receiver no longer knows how much buffer to allocate in order to  
store the fully expanded payload (if that is what the implementation  
wants to do).  Fortunately, the forwarding nodes are only concerned  
with L3 headers and the size difference is currently limited to the  
IPv6 header itself.  Of course, the size difference on the final  
destination could be arbitrary depending on what other upper-layer  
compression is in use.  We could include a expanded size field in each  
fragment, but I'm not sure the overhead outweighs the benefits.
- Breaks existing implementations - I don't think we should be  
terribly concerned with this - most of the implementation work going  
on in the world is happening right now, standards at all layers of the  
stack are still being defined.

I would suggest placing the appropriate text for whatever we decide on  
into the 6lowpan-hc draft.  The draft already updates RFC 4944 and the  
change is relevant to how 6lowpan-hc is defined.  I see this as the  
quickest way forward even though we would have to reset the WGLC on it.

Another issue that comes up with Richard's forward compatibility  
concerns is handling of ICMPv6 errors - specifically, when a forwarder  
needs to generate an error, what should it include in the ICMPv6  
payload?  Though I think this should be left for a different thread.

--
Jonathan Hui

On Feb 18, 2010, at 4:34 AM, Daniel Gavelle wrote:

> subsequent fragments FRAGN.  A new code can be created for a  
> subsequent fragment that contains compressed header information,  
> e.g. 11 101 FRAGC.
>
> implementations that use a bit map manager. A new final fragment  
> code of 11 110 FRAGF can be added to assist implementations that  
> detect the final fragment by (dgram_offset + fragment size ==  
> dgram_size).
>
> (a) It would force major changes on existing implementations.  It  
> would also completely break any compatibility with existing  
> fragmentation code which has already been developed by contributors  
> on this list.
>
> difficult for implementations which use a bitmap based on  
> uncompressed offsets.
>
> (c) Storing the entire packet before decode / forwarding should be  
> an implementation choice rather than something that is enforced  
> because of the way the standard is written. Copying from one buffer  
> to another is wasteful and not a luxury that an embedded engineer  
> can count on, PC implementations aren't going to feel the pain in  
> the same way.  It is important that 6LP can be implemented in real- 
> time with a minimum of buffer copies - all that is needed is a  
> buffer capable of storing 2 (or more) fragments to get the system  
> working when header creep is detected.  When forwarding packets, a  
> router node may need to deal with several packets at once and may  
> not have space to fully buffer each IP datagram.
>
> (d) In the most common case, there is only fragmentation in the  
> first header and then many fragments of uncompressed UDP payload.   
> The existing method of specifying the uncompressed offset allows  
> this UDP payload to be stored in the correct location as the  
> incoming data arrives.
>
> implementation can be solved whether the length and offset are  
> stored compressed or uncompressed. There are different advantages in  
> this case with either method so we shouldn't let this cloud the issue.
>
> packet has been decoded also has implications for ICMP error  
> handling, e.g. MTU exceeded.
>
> Daniel.
>
>>> decompressor across multiple packets, which can get very ugly if  
>>> header fields get split across fragments.
>>>
>>> Either one of your recommended solutions would work, but if you  
>>> want to include the total number of fragments I would recommend  
>>> that each packet should also be identified by a sequence number.   
>>> A simple, but less efficient solution, could also be to simply  
>>> state that any headers which extend past the first fragment after  
>>> compression should not be compressed.
>>> Also, as the author of the 6LoWPAN dissectors for Wireshark, I'm  
>>> pulling my hair out trying to find a way to implement this without  
>>> throwing away all of Wireshark's reassembly support and starting  
>>> from scratch, or implementing some sort of wild speculative  
>>> guessing.

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Thu, 18 Feb 2010 17:34:55 +0100

Hi Daniel, Richard and Owen:

There are a number of reasons why the fragmentation in RFC 4944 is
broken. Richard has just identified a new one. The traditional problem
of loss of fragment is another that might actually hurt a lot sooner. It
is time that we rework the fragment in the light of this experience and
without the need to be backward compatible but rather forward
compatible. We did that for HC already.

http://tools.ietf.org/html/draft-thubert-6lowpan-simple-fragment-recover
y already has the sequence that Owen has been asking for, for the
purpose of recovering fragments. 
It also has an all zeroes ack bitmap that enables the receiver to let
the sender know that the packet is dropped so stop sending fragments for
it; this can be used if the receiver fails to parse a compressed header.
It is compatible with a route over model where fragments are forwarded
without full recomposition ( see
http://www.mail-archive.com/[email-masked]/msg01828.html ).

So my suggestion would be to work on that draft to make it achieve what
you're looking for. What do you think?

Pascal

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On
Behalf Of Daniel Gavelle
Sent: Thursday, February 18, 2010 1:34 PM
To: Richard Kelsey
Cc: [email-masked]; Wayne Ellis
Subject: Re: [6lowpan] RFC 4944 fragmentation and future extensions

(d) In the most common case, there is only fragmentation in the first
header and then many fragments of uncompressed UDP payload.  The
existing method of specifying the uncompressed offset allows this UDP
payload to be stored in the correct location as the incoming data
arrives.

Daniel.

__________________________________________________
Daniel Gavelle, Software Engineer
Tel: +44 114 281 2655
Fax: +44 114 281 2951
Jennic Ltd, Furnival Street, Sheffield, S1 4QT, UK Comp Reg No: 3191371
Registered In England http://www.jennic.com
__________________________________________________

Richard Kelsey wrote:
>> However, I suspect that this assumption is no longer true in the case

>> of the draft header compression. Especially now that IPv6 extension 
>> headers can be compressed (what do we do if ROLL adds ~160 bytes of 
>> routing headers to a packet?).
> 
> Owen,
> 
> A long source route could quite possibly push the transport header 
>> [...]  We get stuck in a catch-22: can't defragment until after 
>> decompression, but can't decompress until after defragmentation.
>>
>> Either one of your recommended solutions would work, but if you want 
>> to include the total number of fragments I would recommend that each 
>> packet should also be identified by a sequence number.  A simple, but

>> less efficient solution, could also be to simply state that any 
>> headers which extend past the first fragment after compression should
not be compressed.
> 
> This could be tricky because the size of the compressed IP header and 
> the MAC addressing and security overhead can change hop-by-hop.  What 
> was in the first fragment on the first hop may not be in the first 
> fragment on a subsequent hop, which causes a problem if the forwarding

> node does not know how to uncompress the entire packet.
> 
> I think that the best solution would be to switch to using the 
> compressed size and offsets in the fragment headers.
> This would allow (un)compression and (de)fragmentation to be done 
> independently.
> 
>> throwing away all of Wireshark's reassembly support and starting from

>> scratch, or implementing some sort of wild speculative guessing.
> 
> Would using the compressed size and offsets in the fragment headers 
> solve this?
>                                       -Richard Kelsey
> 
> ----------------
> This message and the information it contains are the proprietary and 
> confidential property of Ember Corporation and may be privileged.

---

From: Dario Tedeschi <[email-masked]>
Date: Thu, 18 Feb 2010 13:07:16 -0800

Richard and Owen,

>> This would allow (un)compression and (de)fragmentation to
I agree.

I think it should be simplified further by making datagram_offset 11bits 
long. Having datagram_offset as 8bits (forcing fragment length to a 
multiple of 8) may save 1 byte, but it also means that up to 7 bytes at 
the end of a fragment can be left unused (save 1 byte to, potentially, 
lose 7). So how about a new fragmentation header where both 
datagram_size and datagram_offset are 11bits and the datagram_tag is 
reduced to 13bits (how big does the tag need to be anyway). A new 
dispatch pattern would indicate a new fragmentation header where _offset 
and _size are relative to the compressed datagram. Something like the 
following:

                     1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 
6 7 8 9
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|1 1 1 1 1|    datagram_size    |      datagram_tag       |   
datagram_offset   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

 From an implementation point of view, bits 0 to 28 can be used as a 
pseudo tag in identifying fragments that belong to one datagram. It 
could be stored in 4 bytes with the last three bits of the last byte 
masked out, but that's just an idea for trying to be a little more 
efficient.

Regards
Dario

---

From: Jonathan Hui <[email-masked]>
Date: Thu, 18 Feb 2010 13:48:41 -0800

Hi Dario,

I support the effort to make the offset field be in units of single  
octets, rather than 8 octets.  Though we need to be careful about  
shrinking the tag field.  Earlier drafts of RFC 4944 originally made  
the tag field 10 bits, but some comments from IESG review raised  
concerns about whether that was enough.  After some discussion on the  
list, the decision was made to expand the field to 16 bits (though  
only 14 bits was necessary at the time) [1].  The key point is that  
802.15.4 may operate on different PHYs with higher bit rates so we  
should be prepared for those.

In my ideal world, we would reallocate a single fragment header type  
for simplicity.  I'm not concerned about backwards compatibility given  
that we are changing the entire HC format.  And little has been done  
in the way of multi-vendor interoperability with the existing  
formats.  In that case, we include the tag, size, and offset fields in  
every fragment while keeping the tag 16 bits and others 11 bits.  The  
header type would remain at two bits with '11'.  Eliding the offset  
field in the first fragment doesn't give much added benefit.

To me, the above is the most straight-forward approach - maintaining  
most of the existing format structure, while at the same time  
simplifying it and avoids revisiting the same IESG concerns we've had  
in the past.

An alternate approach is to pick up on existing work to redesign the  
fragmentation layer to support more efficient fragment recovery and  
out-of-order delivery, but that would require a bigger change.  I'd  
echo Pascal's request to get additional thoughts there.

[1] http://www.ietf.org/mail-archive/web/6lowpan/current/msg00588.html

--
Jonathan Hui

On Feb 18, 2010, at 1:07 PM, Dario Tedeschi wrote:

> I think it should be simplified further by making datagram_offset  
> 11bits long. Having datagram_offset as 8bits (forcing fragment  
> length to a multiple of 8) may save 1 byte, but it also means that  
> up to 7 bytes at the end of a fragment can be left unused (save 1  
> byte to, potentially, lose 7). So how about a new fragmentation  
> header where both datagram_size and datagram_offset are 11bits and  
> the datagram_tag is reduced to 13bits (how big does the tag need to  
> be anyway). A new dispatch pattern would indicate a new  
> fragmentation header where _offset and _size are relative to the  
> compressed datagram. Something like the following:
>
>                    1                   2                   3
> 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3  
> 4 5 6 7 8 9

---

From: Dario Tedeschi <[email-masked]>
Date: Thu, 18 Feb 2010 17:10:24 -0800

Hi Jonathan

Jonathan Hui wrote:

Hi Dario,

I support the effort to make the offset field be in units of single
octets, rather than 8 octets.  Though we need to be careful about
shrinking the tag field.  Earlier drafts of RFC 4944 originally made
the tag field 10 bits, but some comments from IESG review raised
concerns about whether that was enough.  After some discussion on the
list, the decision was made to expand the field to 16 bits (though only
14 bits was necessary at the time) [1].  The key point is that 802.15.4
may operate on different PHYs with higher bit rates so we should be
prepared for those.

What kind of future bit rates are expected? Isn't it also possible that
even a 16bit tag might not be enough in the future, if bit rates
increase enough. My point here, is that I think the tag size is
somewhat subjective depending on who you talk to and how much of the
future one tries to predict.

Here's a rough calculation of the time it would take for the tag value
in one node to rollover based on worst case packet sizes (i.e.
smallest) and a 1Mb/s bit rate:

1st Packet Size = 128 bytes (PHY + MAC + Frag Hdr + HC + data + FCS)

2nd Packet Size = 18 bytes (PHY + MAC{short addrs} + Frag Hdr + 1 +
FCS)

Total bytes sent for one datagram = 128 + 18 = 146

Total bits sent = 146 x 8 = 1168

Bit Rate = 1Mb/s

Time to send one datagram = 1168 / 1Mb/s =~ 1.168ms

**** Assuming datagrams are continuously sent from one source without
delays (unlikely, but lets ignore any delays for now).

Time for 16bit tag field to rollover = 65536 * 1.168ms = 77s

Time for 13bit tag field to rollover = 8192 * 1.168ms = 10s

So is a 9 second rollover at 1Mb/s not sufficient as a worst case? I
think it would be OK. Also, if we see bit-rates for PHYs increase we
may also see an increase in allowable packet sizes (one can only hope
:-).

However, if it means re-opening a heated debate or causing IESG
problems, perhaps its best to stay with a 16bit tag and avoid wasting
valuable time.

for simplicity.

I agree. An offset of 0 would indicate the first fragment, anyway
(assuming one fragment header type for all fragments and the offset
included).

I'm not concerned about backwards compatibility given
that we are changing the entire HC format.  And little has been done in
the way of multi-vendor interoperability with the existing formats.  In
that case, we include the tag, size, and offset fields in every
fragment while keeping the tag 16 bits and others 11 bits.  The header
type would remain at two bits with '11'.

Sounds plausible, except that I have two reservations:

Having a dispatch pattern of '11xxxxxx' would overlay FRAG1,
FRAGN and LOWPAN_NHC encoding in "draft...-hc-06". FRAG1 and FRAGN are
of greater concern, because there would be no way of filtering out old
fragment headers. I think in this case we'd want to remain backward
compatible.

The _size and _offset fields would no longer be octet aligned
(assuming the same field order).

Eliding the offset field in the first fragment doesn't
give much added benefit.

I agree.

--

Jonathan Hui

Richard and Owen,

I think that the best solution would be
to switch to using

be done independently.

I absolutely agree.

I agree.

I think it should be simplified further by making datagram_offset
11bits long. Having datagram_offset as 8bits (forcing fragment length
to a multiple of 8) may save 1 byte, but it also means that up to 7
bytes at the end of a fragment can be left unused (save 1 byte to,
potentially, lose 7). So how about a new fragmentation header where
both datagram_size and datagram_offset are 11bits and the datagram_tag
is reduced to 13bits (how big does the tag need to be anyway). A new
dispatch pattern would indicate a new fragmentation header where
_offset and _size are relative to the compressed datagram. Something
like the following:

1                   2                   3

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4
5 6 7 8 9

Regards

Dario

6lowpan mailing list

[email-masked]

---

From: Jonathan Hui <[email-masked]>
Date: Thu, 18 Feb 2010 19:13:27 -0800

Hi Dario,

On Feb 18, 2010, at 5:10 PM, Dario Tedeschi wrote:

>> shrinking the tag field.  Earlier drafts of RFC 4944 originally  
>> made the tag field 10 bits, but some comments from IESG review  
>> raised concerns about whether that was enough.  After some  
>> discussion on the list, the decision was made to expand the field  
>> to 16 bits (though only 14 bits was necessary at the time) [1].   
>> The key point is that 802.15.4 may operate on different PHYs with  
>> higher bit rates so we should be prepared for those.
> What kind of future bit rates are expected? Isn't it also possible  
> that even a 16bit tag might not be enough in the future, if bit  
> rates increase enough. My point here, is that I think the tag size  
> is somewhat subjective depending on who you talk to and how much of  
> the future one tries to predict.

When this WG chose to expand the tag field, we considered 2 Mbps 15.4a  
PHY - see the archived thread that I linked.

> Here's a rough calculation of the time it would take for the tag  
> value in one node to rollover based on worst case packet sizes (i.e.  
> **** Assuming datagrams are continuously sent from one source  
> without delays (unlikely, but lets ignore any delays for now).
> may also see an increase in allowable packet sizes (one can only  
> hope :-).

This WG assumed a minimum roll-over period of 60 seconds - following  
the parameters chosen by IPv6 reassembly - again, see the archived  
thread.  I could also see some scenarios where 9 seconds may not be  
enough for some low-power protocols and/or mesh-under networks.

> problems, perhaps its best to stay with a 16bit tag and avoid  
> wasting valuable time.

This is what I'm trying to avoid.

>> In my ideal world, we would reallocate a single fragment header  
>> type for simplicity.

Right.

>>   I'm not concerned about backwards compatibility given that we are  
>> changing the entire HC format.  And little has been done in the way  
>> of multi-vendor interoperability with the existing formats.  In  
>> fragment while keeping the tag 16 bits and others 11 bits.  The  
>> header type would remain at two bits with '11'.
> 	• Having a dispatch pattern of '11xxxxxx' would overlay FRAG1,  
> FRAGN and LOWPAN_NHC encoding in "draft...-hc-06". FRAG1 and FRAGN  
> are of greater concern, because there would be no way of filtering  
> out old fragment headers. I think in this case we'd want to remain  
> backward compatible.

I'm still not convinced on the *need* for backwards compatibility.   
Another option is to take over the NALP value and require any non-IPv6  
protocol operating within a 6LoWPAN network to allocate a dispatch  
value and respect the dispatch encoding.  Are there people using the  
NALP value and would they strongly oppose losing another 6 bits to  
respect the dispatch encoding in order to fit in simplified fragment  
header?

> 	• The _size and _offset fields would no longer be octet aligned  

It's easy to fix some of that by reordering the fields to: offset |  
size | tag.

Again, this path is following the philosophy of simplifying the  
existing fragmentation mechanism.  I'm still looking for feedback on  
whether people would like to see additional fragmentation mechanisms  
at the risk of opening the flood gates.

--
Jonathan Hui

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Fri, 19 Feb 2010 10:19:15 +0100

Hi Jonathan:

I agree we need to maintain the datagram tag to at least 16 bits.
Another (key for me)  reason is its potential use as a label for label
switching fragments in a route over model.

Please find below the exercise of picking your proposal of 11 for
fragments applied to the frag recovery draft. I agree with Dario that we
should reserve 11X00 to avoid confusion, at least for a while. That
could give us:

Forward fragment:

-----------------

                            1                   2                   3

        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

       |1 1 F A 1|   datagram_size     |         datagram_tag          |

       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

       |Sequence |   datagram_offset   |

       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Where F would be (reserved for) Forward ECN and A ACK requested

Reply/ACK:

----------

                           1                   2                   3

        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

       |1 1 B 1 0 0|siz|         datagram_tag          | Ack Bitmap ... 

       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Where B would be (reserved for) Backward ECN, R set if error, and size
used to encode the ack bitmap size (1 to 4 bytes).

The cool thing is that we do not need the complex bitmap encoding that
was introduced recently to we could ack up to 32 fragments. 

Reply/NACK:

-----------

                            1                   2                   3

        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

       |1 1 B 1 0 1|rsv|         datagram_tag          |  error code   |

       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

What do you think?

Pascal

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On
Behalf Of Jonathan Hui
Sent: Thursday, February 18, 2010 10:49 PM
To: Dario Tedeschi
Cc: [email-masked]
Subject: Re: [6lowpan] RFC 4944 fragmentation and future extensions

Hi Dario,

I support the effort to make the offset field be in units of single
octets, rather than 8 octets.  Though we need to be careful about
shrinking the tag field.  Earlier drafts of RFC 4944 originally made the
tag field 10 bits, but some comments from IESG review raised concerns
about whether that was enough.  After some discussion on the list, the
decision was made to expand the field to 16 bits (though only 14 bits
was necessary at the time) [1].  The key point is that

In my ideal world, we would reallocate a single fragment header type for
simplicity.  I'm not concerned about backwards compatibility given that
we are changing the entire HC format.  And little has been done in the
way of multi-vendor interoperability with the existing formats.  In that
case, we include the tag, size, and offset fields in every fragment
while keeping the tag 16 bits and others 11 bits.  The header type would
remain at two bits with '11'.  Eliding the offset field in the first
fragment doesn't give much added benefit.

To me, the above is the most straight-forward approach - maintaining
most of the existing format structure, while at the same time
simplifying it and avoids revisiting the same IESG concerns we've had in
the past.

An alternate approach is to pick up on existing work to redesign the
fragmentation layer to support more efficient fragment recovery and
out-of-order delivery, but that would require a bigger change.  I'd echo
Pascal's request to get additional thoughts there.

[1] http://www.ietf.org/mail-archive/web/6lowpan/current/msg00588.html
<http://www.ietf.org/mail-archive/web/6lowpan/current/msg00588.html> 

--

Jonathan Hui

> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-

> +-+-+-+-+-+-+

> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-

> +-+-+-+-+-+-+

> [email-masked] <mailto:[email-masked]> 

> https://www.ietf.org/mailman/listinfo/6lowpan
<https://www.ietf.org/mailman/listinfo/6lowpan> 

6lowpan mailing list

[email-masked] <mailto:[email-masked]> 

https://www.ietf.org/mailman/listinfo/6lowpan
<https://www.ietf.org/mailman/listinfo/6lowpan>

---

From: Richard Kelsey <[email-masked]>
Date: Fri, 19 Feb 2010 08:54:07 -0500

If we are going to mess with the fragment headers, and not
just change the length and offsets from uncompressed to
compressed, then my preferences would be:

  - Remove the alignment requirement for offsets.

  - Allow for some kind of ack, whether or not a packet is
    fragmented.  802.15.4 acks have their problems, so as
    long as we are adding an ack we should make it a general
    one.

  - Maintain backward compatibility.  While I am not sure
    that backwards compatibility matters a great deal for
    this, being compatible is simpler.  If nothing else, it
    makes life a lot easier for Wireshark and other tools.

I really do not see the need to make the fragment header a
byte shorter at the cost of using up a large block of
dispatch bytes.  We can't be sure we are getting
fragmentation right this time, either.

A proposal:

Fragment:

   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  | Fragment (TBD)|         datagram_tag          |A|R|  datagram_
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    size    |   datagram_offset   |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  A = ack requested
  R = reserved

  Datagram_size and datagram_offset are with respect to the
  compressed datagram.  datagram_offset is zero for the first
  fragment.

Ack request for an unfragmented packet:

   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  | Ack Req (TBD) |         datagram_tag          |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Ack:

   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |  Ack (TBD)  |M|         datagram_tag          |  
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                 [mask]                           |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  M = mask present
  
  An ack with no mask acks the entire datagram, fragmented
  or not.

                                    -Richard Kelsey

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Fri, 19 Feb 2010 15:18:10 +0100

Hi Richard:

I'm confused by 2 things.

1) why do you desire to suppress alignment? 
2) The ack is in the fragment header, so how'd you ack a non fragmented
packet?

Cheers,

Pascal

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On
Behalf Of Richard Kelsey
Sent: Friday, February 19, 2010 2:54 PM
To: Jonathan Hui
Cc: [email-masked]
Subject: Re: [6lowpan] RFC 4944 fragmentation and future extensions

If we are going to mess with the fragment headers, and not just change
the length and offsets from uncompressed to compressed, then my
preferences would be:

I really do not see the need to make the fragment header a byte shorter
at the cost of using up a large block of dispatch bytes.  We can't be
sure we are getting fragmentation right this time, either.

A proposal:

Fragment:

  A = ack requested
  R = reserved

Ack:

  M = mask present

                                    -Richard Kelsey

---

From: Richard Kelsey <[email-masked]>
Date: Fri, 19 Feb 2010 09:20:57 -0500

> Date: Fri, 19 Feb 2010 15:18:10 +0100
> From: "Pascal Thubert (pthubert)" <[email-masked]>
> 

Because requiring the fragment offset be aligned on an
eight-byte boundary adds complexity and wastes payload.

To be clearer, I meant the alignment of the fragment
boundaries, not the alignment of the offset field within the
fragment header.  I would prefer to keep the fields
themselves aligned on byte boundaries, but it is difficult
to to so with two eleven-bit fields.  They could easily
be put on four-bit (nibble) boundaries, if that is a help.

> 2) The ack is in the fragment header, so how'd you ack a
> non fragmented packet?

What do you mean by "the ack is in the fragment header"?
The ack is its own dispatch type, distinct from the
fragment dispatch.
                                   -Richard

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Fri, 19 Feb 2010 16:06:37 +0100

Hi Richard:

> Because requiring the fragment offset be aligned on an eight-byte
boundary adds complexity and wastes payload.

How, that much we agree upon. But on your example you had the
datagram_size over a DWORD barrier and I think that can be an issue.

>> 2) The ack is in the fragment header, so how'd you ack a non 

>> fragmented packet?

>The ack is its own dispatch type, distinct from the fragment dispatch.

I see.  So you'd use a new dispatch to make any packet ackable. Looks
good to me. Do you have details on problems that this solves?

As I had before, we need the capability to inform of congestion and
error. In particular propagate a NACK to a source.

Cheers,

    -Richard

---

From: Richard Kelsey <[email-masked]>
Date: Fri, 19 Feb 2010 10:38:33 -0500

> Date: Fri, 19 Feb 2010 16:06:37 +0100
> But on your example you had the datagram_size over a DWORD barrier
> and I think that can be an issue.

The alternative would be to put the length and offset first,
and the tag second:

   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  | Fragment (TBD)|A|    datagram_size    |R|                     |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |        datagram_tag           |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  A = ack requested
  R = reserved

Personally, I would prefer the tag first and not worry about
the DWORD barrier, but it isn't something I feel strongly
about.

> good to me. Do you have details on problems that this solves?

The 802.15.4 ack is basically a phy ack.  It tells you that
the bits were received, but that is all.  It doesn't tell
you that the incoming message passed any security checks or
if the receiver has space to buffer it.  The 802.15.4 ack
has no security and no source address.

A 6LoWPAN ack could be secured and would indicate that the
routing layer actually received the packet.

> error.

Doing this would raise a lot of questions that I was trying
to avoid.  Handling congestion can get quite complicated,
and seems only marginally related to the handling of any
particular message.
                                -Richard

---

From: Jonathan Hui <[email-masked]>
Date: Fri, 19 Feb 2010 07:55:17 -0800

Hi Pascal and Richard,

Since we are going down this path,  I'd like to agree on what the  
6lowpan ack/frag mechanism should provide before we get into the  
details of bits/bytes.  Let me throw out my preferences:

- Ability to ACK/NACK and 6lowpan packet, whether or not it is  
fragmented.  The NACK is really useful to explicitly indicate that the  
receiver does not currently have the buffer space to store the 6lowpan  
packet.

- Ability to carry arbitrary payloads in the ACK/NACK.  There have  
been many times in the past where being able to piggyback information  
on the ACK/NACK can be helpful.  We would probably initially define a  
SNACK option, but I'd really like to keep it general enough so that we  
aren't stuck with whatever we define immediately.

- Carry a 6lowpan packet sequence number/tag separate from the MAC  
header to assist in identifying 6lowpan fragments/packets as well as  
suppressing dups.

- Maintain backwards compatibility with the existing RFC 4944 fragment  
header.

Thoughts on the above?

--
Jonathan Hui

On Feb 19, 2010, at 7:06 AM, Pascal Thubert (pthubert) wrote:

> I see.  So you’d use a new dispatch to make any packet ackable.  
> Looks good to me. Do you have details on problems that this solves?

---

From: Jonathan Hui <[email-masked]>
Date: Fri, 19 Feb 2010 08:25:58 -0800

> has no security and no source address.

As FYI, the 802.15.4e TG is actively working on resolving these issues  
with the existing 802.15.4-2006 ACK.  Regardless, however, I still see  
benefits in a 6lowpan-layer ACK.

--
Jonathan Hui

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Fri, 19 Feb 2010 17:56:27 +0100

Hi Jonathan:

I'm with you so far. Richard?

Pascal

-----Original Message-----
From: Jonathan Hui [mailto:[email-masked]] 
Sent: Friday, February 19, 2010 4:55 PM
To: Pascal Thubert (pthubert)
Cc: Richard Kelsey; [email-masked]
Subject: Re: [6lowpan] RFC 4944 fragmentation and future extensions

Hi Pascal and Richard,

Since we are going down this path,  I'd like to agree on what the
6lowpan ack/frag mechanism should provide before we get into the details
of bits/bytes.  Let me throw out my preferences:

- Ability to carry arbitrary payloads in the ACK/NACK.  There have been
many times in the past where being able to piggyback information on the
ACK/NACK can be helpful.  We would probably initially define a SNACK
option, but I'd really like to keep it general enough so that we aren't
stuck with whatever we define immediately.

Thoughts on the above?

--
Jonathan Hui

On Feb 19, 2010, at 7:06 AM, Pascal Thubert (pthubert) wrote:

> I see.  So you'd use a new dispatch to make any packet ackable.

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 19 Feb 2010 18:01:26 +0100

On Feb 19, 2010, at 16:55, Jonathan Hui wrote:

> Since we are going down this path,

Are we?

My advice would be to read RFC 3819 before designing another L2 protocol.

Gruesse, Carsten

---

From: Jonathan Hui <[email-masked]>
Date: Fri, 19 Feb 2010 09:08:58 -0800

Hi Carsten,

> My advice would be to read RFC 3819 before designing another L2  

The issues with 802.15.4-2006 ACKs are widely known (security,  
inability to nack, etc.).  Waiting for the 802.15.4e TG to fix these  
issues (and then waiting for silicon vendors to implement those  
changes) is going to be a long process (if you've been following the  
discussions, it's already a big can of worms).  Properly supporting  
security has already opened a big can of worms w.r.t. timing  
constraints, timeline to implement in hardware, etc.  Defining an  
optional 6lowpan-layer ACK would solve many of these problems.

--
Jonathan Hui

---

From: Dario Tedeschi <[email-masked]>
Date: Fri, 19 Feb 2010 10:38:09 -0800

Hi Jonathan

Jonathan Hui wrote:
>> that even a 16bit tag might not be enough in the future, if bit rates 
>> value in one node to rollover ... --- removed for brevity --- ...
>>
OK, fair enough. 16bits it is.

>>> of multi-vendor interoperability with the existing formats.  In that 
>>> while keeping the tag 16 bits and others 11 bits.  The header type 
>>> would remain at two bits with '11'.
Certainly a possibility, if we are willing to drop NALP.

> It's easy to fix some of that by reordering the fields to: offset | 
OK, that would only align "size" and "tag". Ideally I'd rather see all 
fields aligned, but given that aligning two 11bit fields is awkward, I'd 
rather see offset aligned since that's the one field that's used more 
often. How about: tag | size | offset  (still assuming a philosophy of 
simplifying fragmentation)?

--
Dario

---

From: Richard Kelsey <[email-masked]>
Date: Fri, 19 Feb 2010 13:28:42 -0500

> From: Carsten Bormann <[email-masked]>
> Date: Fri, 19 Feb 2010 18:01:26 +0100
> 

I think we have to do something, because the RFC 4944
fragmentation does not work well if the compressed headers
extend beyond the first fragment, or if a new transport
compression is introduced.  The HC draft makes this more
important because it includes encodings for IPv6 extension
headers.

Whether this is fixed via a minimal update or a complete
redesign is another question.  I think it is worth spending
a little time on what a redesign might look like, even if
we end up leaving the fragmentation much as it is now.

                               -Richard Kelsey

---

From: Richard Kelsey <[email-masked]>
Date: Fri, 19 Feb 2010 13:36:14 -0500

> From: Jonathan Hui <[email-masked]>
> Date: Fri, 19 Feb 2010 07:55:17 -0800
>

Jonathan,

That seems like a good list of goals, as long as nothing
too complicated results.

For arbitrary payloads, would it be enough to allow the
ACK/NACK to be followed by additional LoWPAN frames?  The
piggybacked data could then be defined completely separately.

                               -Richard Kelsey

---

From: Dario Tedeschi <[email-masked]>
Date: Fri, 19 Feb 2010 11:41:25 -0800

Hi Carsten and Richard

Richard Kelsey wrote:
> redesign is another question.
I think an updated, simplified fragmentation header to support the HC 
draft would be the quickest way forward. Let's fix the main problem that 
started this thread: Namely, that compressed headers spanning multiple 
RFC4944 fragments presents huge complexities in re-assembly.

Sure, having a 6LoWPAN ACK may solve some issues, but how many use-cases 
does it cover and is it worth all the extra work at this point. Couldn't 
ACKing be added as an optional protocol some time in the future, for 
those that need it?

Regards
Dario

---

From: "Pete St. Pierre" <[email-masked]>
Date: Fri, 19 Feb 2010 11:53:09 -0800

> I think an updated, simplified fragmentation header to support the HC draft would be the quickest way forward. Let's fix the main problem that started this thread: Namely, that compressed headers spanning multiple RFC4944 fragments presents huge complexities in re-assembly.

I couldn't agree more!  I don't think we want to see 4944 completely re-whacked at this point. Let's fix the problem at hand.
> 
> Sure, having a 6LoWPAN ACK may solve some issues, but how many use-cases does it cover and is it worth all the extra work at this point. Couldn't ACKing be added as an optional protocol some time in the future, for those that need it?

Agreed.  We're getting more and more 4944 compliant implementations and should be careful before we fragment our developers (pun intended).  

IIRC, this was discussed in Stockholm and while there was some interest in continuing the work, there was definitely no consensus to re-do the entire header and deprecate 4944.

My $,02 (FWIW),
...Pete

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 20 Feb 2010 08:19:17 +0100

On Feb 19, 2010, at 20:41, Dario Tedeschi wrote:

> compressed headers spanning multiple RFC4944 fragments presents huge complexities in re-assembly

My first reaction would be "don't do that, then!".
(http://catb.org/jargon/html/D/Don-t-do-that-then-.html)

I don't think that 4944 specifies anything that lets you have the compressed header extend beyond the first frame.
I'm not sure we explicitly discussed this, but certainly I didn't see a need.

If we now do, I'd like to first learn more about this use case.

Gruesse, Carsten

---

From: Jonathan Hui <[email-masked]>
Date: Sat, 20 Feb 2010 00:42:57 -0800

On Feb 19, 2010, at 11:19 PM, Carsten Bormann wrote:

> I don't think that 4944 specifies anything that lets you have the  
> compressed header extend beyond the first frame.

I'm not convinced of this use case either and I'm fine with limiting  
use of IPHC/NHC to the first fragment.

Richard's future-compatibility case that he spawned this thread with  
is more important to consider.

--
Jonathan Hui

---

From: Richard Kelsey <[email-masked]>
Date: Sat, 20 Feb 2010 09:14:21 -0500

> Date: Sat, 20 Feb 2010 08:19:17 +0100
> 
> > compressed headers spanning multiple RFC4944 fragments
> > presents huge complexities in re-assembly
> 

Even if I don't do it, how can I be sure that no one else
will?  In the absence of any indication one way or the
other in 4944, it would be dangerous to assume that no
incoming packet will have a compressed header in the second
fragment.

> compressed header extend beyond the first frame.

4944 doesn't say anything one way or the other, as far as I
can tell.  I can't see it happening with the 4944
compression, which may be why 4944 is silent on the issue.
With the HC draft allowing compressed UDP headers to follow
routing headers it is much more of a possibility.

> I'm not sure we explicitly discussed this, but certainly I didn't see

My one concern would be source routes, which are eminently
suitable for compression and even when compressed could
conceivably extend into the second fragment.  It would be a
shame to be able to compress short source routes but not
long ones.  
                                  -Richard Kelsey

---

From: Richard Kelsey <[email-masked]>
Date: Sat, 20 Feb 2010 09:48:52 -0500

> Date: Sat, 20 Feb 2010 00:42:57 -0800
> 

The two issues are related.  If it were possible to
guarantee that no compressed header would ever extend into
the second fragment, then future compatibility would not be
a problem.  A relaying node could treat any unknown
compressed header as if it and all subsequent headers were
uncompressed.  As long as compression only occured in the
first fragment the second fragment's offset would be
correct.

Similarly, if all relayed messages could be completely
uncompressed, we could mandate that compressed headers must
never extend into the second fragment.  A relaying node
would stop compressing when it reached the end of the first
outgoing fragment.

The rub is that we can't enforce the requirement in
precisely the case where it matters, which is when a
relaying node gets a header that it doesn't know how to
uncompress.  The boundary between the first two fragments
can move a lot from one hop to another, especially with
anything like RPL's piecewise source routes in use.  If a
relay gets an unknown compressed header and has to add, say,
an eight-hop source route, what can it do?  How can it
know that it hasn't pushed part of the compressed header
into the second fragment?

The use of the compressed offsets in the fragment headers
introduces a number of complicated layering issues.  It
makes fragmentation and compression interdependent.  It
makes relaying nodes process transport headers.  It
interacts with the use of routing headers.  What benefits
does it bring that make it worth all of the headaches?
      
                             -Richard Kelsey

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 20 Feb 2010 17:14:36 +0100

On Feb 20, 2010, at 15:14, Richard Kelsey wrote:

> will?  

By dropping their packets.
(Seriously.)

> In the absence of any indication one way or the
> other in 4944,

I'm reading this text/figure in section 5 as pretty strong support:

   A LoWPAN encapsulated LOWPAN_HC1 compressed IPv6 datagram that
   requires fragmentation:

      +-----------+-------------+--------------+------------+---------+
      | Frag Type | Frag Header | HC1 Dispatch | HC1 Header | Payload |
      +-----------+-------------+--------------+------------+---------+

Apart from that, 4944 simply does not say you *can* do it.
It would have to, because it would have to define the semantics of "datagram_offset" for this specific case; the current text in 4944 clearly assumes that additional fragments only carry payload.

The new HC of course can simply make it more explicit you can't continue compressed headers into the additional fragments; that would be one WGLC comment that could come out of this discussion.

> shame to be able to compress short source routes but not

It may be worthwhile to compress source routes as part of the HC payload, not in HC itself.

But then again, I find it weird to spend work to accommodate fragmentation of routing headers (more precisely: to make that case. of splitting into multi-frame packets just for routing, more efficient -- you can always choose to not compress!).
Fragmenting for routing goes against a lot of things that 6lowpan stands for.

Gruesse, Carsten

---

From: Jonathan Hui <[email-masked]>
Date: Sat, 20 Feb 2010 10:26:29 -0800

On Feb 20, 2010, at 6:48 AM, Richard Kelsey wrote:

> The two issues are related.
> [... Richard's explanation of how the issues are related ...]

Yes, I agree they are related.  But I find it more useful to think of  
it as a forward-compatibility issue.  Note that RFC 4944 did not allow  
any kind of forward compatibility.  If you need to insert a routing  
header, the encoding of Section 10 of RFC 4944 would require expansion  
of any compressed UDP/ICMPv6/TCP headers.  Because 6lowpan-hc  
decouples the encoding for each individual header, we now have the  
chance to start considering forward compatibility for higher-layer  
headers.  If we were to continue the limitations implicitly defined by  
RFC 4944, then there would be no fundamental flaws in the  
fragmentation mechanism to talk about.

> does it bring that make it worth all of the headaches?

I think you meant to say "The use of *uncompressed* offsets".  I  
completely agree and I noted the same layering/dependency issues in an  
earlier mail.  Viewing the fragmentation payload as opaque would  
eliminate these dependencies.  Carsten suggested that we could treat  
higher-layer compression as payload of 6lowpan-hc (rather than as a  
part of), but that would eliminate any benefit from using uncompressed  
offsets.

Regardless of what we decide to do with the fragmentation header, I  
would really like to see a clean separation between fragmentation and  
header compression.  Note that saying "compressed offsets/lengths" is  
a bit of a misnomer.  It's better to say that offsets and lengths  
simply reflect the actual payload being fragmented (which may or may  
not use header compression).

--
Jonathan Hui